how computational models help explain the origins of reasoning

Abstract: Developmentalpsychology is ready to blos-

som into a modern science thatfocuses on causal mechanistic

explanations of developmentrather than just describing and clas-

sifying the skills that children show atdifferent ages. Computational modelsof cognitive development are formal

systems that track the changes in infor-mation processing taking place as abehavior is acquired. Models are general-ly implemented as psychologically con-strained computer simulations that learntasks such as reasoning, categorization, andlanguage. Their principal use is as tools forexploring mechanisms of transition (devel-opment) from one level of competence tothe next during the course of cognitive devel-

opment. They have been used to probe ques-tions such as the extent of ‘pre-programmed’ or

innate knowledge that exists in the infant mind,and how the sophistication of reasoning can

increase with age and experience.

I. Understanding theOrigins of Reasoning

The Swiss psychologist Jean Piaget was possi-bly the first man to ask how thinking emergedfrom the simple reflexes of the newborn to the

abstract logical reasoning of the adult (Piaget 1971; Boden 1995).He saw himself as an empirical philosopher whose goal it was to answer

the fundamental questions of epistemology (the study of the origins ofknowledge) through rigorous experimentation. He asked how knowledge,

especially abstract conceptual knowledge and logic-based reasoning, couldemerge from a child’s interactions with the world. Piaget produced a vast body ofwork exploring the development of concepts like Space, Time, Number, andCausality. He is widely recognized as having identified the key questions that haveset the agenda for cognitive development research over the last 70 years.

Piaget was greatly influenced in his thinking by the philosophies of Kant andBergson, but also by the Cybernetics movement of the early twentieth century.He believed that children constructed an understanding of the world throughactive engagement with the world, and that feedback on one’s actions played

a crucial role in learning and development. Unfortunately, he failed toground many of his theoretical proposals because he lacked an appro-

priate vocabulary with which to express his dynamic and mech-anistic ideas (Boden 1988). The arrival of computational

modeling has provided a suite of powerful con-ceptual tools for addressing many of

Piaget’s original ideas.

HowComputationalModels HelpExplain theOrigins ofReasoning

HowComputationalModels HelpExplain theOrigins ofReasoning

Denis Mareschal and Michael S.C. ThomasBirkbeck University of London, UK

32 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | AUGUST 2006 1556-603X/06/$20.00©2006IEEE

© DIGITALVISION & ARTVILLE

Contemporary theories of cognitive development liebroadly along two distinct (albeit related) dimensions. Oneof these is the Nativist vs. Empiricist dimension (sometimesknown as the nature-nurture debate). Radical Nativists believethat almost all knowledge is available to the infant prior toany experience. Learning only serves to fill in minor details.Radical Empiricists believe that the infant is born with pow-erful learning abilities but no prior knowledge. All knowl-edge is acquired through some form of experience with theworld. Computationally, the approaches differ in the con-straints that are placed on learning. In a cognitive computa-tional model, the researcher uses psychological empiricaldata to constrain choices about representations and trainingsets. For example, a cognitive model designed to learn gram-mar might include input representations of individual wordsand output representations of who-did-what-to-whom. Itmight then be trained on samples of child-directed speech.Psychological approaches with a nativist leaning will producemodels with tight constraints on their architectures, activa-tion dynamics, input/output representations, or learningalgorithms, so that the system can only support a restrictedset of input-output functions. Exposure to a training set justserves to push the system into one of this limited set ofstates. For example, in the Chomskian theory of languageacquisition, mere exposure to language is held to ‘trigger’the selection of the correct subset of a Universal Grammarthat is present at birth (see for example, Buttery 2004).Internal functions would be limited to tree-structures andoperations over tree-structures. The latent structure of child-directed speech would have little causal role in shaping thefinal internal function. By contrast, models of a more empiri-cist bent will have fewer constraints on the functions that canbe supported, so that the information in the training set playsa stronger role in determining which function is acquired.For example, some theories of the development of the visualsystem argue that when a fairly general self-organizing learn-ing system is exposed to natural visual scenes, the latent sta-tistical structure of these scenes is sufficient to generate manyof the kinds of representational primitives observed in thelow-level visual system, such as center-surround receptivefields (Field 1999a, b).A second dimension that distinguishescontemporary theories of cognitive development is the dis-tinction between symbolic vs. sub-symbolic representations.Those in the symbolic camp believe that cognition is bestcharacterized as a rule-governed physical symbol system. Inthis view, cognitive development consists in the modifica-tion of mental rules. By contrast, those in the sub-symboliccamp see cognition as a highly interactive dynamic system(e.g., an artificial neural network). In this system, the causalentities are continuous, distributed and cycling patterns ofactivation. Such networks do not operate as physical symbolsystems, or at best approximate them in certain narrow cir-cumstances. In this latter view, development consists in thecontinuous tuning of the underlying parameters of the cog-nitive system.

Piaget concentrated on studying development in nor-mal populations. His theory therefore aimed to character-ize the cognitive stages through which the ‘average’ childpasses. However, in contemporary psychological theory,normal cognitive development is increasingly seen withinthe context of the ways that development can go wrong oroperate sub-optimally. For example, children with geneticdisorders can exhibit uneven cognitive profiles and some-times learning disabilities, such as in Down syndrome,autism, and Williams syndrome. Apparently more circum-scribed disorders can be found that differentially impact onlanguage development or on the acquisition of reading,observed in Specific Language Impairment or dyslexia.Even within the normal population, children of the sameage can differ in their cognitive ability. At theupper end, children are viewed as gifted,while at the extreme lower end, normaldevelopment begins to resemble dis-ability. These variations in develop-ment throw into rel ief theboundary conditions that mustshape normal development(Karmiloff-Smith 1998). Inrecent years , computat ionalmodels of development haveprovided a productive tool toinvestigate how variations in theproperties of learning systems—orthe potentially enriched or impover-ished learning environments to whichthey are exposed—can help or hinder theacquisition of cognitive abilities (Thomas andKarmiloff-Smith 2003).

In the rest of this article, we present a number of mod-els to illustrate different aspects of cognitive developmentwhere computational approaches have produced materialadvances in our understanding of the origins of knowl-edge. As will become apparent in this review, most symbol-i c models have emphasized the tractabi l i ty of theknowledge representations involved in cognitive develop-ment at the expense of implementing explicit transitionmechanisms. In contrast most sub-symbolic models haveemphasized the specification of a developmental mecha-nism at the expense of the tractability of the knowledgerepresentations. In other words, to the extent that thehuman mind needs complicated and densely structuredmental representations to deliver a cognitive skill, itbecomes hard to see how these representations areacquired. The theorist is left with three ways out of thisconundrum. Either complex behavior are generated byrepresentations that are in large part innate; or we don’tyet understand the full repertoire of learning mechanismsavailable to the human mind; or we are currently overesti-mating the complexity of the representations that thehuman mind needs to generate its complex behaviors.

AUGUST 2006 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 33

34 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | AUGUST 2006

II. Why Build Computational Models of Cognitive Development

A. The Computer Modeling MethodologyComputer models are invaluable tools for transforming devel-opmental psychology from a descriptive science into a matureexplanatory science (Mareschal and Thomas in press). When aresearcher has to translate his or her underlying theory into anexplicit computer model, he or she must now specify preciselywhat is meant by the various terms. Terms such as representa-tions, symbols, and variables must have an exact definition topermit implementation. The degree of precision required toconstruct a working computer model avoids the possibility ofarguments arising from the misunderstanding of imprecise ver-bal theories. For example, ‘short-term memory’ convenientlysummarizes a cluster of human behaviors, yet it is another thingto build the persistence of information over time into a process-ing system. How much information is stored? How long is itstored for? There is no longer any room for vagueness!

Secondly, building a model that implements a theory pro-vides a means of testing the internal self-consistency of the the-ory. A theory that is in any way inconsistent or incompletewill become immediately obvious when trying to implement itas a computer program. The inconsistencies will lead to con-flict situations in which the computer program will not be ableto function. Such failures point to a need to reassess the situa-tion and to the re-evaluate the theory.

One implication of these two points is that the model canbe used to work out unexpected implications of a complextheory. Because the cognitive system operates in a highly com-plex world, with a multitude of information sources constantlyinteracting, even a simple process theory can lead to uninter-pretable behaviors. Here again, the model provides a tool forteasing apart the nature of these interactions and corroboratingor falsifying the theory. One of the earliest applications of arti-ficial neural networks to cognitive processing (McClelland andRumelhart 1981) was able to demonstrate how constrainedinteractivity could solve the following puzzle: it is easier torecognize a written letter when it is presented in the middle ofa word than in a nonsense string—but how does the readerknow that the letter is in a word or a nonsense string beforehaving recognized it?

Perhaps the main contribution made by computationalmodels of cognitive development is to provide an account ofthe representations that underlie performance on a task thatalso incorporates a mechanism for representational change.One of the greatest unanswered questions of cognitive devel-opment is the nature of the transition mechanisms that canaccount for how one level of performance is transformed intothe next level of performance at a later age. This is a difficultquestion because it involves observing how representationsevolve over time, and tracking the interactions between thedeveloping components of a complex cognitive system. Build-ing a model and observing how it evolves over time provides atangible means of doing this.

Formulating development in computational terms forcesthe theoretician to be explicit about the transitional mecha-nisms that underlie information processing. Piaget’s ownmechanistic theory provides an excellent example of why thisis necessary. He described cognitive development in terms ofthree processes: assimilation, accommodation, and equilibra-tion. Assimilation consisted in adapting or filtering incominginformation to make it more compatible with existingknowledge representations. In contrast, accommodation con-sisted in adapting one’s knowledge representations to makethem more consistent with novel information. Equilibrationwas the process by which assimilation and accommodationinteracted to cause cognitive development and can thereforebe understood as the play-off between these processes of sta-bility and change. Now, while assimilation and accommoda-tion may capture intuitive notions of what is involved incognitive development, they are too loosely defined to be ofany explanatory value. For example, could this theory predicthow many errors should be enough to trigger accommoda-tion? Several artificial neural network (or ‘connectionist’)computational models of cognitive development have soughtto address this vagueness by providing computational imple-mentations of assimilation and accommodation (e.g.,McClelland 1995; Shultz, Schmidt, Buckingham andMareschal 1995). In these models, assimilation corresponds toactivation flow through the neural network while accommo-dation corresponds to updating connection weights or thenetwork architecture to reduce output error.

III. Models of Development in InfancyInfancy is an ideal age range to begin modeling because infantbehaviors are not complicated by the presence of language andsophisticated meta-cognitive strategies. Infant abilities areclosely tied to their developing sensori-motor skills.

A. Object-Directed BehaviorsKant identified objects as a fundamental category of cogni-tion. The ability to represent hidden objects liberates infantsfrom the tyranny of direct perception. It is the first steptowards representational thought. Piaget (1954) suggestedthat infants’ progress through 6 stages on the way towardsreaching an adult level of understanding of object perma-nence at the age of 2. Many of Piaget’s original findingshave been replicated. However, changes in methodology,such as relying on where infants look rather than their abilityto manually reach for objects, have suggested that infant’sunderstanding of hidden objects is far more precocious(Mareschal 2000). These more recent studies have tended tofocus on infant competence at different ages but not on themechanisms of development from one level of competenceto the next. How can the infant’s knowledge of the perma-nence of objects improve?

There are relatively few computational models of infantobject-directed behaviors. Early models adopted a symbolicstance on the mechanisms that drive behavior and were thus


implemented in rule-based production systems (e.g., Luger,Bower, and Wishart 1983; Prazdny 1980). Unfortunately,these were basically competence models that described infantbehaviors but did not provide a mechanistic account ofdevelopment. They proposed different sets of rules todescribe behavior at different ages, but did not explain hownew rules could be acquired or how oneset of rules was transformed into anotherset of rules. More recent symbolic mod-els have turned to attention-basedaccounts of object processing in anattempt to explain infant behaviors(Simon 1998). Unfortunately, thesemodels still fail (by and large) to imple-ment any account of how development might occur.

One mechanistic learning model has implemented a parallelprocessing version of Piaget’s sensori-motor theory of infantdevelopment. Drescher (1991) tried to show how the co-ordination of intra- and inter-modal perceptual motor schemascould lead to a single unified representation of object. Percep-tual motor schemas were encoded as “context-action-result”rules and implemented in a parallel processing machine. Learn-ing consisted in using marginal probabilities to fill in contextand results slots in appropriate perceptual-motor schemas.Although this system developed an intricate network of intra-and inter-modal schemas that mimicked the infant’s sensori-motor integration, it did not develop according to the patterndescribed by Piaget.

A number of connectionist models have been proposedrelying on sub-symbolic representations. In one family ofmodels, a partially recurrent autoencoder network learns topredict the reappearance of a stationary object from behind amoving screen that temporarily hides the object (Munakata,McClelland, Johnson and Siegler 1997). Network perfor-mance is measured by taking the difference in response of thenodes coding the location of the hidden object when anobject should be revealed, and subtracting it from theresponse of the node when an object should not be revealed.An increase in this difference is interpreted as increasedknowledge of hidden objects. This model demonstrates thatthe knowledge of objects necessary to retrieve them frombehind a screen can be graded and arise incrementally thoughinteractions with an environment.

Mareschal, Plunkett and Harris (1999) described an alter-native connectionist model that is more closely tied to theneuropsychological finding that knowledge about anobject’s identity and its location are processed along separateneural pathways. This model uses a combination of modulesto implement dual-route processing. One route learns toprocess spatial-temporal information while the other routelearns to process feature information. Finally, a responsemodule recruits and co-ordinates the representations devel-oped by the other modules as and when required by aresponse task such as reaching. The specialization of the tworoutes is initially defined only in terms of different associa-

tive learning mechanisms that act on the same input.Empirical studies inspired by this developmental modelhave subsequently found strong evidence for a dissociationbetween location information and identity information inthe memory of young infants for hidden objects (Mareschaland Johnson 2003).

B. Perceptual Categorization Categorization lies at the heart of cognition because it is theprocess we relate new individual experiences to our existingknowledge. It is therefore not surprising to find that greatdeal of effort has been exerted in trying to understand theearly roots of category formation. Many infant categoriza-tion tasks rely on preferential looking or habituation tech-niques, based on the finding that infants direct moreattention to unfamiliar or unexpected stimuli (Mareschaland Quinn 2001). In a preferential looking experiment, theinfant is offered two stimuli to look at: preference for oneindicates an ability to distinguish between them. Habitua-tion relies on the fact that infants become board by the rep-etition of a sequence of identical stimuli. Re-engagementwith a new stimulus is evidence that the infant can distin-guish it from the previous items. Connectionist autoen-coder networks have been used to model the relationbetween sustained attention and the real-time constructionof mental representations in the infant (Mareschal, Frenchand Quinn 2000). The successive cycles of training in theautoencoder reflect an iterative process by which a reliableinternal representation of the visual input is established.This approach assumes that infant looking times are posi-tively correlated with the network error. That is, thegreater the error, the more novel the stimulus, because ittakes more training cycles to reduce the error. The morenovel the stimulus, the longer the looking time.

The perceptual categories formed by infants are not alwaysthe same as the corresponding adult categories. For example,when 3 to 4-month-old infants are shown a series of cat pho-tographs, they will form a category of CAT that includesnovel cats and excludes dogs (as will adults). Thus, after aseries of cats, a novel cat will not be interesting to the infantbut a novel dog will be. However, when shown a series ofdog photographs, the same infants will form a category ofDOG that includes novel dogs but also includes cats (in con-trast to adults). Many aspects of early infant perceptual catego-rizations (including this asymmetric exclusivity of CAT andDOG categories) are captured by the connectionist autoen-coder model. While adults apply top-down schemas whenrecognizing photographs of cats and dogs, both 3- to 4-

In contemporary psychological theory, normal cognitivedevelopment is increasingly seen within the context ofthe ways that development can go wrong or operatesub-optimally.


month-olds and the autoencoder networks simply process thebottom-up information in these images. Hence, their internalcategory representations are yoked to the distributional prop-erties of features in the images. The model validates the ideathat categorical representations can self-organize in a neuralsystem as a result of exposure to the familiarization exemplarsencountered within the test session itself.

This model can also be used to make new predictionsabout what kinds of categories infants will form when pre-sented with particular Cat and Dog pictures. French andcolleagues (French, Mareschal, Mermillod and Quinn 2004)generated sets of cat and dog pictures that all looked equiv-alent to adults, but which led to very different categoriza-tion behaviors in the infants. Because the infants wereattending to low-level features of the image, their responsescould be manipulated by changing the distribution of thesefeatures. This was not the case with the adults who, drivenby their high-level schemas, continued to respond to allimages in the same way.

IV. Models of Development in ChildhoodLanguage acquisition marks the end of infancy and the begin-ning of childhood. Reasoning and conceptual development arethe hallmark of cognitive development in childhood. Themodels in this section all focus on some aspect of reasoningdevelopment. We begin by reviewing models that haveexplicitly tried to implement Piagetian ideas, that is, develop-ment that passes through a sequence of stages of increasinglysophisticated reasoning. This is followed by a review of workthat breaks away from the Piagetian tradition.

A. Modeling Piagetian Stage DevelopmentThere have been several attempts to explain the apparentstage-like growth of competence in children in terms of self-organization in dynamic systems, competition between cogni-tive growers, and bifurcation theory (e.g., van der Maas andMolenaar 1992; van Geert 1998). However such accountshave tended to rely only on mathematical descriptions that areeither not implemented in running computer models or notgrounded in measurable information processing components ofthe cognitive system.

Researchers trying to implement Piagetian notions of devel-opment have applied their models to simulating children’s per-formance on key tasks. One such task is known as conservation,where children learn that certain properties of objects are pre-served through transformations while others are not. Thus, thevolume of water is not altered by transferring it between differ-ent shaped jugs. Several models have applied themselves to the

process of learning these invariant properties under transforma-tion (Klahr and Wallace 1976; Richardson, Forrester et al.,2006; Shultz 1998). Another such task is the seriation (or sorting)task. Piaget found that children’s ability to order a set of sticksaccording to length developed through a number of stages. In afirst stage, children were unable to sort the sticks. In a secondstage, they were able to apply local ordering relations but could

not extend the order to the set as a whole.In the third stage, they were able to sortthe set of sticks, but only by applying acostly trial and error strategy. Finally, inthe fourth stage, children were able to sortthe set quickly and efficiently by applyinga systematic selection strategy.

Young (1976) approached this task from an informationprocessing perspective. He carried out detailed analyses of theactions children carried out at different ages when sortingblocks. Based on the results of protocol analyses, he devel-oped a rule-based production system that captured children’sperformance at each stage of development. Progress from onestage to the next was modeled by the (hypothesized) modifi-cation of the rules. Although this model provided a good fitto children’s behavior at individual stages, the model doesnot include a working account of how those rules are modi-fied. By contrast, a more recent connectionist model of theseriation task explicitly demonstrates how development couldoccur (Mareschal and Shultz 1999). In this model, develop-ment consists in the gradual tuning of connection weights asthe model is exposed to stick-sorting problems of varyingcomplexity. As it learns, the model exhibits a gradual exten-sion of knowledge about small sets to larger sets. It not onlycaptures the stage progression described by Piaget, but it alsocaptures the variability in sorting behaviors observed bothwithin and between different children.

B. Beyond Piaget: The Balance-Scale TaskA recent benchmark of cognitive development is the bal-ance-scale task. This was first developed by Inhelder andPiaget in the 1950s and later significantly extended byRobert Siegler. Siegler (1976) explored children’s develop-ing abilities to reason about a balance scales (like a scaled-down version of a see-saw). In these problems, childrenwere presented with a symmetric balance scale with 5equally spaced pegs on either side of the fulcrum. Differentnumbers of weights were then placed on pegs to the leftand the right of the fulcrum and children were asked topredict whether the balance scale would tip to the left, tothe right, or remain balanced.

Children show an increasingly sophisticated ability to rea-son about these problems with increasing age. Seigler (1976)demonstrated that children’s strategies at different ages couldbe characterized by different rules. Rule 1 children, forinstance, relied only on a dominant dimension (weight) topredict which side the balance scale would tip. Thesechildren would predict that the side with the most weights

Computer models are invaluable tools for transformingdevelopmental psychology from a descriptive science intoa mature explanatory science.


would be the side that the balance scale would tip. Rule 2children would apply the same rule as the proceeding chil-dren, but had an additional rule stating that if the number ofweights was equal on both side, the side with the greatestdistance would predict the side on which the balance scalewould tip. Rule 3 children behaved like the Rule 2 childrenwith the exception that if the weight and distance cues pro-vided conflicting answers they wouldguess which side went down. Finally,Rule 4 children had a set of rules thateffectively computed the torque on bothsides of the balance scale (the number ofweights multiplied by the distance fromthe fulcrum) and chose the side with thegreatest torque. Seigler suggested that this knowledge wasrepresented in the form of a growing decision tree. Klahr(1992) pointed out the equivalence of this representationalformat to rules—a format consistent with production systemmodels of cognitive development.

Both connectionist (McClelland 1995; Richardson,Baughman et al., 2006; Shultz, Mareschal and Schmidt1994) and decision-tree approaches (Schmidt and Ling1997) have subsequently been applied to capturing thedevelopmental stages of the balance scale task have beenproposed. The connectionist models construe the problemas one of integrating information from two sources. Theycapture stage development in terms of continuous gradualweight changes and/or recruitment of additional computa-tional resources (internal units) to advance learning. Anassumption of these models is that children have greaterexperience with weight comparisons than distance compar-isons. The decision tree model uses the C4.5 tree-inducingalgorithm. Children were hypothesized to have an increas-ing memory capacity and to care increasingly about thedetailed correctness of their answers. While this latter modelcaptures the main features of children’s performance, thedecision trees it developed did not map onto those pro-posed by Siegler to reflect children’s knowledge at differentages. Simulations of children’s performance on the balancescale task bring into relief the distinction between symbolic,rule-based models and sub-symbolic, network-based models,even when it is agreed that the behavior itself can bedescribed as rule following.

C. Modeling of the Development of ReasoningMany of the successful developmental models described aboveare connectionist models. Such models process informationbased on the surface similarity between different exemplars.However, there are cases when children’s (and indeed adults’)reasoning does not follow surface similarity. Analogical reason-ing, for example, requires the child to distance his or herselffrom the surface similarity between the target and vehicledomains. Solving a problem such as “A cat to a kitten is like adog to a ?” probes for knowledge of abstract relations like off-spring. This contrasts with the example of infant categorization,

where relationships between cats and dogs depending on thesimilarity of visual features. For analogical reasoning problems,children move from basing their analogies on surface similaritybetween the two domains (such as color) to structural similarity(such as the function of an object) between the ages of 6 and 9years. On the face of it, this reliance on structural similarity isdifficult for connectionist systems to capture, given that the

models we have described are driven by learned associationsbetween surface features.

Gentner and colleagues (Gentner et al., 1995) have sug-gested that adults and children solve analogical problems bycomparing mental representations via a structure-mappingprocess of alignment of conceptual representations. A struc-turally consistent match conforms to a one-to-one mappingconstraint between the domains. For our example, the rela-tion offspring derived from the cat-kitten comparison iscombined with dog to access the structured representation<dog has offspring puppy>. The process is implemented inthe Structural Mapping Engine (SME) model. This SME isused to model the relational shift in children’s analogicalreasoning in terms of increased domain knowledge. Astheir knowledge of domain relations increases and conceptslike offspring are acquired, so children’s relational represen-tations within a domain become richer and deeper. Thisincreases the likelihood that their comparisons will focuson matching relations rather than surface features. In thisview, what develops between 6 and 9 years is only knowl-edge and not processing.

Recent attempts to explain the origins of analogical reason-ing in terms of connectionist neural network properties havede-emphasized the importance of structural alignment in ana-logical completion. Leech and colleagues (e.g., Leech,Mareschal and Cooper 2003) have suggested that, at least inyoung children, a form of semantic priming in a large semanticnetwork can best explain analogical completion of the sortdescribed above.

Shrager and Seigler (1998) presented a model of strategychoice, with the intention of providing an account of therange and variability of strategies observed in young children’sproblem solving. Their model addressed the strategies used bychildren when adding integers. The strategies are explicitlyrepresented in terms of rules. Strategy choice is probabilistic.The probability of retrieving and executing a strategy dependson the previous association of that strategy with an outcome inconjunction with considerations of cost and efficiency. Thestrategy pool evolves according to a Darwinian procedure inwhich in frequently used strategies die off and new strategiesenter the pool via random perturbation of existing strategies.

Researchers have begun to use ‘synthetic brain imaging’as a way to observe the fluid dynamics of damage andrecovery in artificial neural networks.


V. Understanding Information Processing in the SystemOne drawback of simulations of complex systems is that theinteractions underlying the model’s overt behavior can beopaque to the modeler, and therefore compromise theory

development. Models that simulate human behavior but arethemselves impenetrable are of no use to the psychologist. Insymbolic, rule-based models, the explicit rules that drive behav-ior offer transparent explanations. However, sub-symbolicmodels offer a greater challenge and motivate new methods forunderstanding system performance. For example, researchershave begun to use ‘synthetic brain imaging’ as a way toobserve the fluid dynamics of damage and recovery in artificialneural networks (e.g., Cangelosi and Parisi 2004). Figures 1and 2 exemplify this approach, using the simulations ofThomas and Karmiloff-Smith (2002) that explored the conse-quences of early brain damage. Those researchers used a partic-ular aspect of language development, the English past tense, asa test domain. The English past tense has a dual nature, includeboth regular (talk-talked) and irregular (sing-sang, go-went,

hit-hit) past tense forms. Some researchers argue that the cog-nitive system uses different mechanisms to learn each type ofverb (Pinker 1991) and even that there are separate areas in thebrain for each mechanism (Tyler, Marslen-Wilson, and Sta-

matakis 2005). Other researchers arguethat sub-symbolic connectionist systemscan acquire both regular and irregularmappings in the same network. Thomasand Karmiloff-Smith (2002) demonstrat-ed that in a dual-route network, emer-gent specialization of function couldoccur so that across development, oneroute came to specialize in processingregular verbs while the other processed

irregular verbs. Specialization is driven by the greater complex-ity of irregular mappings and the fact that only one route inthis model included the hidden units that are necessary to learnlinearly inseparable irregular mappings.

Figure 1 shows a schematic of the architecture of thisdual-route backpropagation network. This is a standardfeed-forward connectionist neural network that is trained toassociated the stem of a verb with its English past tense. Fig-ure 2 (left column) uses the synthetic brain-imaging tech-nique to demonstrate the involvement of each of the tworoutes in producing the regular past tense across develop-ment. The colors depict how hard each route is driven bythe input. The other two columns reveal what happens inthe network when it experiences damage to either the left orright route prior to training.

The results highlight the dynamicsof development and compensation.Under conditions of normal develop-ment, there is partial specialization offunction to different components ofthe system (e.g., specialization in theleft or right brain hemispheres by latechildhood). However, when one com-ponent is damaged early in develop-ment by lesioning a large proportion ofthe connections, the other componentis able to compensate to some extent,taking on the overall function. Never-theless, residual resources in the dam-aged component may still be recruitedwhere possible.

There is some evidence for this inrecovery after early child brain damage(Liégeois et al., 2004). After early unilat-eral brain damage, children are delayedin their development but usually recoverto fall within the normal cognitive rangeby adolescence. However, there mayremain subtle deficits in behavior thatdepend on the original side of damage,an effect also exhibited by the dual-route

FIGURE 1 Architecture of an associative connectionist network learning the English past tenseproblem, used to study the dynamics of damage and recovery in developing cognitive systems(Thomas and Karmiloff-Smith 2002).

"T A L K""S I N G"

"T A L K ED""S A N G"

The nature-nurture debate has thus been rephrased intoa question of how detailed the initial computationalconstraints must be in the cognitive system, given the latentstructure that we now know is present in the physical andsocial environment to which infants are exposed.


model. Interestingly, these hemisphere-dependent deficits aremore prominent in the development of visuo-spatial process-ing than in language processing (Bates and Roe 2001; Stiles2005). The fluid compensation exhibited by children afterearly focal brain damage and captured inthis model is less evident in adults, wherethere appears to be reduced scope forreorganization. A related avenue of com-putational modeling work has thereforebegun to explore the factors that explainwhy the plasticity of the cognitive systemappears to reduce with age (Thomas andJohnson 2006).

VI. Challenges to Current Models of Cognitive DevelopmentWe have seen how a shift to more formal, computationalapproaches to cognitive development has forced psychologiststo confront some of the hard theoretical problems, and in par-ticular, the nature of transitional mechanisms. Models differ intheir appeal to symbolic or sub-symbolic forms of computation.Symbolic computation offers a better characterization of theabstract reasoning skills observed in older children and adultsand of, for example, the complexities of syntaxin language. However, models relying on rule-based formats have continued to struggle to elu-cidate the process by which more complex rulesare acquired. The sub-symbolic approach makesdevelopment seem more straightforward, since itinvolves the tuning of continuously valued para-meters through experience. Yet its domains ofgreatest success tend to be those involving theassociation of features and pattern recognitionskills. The reconciliation of these two approach-es is a challenge that lies ahead.

The nature of representation is tied up withtwo other long-standing debates. How muchof our cognitive system is genetically specified,and how much is a product of our cultures andenvironments? Modeling has not settled thisissue but offers tools to construe what ‘geneti-cally specified’ might mean. In the context ofcapturing behavioral deficits in genetic devel-opmental disorders, for instance, it correspondsto computational constraints that affect the tra-jectories of learning. The nature-nurturedebate has thus been rephrased into a questionof how detailed the initial computational con-straints must be in the cognitive system, giventhe latent structure that we now know is pre-sent in the physical and social environment towhich infants are exposed. These days, thefocus is on exploring how the process ofdevelopment could happen given a particularlearning system and a particular training set.

Finally, while computer models allow us to formulatequestions about what can possibly cause cognitive develop-ment (e.g., changes in processing capacity, processing speed,knowledge, and strategy choice) more constraints are

required to identify the actual mechanisms involved in chil-dren’s cognitive development. Recent advances in neu-roimaging techniques have allowed us to place greaterconstraints on how information is processed in the brain.Many of the models above make little use of these con-straints and, in the future, such constraints should be incor-porated in any functional models of development(Westermann, Sirois, Shultz, and Mareschal 2006). Further-more, cognitive development does not occur in a social vac-uum. Vygotsky (1978) has emphasized the role of socialinteractions in cognitive development. Society provides a

FIGURE 2 The use of ‘synthetic brain imaging’ to explore the involvement of each routein processing regular verbs across development under normal conditions (column A),and after initial unilateral damage to the left route (column B) or right route (columnC). Damage corresponded to lesioning 80% of connections. Colors demonstrate howhard each route is driven by the input (dark blue = least, yellow = most). Formally, val-ues correspond to the product of activations along each weight and the magnitude ofthe weight, summed for each unit. Excitation and inhibition are therefore treated equiv-alently; they are also indistinguishable in human brain imaging techniques.

Developm

ent

(a) (b) (c)

Formal systems offer the ideal framework, and indeedperhaps the only viable method, with which to capturethe multiple influences that shape the genesis of thehuman mind.


kind of cognitive scaffolding that nurtures and aids thechild’s cognitive development by actively selecting and fil-tering the type of problems the child is faced with at anyage. Future models will need to consider these constraints toreflect the child’s learning environment more accurately.Indeed, significant steps are being made towards developingmodels that incorporate constraints from multiple levels ofdescription . . . be it at the cellular level the functional brainsystem level, the cognitive level or the social level of devel-opment (Mareschal et al, in press). Formal systems offer theideal framework, and indeed perhaps the only viablemethod, with which to capture the multiple influences thatshape the genesis of the human mind.

AcknowledgmentsThis research was supported by European Commission Frame-work 6 grant 516542-NEST, and by UK MRC grant(G0300188) awarded to MT.

References[1] E. Bates and K. Roe, “Language development in children with unilateral brain injury,” InC.A. Nelson and M. Luciana (Eds.), Handbook of developmental cognitive neuroscience, Cambridge,MA: MIT Press, pp. 281–307, 2001. [2] M.A. Boden, Artificial Intelligence in Psychology. Cambridge, MA: MIT: Press, 1988. [3] M.A. Boden, Piaget (2nd edition), Modern Masters, Fontana Press, 1995. [4] P. Buttery, “A quantitative evaluation of naturalistic models of language acquisition: Theefficiency of the triggering learning algorithm compared to a categorical grammar learning,” InW. Sakas (Ed.), Proceedings of the COLING 2004 Workshop “Psychocomputational Models ofHuman Language Acquisition” Geneva: COLING, pp. 1–8, 2004. [5] A. Cangelosi and D. Parisi, “The processing of verbs and nouns in neural networks:Insights from synthetic brain imaging,” Brain and Language, vol. 89, no. 2, pp. 401–408, 2004. [6] G.L. Drescher, Made-up Minds. A Constructivist Approach to Artificial Intelligence, MIT Press,1991.[7] D.J. Field, “What is the goal of sensory coding?” Neural Computation, vol. 6, pp. 559–601,1999a. [8] D.J. Field, “Wavelets, vision and the statistics of natural scenes.” Philosophical Transactions ofthe Royal Society of London A, vol. 357, pp. 2527–2542, 1999b. [9] J. Fodor, “Fixation of belief and concept acquisition,” In: Piatelli-Palmarini, M (ed.) Lan-guage and learning: The debate between Chomsky and Piaget, Harvard University Press, pp. 143–149,1980. [10] R.M. French, D. Mareschal, M. Mermillod, and P. Quinn, “The role of bottom-up pro-cessing in perceptual categorization by 3- to 4-month old infants: Simulations and data,” Jour-nal of Experimental Psychology: General, vol. 133, 382–397, 2004. [11] D. Gentner, M.J. Rattermann, A.B. Markman, and L. Kotovsky, “Two forces in thedevelopment of relational similarity,” In T.J. Simon and G.S. Halford (Eds.), Developing cogni-tive competence: New approaches to process modeling, Hillsdale, NJ: LEA. pp. 263–313, 1995. [12] A. Karmiloff-Smith, “Development itself is the key to understanding developmental disor-ders,” Trends in Cognitive Sciences, vol. 2, 389–398, 1998.[13] D. Klahr and J.G. Wallace, Cognitive development: An information processing view, Hillsdale,NJ: Erlbaum, 1976. [14] D. Klahr, “Information processing approaches to cognitive development,” In BornsteinMH and Lamb ME (Eds.), Developmental psychology: An advanced textbook (3rd ed.), Hillsdale,NJ: Lawrence Erlbaum Associates, pp. 273–335, 1992. [15] R. Leech, D. Mareschal, and R. Cooper, “A connectionist account of analogical develop-ment,” In R. Altman and D. Kirsh (eds.) Proceedings of the twenty-fifth annual conference of the Cog-nitive Science Society, London: LEA, pp. 710–715, 2003. [16] F. Liégeois, A. Connelly, J. Helen Cross, S.G. Boyd, D.G. Gadian, F. Vargha-Khadem,and T. Baldeweg, “Language reorganization in children with early-onset lesions of the lefthemisphere: An fMRI study,” Brain, vol. 127, no. 6, pp. 1229–1236, 2004. [17] G.F. Luger, T.G.R. Bower, and J.G. Wishart, “A model of the development of the earlyobject concept,” Perception, vol. 12, pp. 21–34, 1983. [18] D. Mareschal, “Infant object knowledge: Current trends and controversies” Trends in Cog-nitive Science, vol. 4, pp. 408–416, 2000. [19] D. Mareschal, R.M. French, P. Quinn, “A connectionist account of asymmetric categorylearning in infancy,” Developmental Psychology, vol. 36, pp. 635–645, 2000.[20] D. Mareschal and M.H. Johnson, “The “What” and “Where” of infant object representa-tions,” Cognition, vol. 88, pp. 259–276, 2003.

[21] D. Mareschal, M.H. Johnson, S. Sirois, M. Spratling, M. Thomas, and G. Westermann,Neuroconstuctivism Vol. 1: How the brain constructs cognition, Oxford UK: OUP, in press, 2006. [22] D. Mareschal, K. Plunkett, and P. Harris, “A computational and neuropsychologicalaccount of object-oriented behaviors in infancy,” Developmental Science, vol. 2, pp. 306–317,1999. [23] D. Mareschal and P.C. Quinn, “Categorization in infancy,” Trends in Cognitive Science,vol. 5, pp. 443–450, 2001. [24] D. Mareschal and T.R. Shultz, “Children’s seriation: A connectionist approach,” Connec-tion Science, vol. 11, pp. 153–188, 1999. [25] D. Mareschal and M.S.C. Thomas, “What are developmental psychologists looking for ina model?” IEEE Transactions on Evolutionary Computation, Special Issue on Autonomous MentalDevelopment, in press. [26] J.L. McClelland, A connectionist perspective on knowledge and development, In T.Simon and G. Halford (Eds.), Developing Cognitive Competence: New Approaches to Process Model-ing, Hillsdale, NJ: Erlbaum, pp. 157–204, 1995. [27] J.L. McClelland and D.E. Rumelhart, “An interactive-activation model of context effectsin letter perception: Part 1, an account of basic findings,” Psychological Review, vol. 88,pp. 375–407, 1981. [28] Y. Munakata, J.L. McClelland, M.H. Johnson, and R.S. Siegler, “Rethinking infantknowledge: Towards an adaptive process account of successes and failures in object perma-nence tasks,” Psychological Review, vol. 104, pp. 686–713, 1997.[29] J. Piaget, Psychology and epistemology: Towards a theory of knowledge. Middlesex, UK: PenguinBooks, 1971. [30] J. Piaget, The Construction of Reality in the Child. New York: Basic Books, 1954. [31] S. Pinker, “Rules of language,” Science, vol. 253, pp. 530–535, 1991. [32] D.C. Plaut, J.L. McClelland, M.S. Seidenberg, and K.E. Patterson, “Understanding nor-mal and impaired word reading: Computational principles in quasi-regular domains,” Psycholog-ical Review, vol. 103, pp. 56–115, 1996. [33] S. Prazdny, “A computational study of a period of infant object-concept development,”Perception, vol. 9, pp. 125–150, 1980. [34] F.M. Richardson, F.D. Baughman, N.A. Forrester, and M.S.C. Thomas, “Computationalmodeling of variability in the balance-scale task,” Proceedings of the 7th International Conference ofCognitive Modeling, Trieste, Italy: Edizioni Goliardiche, pp. 256–261, 2006. [35] F.M. Richardson, N.A. Forrester, F.D. Baughman, and M.S.C. Thomas, “Computationalmodeling of variability in the conservation task,” In Proceedings of the 28th Annual Conference ofthe Cognitive Science Society, Vancouver, BC, Canada, July 26–29, 2006. [36] B. Rogoff, Apprenticeship in thinking. Oxford, UK: Oxford University Press, 1990. [37] W.C. Schmidt and C.X. Ling, “A decision-tree model of balance scale development,”Machine Learning, vol. 18, pp. 1–30, 1996. [38] J. Shrager and R.S. Siegler, “SCADS: A model of children’s strategy choices and strategydiscoveries,” Psychological Science, vol. 9, pp. 405–410, 1998. [39] T.R. Shultz, “A computational analysis of conservation,” Developmental Science, vol. 1,pp. 103–126, 1998.[40] T.R. Shultz, D. Mareschal, and W.C. Schmidt, “Modeling cognitive development onbalance scale phenomena,” Machine Learning, vol. 16, pp. 59–88, 1994. [41] T.R. Shultz, W.C. Schmidt, D. Buckingham, and D. Mareschal, Modeling cognitivedevelopment with a generative connectionist algorithm, In T. Simon and G. Halford (Eds.),Developing Cognitive Competence: New Approaches to Process Modeling, Hillsdale, NJ: Erlbaum,pp. 347–362, 1995. [42] R.S. Siegler, “Three aspects of cognitive development,” Cognitive Psychology, vol. 8,pp. 481–520, 1976.[43] T.J. Simon, “Computational evidence for the foundations of numerical competence,”Developmental Science, vol. 1, pp. 71–78, 1998. [44] J. Stiles, J. Reilly, B. Paul, and P. Moses, “Cognitive development following early braininjury: Evidence for neural adaptation,” Trends in Cognitive Sciences, vol. 19, no. 3, pp.136–143. 2005. [45] M.S.C. Thomas and M.H. Johnson, “The computational modeling of sensitive periods,”Developmental Psychobiology, vol. 48, pp. 337–344, 2006. [46] M.S.C. Thomas and A. Karmiloff-Smith, “Are developmental disorders like cases of adultbrain damage? Implications from connectionist modeling,” Behavioral and Brain Sciences, vol. 25,no. 6, pp. 727–780, 2002. [47] M.S.C. Thomas and A. Karmiloff-Smith, “Connectionist models of development, devel-opmental disorders and individual differences,” In R.J. Sternberg, J. Lautrey, and T. Lubart(Eds.), Models of Intelligence: International Perspectives, American Psychological Association.pp. 133–150, 2003. [48] L.K. Tyler, W.D. Marslen-Wilson, and E.A. Stamatakis, “Differentiating lexical form,meaning and structure in the neural language system,” Proceedings of the National Academy of Sci-ences, vol. 102, no. 23, pp. 8375–8380, 2005. [49] H. Van der Maas and P. Molenaar, “A catastrophe-theoretical approach to cognitivedevelopment,” Psychological Review, vol. 99, pp. 395–417, 1992. [50] P. van Geert, “A dynamic systems model of basic developmental mechanisms: Piaget,Vygotsky, and beyond,” Psychological Review, vol. 105, pp. 634–677, 1998. [51] L.S. Vygotsky, Mind in Society: The development of higher psychological processes. Cambridge,MA: Harvard University Press, 1978. [52] G. Westermann, S. Sirois, T.R. Shultz and D. Mareschal, “Modeling developmental cog-nitive neuroscience,” Trends in Cognitive Sciences, vol. 10, pp. 227–232, 2006. [53] R. Young, Seriation by children: An artificial intelligence analysis of a Piagetian task. Basel:Birkhauser 1976.

how computational models help explain the origins of reasoning

Documents