How Cognition Could Be ComputingSemiotic Systems, Computers, & the Mind

William J. Rapaport

Department of Computer Science & Engineering,Department of Philosophy, Department of Linguistics,

and Center for Cognitive Science

rapaport@buffalo.eduhttp://www.cse.buffalo.edu/~rapaport

Summary

• Computationalism = cognition is computable.

– Mental processes can be the result of algorithmic procedures…

– …that can be affected by emotions/attitudes/individual histories.

• Computers that implement these (cognitive) procedures really exhibit those mental processes.– They are “semiotic” (= sign-using) systems.

– They really think.

• “Syntactic semantics” explains how all this is possible.

What Is “Computationalism”?• “Computationalism” =? cognition is computation

– Hobbes, McCulloch/Pitts, Putnam, Fodor, Pylyshyn, …– interesting, worth exploring, possibly true

– BUT:• Not what “computational”/“computable”

usually mean!• What should “computationalism” be?• Must preserve crucial insight:

– cognition is explainable via mathematical theory of computation

“Computable”

• Task / goal / field of study G is computable iff

algorithm(s)formal for G

The Proper Treatment of Computationalism

• Computationalism ≠ Cognition is computation

The Proper Treatment of Computationalism

• Computationalism = Cognition is computable– i.e., algorithm(s) that compute cognitive functions

a) Working assumption of computational cognitive science:• All cognition is computable

b) Basic research question of computational cognitive science:• How much of cognition is computable?

– …

Proper Treatment of Computationalism

c) Implementational implication(multiple realization):

• If cognition is computable, then: anything that implements cognitive computations would be cognitive (would really think)

• even if humans don’t do it that way!– Turing:

brain might be analog, but digital computer can still pass TT

– Piccinini: neural spike trains are not representable as digit strings;

not computational / brain does not compute BUT:

cog. functions whose O/P they produce are computable

: human cognition is computable but not computed

II. Syntactic Semanticsas a theory underlying computationalism

1. Cognition is internal• Cognitive agents have direct access

only to internal representatives of external objects

2. Semantics is syntactic• Words, meanings, & semantic relations between them

are all syntactic items

3. Understanding is recursive• Recursive Case:

– We understand one thing in terms of another that must already be understood;

• Base Case:1. We understand something in terms of itself

(syntactic understanding)

Syntactic Semantics

1. Internalism:Cognitive agents have direct access only to internal representatives of external objects

– A cognitive agent understands the world by “pushing the world into the mind” (Jackendoff 2002)

– “Output of sensory transducers is the only contact the cognitive system ever has with the environment” (Pylyshyn 1985)

Both words & their meanings (including external objects) are represented internally in a single LOT

• Humans: biological neural network

• Computers:– artificial neural network

– symbolic knowledge-representation & reasoning system

Syntactic Semantics

2. (Internalism ) Syntacticism Words, meanings, & semantic relations between them are all syntactic• syntax ⊋ grammar

Syntactic Semantics

2. (Internalism ) Syntacticism Words, meanings, & semantic relations between them are all syntactic• syntax = study of relations among members of a single set

– set of signs / uninterpreted marks / neuron firings / …

• semantics = study of relations between members of two sets 2. set of signs / marks / neuron firings / …3. & set of (external) meanings / … (with its own syntax!)

3. “Pushing” meanings into same set as symbols for them allows semantics to be done syntactically• turns semantic relations between 2 sets (internal signs, external meanings)

into relations among the marks of a single (internal) LOT/ syntax• e.g.: truth tables & formal semantics are both syntactic• e.g.: neuron firings representing both signs & external meanings

4. Symbol-manipulating computers can do semantics by doing syntax

• •SYNDOM Syntax

• •SYNDOM Syntax

• •SYNDOM • •

SEMDOM

Semantics

• •SYNDOM Syntax

• •SYNDOM • •

SEMDOM

Semantics

• • • •

Syntacticsemantics

Syntactic Semantics

3. Understanding must be understood recursively:– Recursive cases:

• We understand a syntactic domain (SYN-1) indirectly by interpreting it in terms of a semantic domain (SEM-1)– e.g.) understanding relevance logic in terms of Routley-Meyer ternary relation on points.

– but SEM-1 must be antecedently understood

• SEM-1 can be understood by considering it as a syntactic domain SYN-2 interpreted in terms of yet another semantic domain– e.g.) understanding RM ternary relation in terms of situation semantics

– which also must be antecedently understood, etc.

– Base case:• A domain that is understood directly (i.e., not “antecedently”)

– in terms of itself (in terms of relations among its symbols)– i.e., syntactically & holistically

III. Rapaport’s Thesis

Syntax suffices for semantic cognition cognition is computable& computers are capable of thinking

James H. Fetzer’s Thesis

• It doesn’t,– it isn’t,

& they aren’t

Fetzer’s Thesis

• Computers differ from cognitive agents in 3 ways:1. statically (symbolically)

2. dynamically (algorithmically)

3. affectively (emotionally)

• Simulation is not the real thing

Fetzer’s Static Difference

ARGUMENT 1: Computers are mark-manipulating systems, minds are not. Premise 1: Computers manipulate marks on the basis of their size, shapes, and relative locations.

Premise 2: (a) These shapes, sizes, and relative locations exert causal influence upon computers

but (b) do not stand for anything for those systems.

Premise 3: Minds operate by utilizing signs that stand for other things in some respect or other for them as sign-using (or “semiotic”) systems. __________________________________________________________________

Conclusion 1: Computers are not semiotic (or sign-using) systems. ___________________________________________________________________

Conclusion 2: Computers are not the possessors of minds.

Figure 10. The Static Difference

Fetzer |- Computers Are Not Semiotic Systems

1. In a “semiotic system” (e.g., a mind):– something (S) is a sign of something (x) for somebody (z)

a) x “grounds” sign S

b) x “is an interpretant w.r.t. a context” to sign-user z

c) S is in a “causation” relation with z

Fetzer |- Computers Are Not Semiotic Systems

2. In a computer (I/O) system:a) input i (playing role of sign S)

is in a “causation” relation with computer c (playing role of sign-user z)

b) output o (playing role of thing x) is in an “interpretant” relation with computer c

c) BUT: No “grounding” relation between i & o

Fetzer |- Computers Are Not Semiotic Systems

3. Computers only have causal relationships, no mediation between I/P & O/P (?!)

4. But semiotic systems require such mediation• Peirce:

interpretant is “mediately determined by” the sign– [ “interpretant” is really the sign-user’s mental concept

of the thing x (!!) ]

5. Computers are not semiotic systems6. But minds are.7. Minds are not computers

& computers can’t be minds.

Three Arguments against Static Difference

A. Incardona |- Computers are semiotic systems!

1. X is a semiotic system iff X carries out a process that mediates between a sign & its interpretant

– Semiotic systems interpret signs

2. Algorithms describe processesthat mediate between I/Ps & O/Ps

– An algorithm’s O/P is an interpretation of its I/P– Algorithms ground the I/O relation

3. Computers are algorithm machines

Computers are semiotic systems

Three arguments against the Static Difference

B. Argument that computers are semiotic systems from embedding in the world:

– Fetzer’s (counter?)example:• “A red light at an intersection stands for

applying the brakes and coming to a complete halt, only proceeding when the light turns green, for those who know ‘the rules of the road’.”

– Can such a red light stand for applying the brakes, etc., for a computer?• It could, if the computer “knows the rules of the road”

B. But a computer can “know” those rules…• if it has those rules stored in a knowledge base• and if it uses those rules to drive a vehicle

– cf. 2007 DARPA Urban Grand Challenge» Parisien & Thagard 2008, “Robosemantics:

How Stanley Represents the World”, Minds & Machines 18

Three Arguments against the Static Difference

C. Goldfain |- Computer’s marks stand for something for it

• Does a calculator that computes GCDs understand them?– Fetzer & Rapaport: No

• Could a computer that computes GCDs understand them?– Fetzer: No

– Goldfain & Rapaport: Yes, it could…

as long as it had enough background / contextual / supporting info a computer with a full-blown theory of math

at the level of an algebra student learning GCDscould understand GCDs as well as the student

Summary: No “Static Differences”

• Both computers & minds manipulate marks

• The marks can “stand for something” for both computers & minds

• Computers (and minds) are “semiotic systems”

• Computers can possess minds

Fetzer’s Dynamic Difference

ARGUMENT 2: Computers are governed by algorithms, but minds are not.

Premise 1: Computers are governed by programs, which are causal models of algorithms.

Premise 2: Algorithms are effective decision procedures for arriving at definite solutions to problems in a finite number of steps.

Premise 3: Most human thought processes, including dreams, daydreams, and ordinary thinking, are not procedures for arriving at solutions to problems in a finite number of steps.

______________________________________________________________________

Conclusion 1: Most human thought processes are not governed by programs as causal models of algorithms.

_______________________________________________________________________

Conclusion 2: Minds are not computers.

Figure 11. The Dynamic Difference

The Dynamic Difference

• Premises 1 & 2:

– Def of ‘algorithm’ is OK

– But algorithms may be the wrong entity• may need a more general notion of “procedure”

(Shapiro)

• like an algorithm, but:

– need not halt

– need not yield “correct” output

– can access external KB (Turing “oracle” machine)

The Dynamic Difference

• Premise 3: Most human thinking is not algorithmic

– Dreams are not algorithms– Ordinary stream-of-consciousness thinking is not “algorithmic”

• BUT:

– Some human thought processes may indeed not be algorithms• Consistent with “proper” treatment of computationalism

– Real issue is…

• Could there be algorithms/procedures that produce these(or other mental states or processes)?– If dreams are our interpretations of random neuron firings during sleep,

as if they were due to external causes…• …then: if non-dream neuron-firings are computable

(& there’s every reason to think they are) then so are dreams

– Stream of consciousness might be computable • e.g., via spreading activation in a semantic network

The Dynamic Difference

• Whether a mental state/process is computable is at least an empirical question

– Must avoid the Hubert Dreyfus fallacy:• one philosopher’s idea of a non-computable process

is another computer scientist’s research project• what no one has yet written a program for

is not thereby necessarily non-computable

– In fact: Mueller, Erik T. (1990), Daydreaming in Humans &

Machines: A Computer Model of the Stream of Thought (Ablex)

• Cf. Edelman, Shimon (2008), Computing the Mind (Oxford)

burden of proof is on Fetzer!

The Dynamic Difference

• Dynamic Conclusion 2:

– Are minds computers?

• Maybe, maybe not

• I prefer to say (with Shimon Edelman, et al.):

– The (human) mind is a virtual machine,computationally implemented (in the nervous system)

Summary: No “Dynamic Difference”

• All (human) thought processes are/might be describable by algorithms/procedures= computationalism properly treated

Fetzer’s Affective DifferenceARGUMENT 3: Mental thought transitions are affected by emotions, attitudes,

and histories, but computers are not.

Premise 1: Computers are governed by programs, which are causal models of algorithms.

Premise 2: Algorithms are effective decisions, which are not affected by emotions, attitudes, or histories.

Premise 3: Mental thought transitions are affected by values of variables that do not affect computers. _____________________________________________________________________

Conclusion 1: The processes controlling mental thought transitions are fundamentally different than those that control

computer procedures. _____________________________________________________________________

Conclusion 2: Minds are not computers.

Figure 12. The Affective Difference

Contra Affective Premises 2 & 3:

• Programs can be based on (idiosyncratic)emotions, attitudes, & histories

– Rapaport-Ehrlich contextual vocabulary acquisition program• Learns a meaning for an unfamiliar word from:

– the word’s textual context

– integrated with the reader’s idiosyncratic …

» “denotations”, “connotations”,

» emotions, attitudes, histories,

» & prior beliefs

– Sloman, Picard, Thagard• Developing computational theories of affect, emotion, etc.

• Emotions, attitudes, & histories can affect computers that model them.

Summary: No “Affective Differences”

• Processes controlling mental thought transitions are not fundamentally different from those controlling algorithms/procedures.

• Algorithms can take emotions/attitudes/histories into account.

• Both computers & minds can be affected by emotions/attitudes/histories

The Matter of SimulationARGUMENT 4: Digital machines can nevertheless simulate thought processes

and other forms of human behavior.

Premise 1: Computer programmers and those who design the systems that they control can increase their performance capabilities, making them better and better simulations.

Premise 2: Their performance capabilities may be closer and closer approximations to the performance capabilities of human beings without turning them into thinking things.

Premise 3: Indeed, the static, dynamic, and affective differences that distinguish computer performance from human performance preclude them from being thinking things.

______________________________________________________________________________ Conclusion: Although the performance capabilities of digital machines can

become better and better approximations of human behavior, they are still not thinking things.

Figure 15. The Matter of Simulation

Argument from Simulation

• Agreed:A computer that “simulates” some process P is not necessarily “really” doing P– But what is “really doing P” vs. “simulating P”?– What is the “scope” of a simulation?

• Computer simulations of hurricanes don’t get real people really wet

– Real people are outside the scope of the simulation– BUT: a computer simulation of a hurricane could get

simulated people simulatedly wet

• Computer simulation of the daily operations of a bank is not thereby the daily operations of a (real) bank

– BUT: I can do my banking online

– Simulations can be used as if they were real

Argument from Simulation

• Some simulations of Xs are real Xs:– scale model of a scale model of X is a scale model of X– Xeroxed/faxed/PDF copies of documents

are those documents

– A computer that simulates an “informational process” is thereby actually doing that informational process

• Because a computer simulation of information is information…

Argument from Simulation

• Computer simulation of a picture is a picture– digital photography

• Computer simulation of language is language– computers really do parse sentences (Woods)– IBM’s Watson really answers questions

• Computer simulation of math is math– “A simulation of a computation and the computation itself

are equivalent: try to simulate the addition of 2 and 3, and the result will be just as good as if you ‘actually’ carried out the addition—that is the nature of numbers” (Edelman)

• Computer simulation of reasoning is reasoning– automated theorem proving, computational logic,…

Argument from Simulation

• Computer simulation of cognition is cognition

– “if the mind is a computational entity, a simulation of the relevant computations would constitute its fully functional replica” (Edelman)

– cf. “implementational implication”

Summary: Simulation Can Be(come) the Real Thing

• Close approximation to human thought processes can turn computers into thinking things– actually?– only asymptotically?– merely conventionally?

• Turing said…

• “the use of words and general educated opinion will alter so much that one will be able to speak of machines thinking without expecting to be contradicted.” (Turing 1950)

– “general educated opinion” • changes when we abstract & generalize

– “the use of words” • changes when reference shifts

from word’s initial / narrow application to more abstract / general phenomenon

– cf. “fly”, “compute”, “algorithm”– ditto for “cognition” / “think”

Summary

• Computers are “semiotic (sign-using) systems”.

• Computationalismproperly treated = cognition is computable…• …not necessarily computational.• Any non-computable residue will be negligible

– Mental processes are describable (governable) by algorithmic procedures…– …that can be affected by emotions/attitudes/individual histories. – Computers that implement these cognitive procedures

really exhibit those cognitive behaviors.• They really think.

– Computers can possess minds.

• “Syntactic semantics” explains how all this is possible.

• Rapaport, William J. (2012),

– “Semiotic Systems, Computers, and the Mind: How Cognition Could Be Computing”,

– International Journal of Signs and Semiotic Systems 2(1) (January-June): 32–71.