Chapter 10Chapter 10

Models of ComputationModels of Computation

What is a model?What is a model?

Captures the essence – the important Captures the essence – the important properties– of the real thingproperties– of the real thing

Probably differs in scale from the real Probably differs in scale from the real thingthing

Suppresses details of the real thingSuppresses details of the real thing Lacks the full functionality of the real Lacks the full functionality of the real

thingthing

ExamplesExamples

Physical modelsPhysical models• Model carsModel cars• DollsDolls• Velcro-covered balls stuck together in a certVelcro-covered balls stuck together in a cert

ain way to represent the molecular structurain way to represent the molecular structure.e.

Mathematical modelsMathematical models• d=rxtd=rxt• Weather systemWeather system

ModelsModels

Models Models • Can be used to predict behavior of Can be used to predict behavior of

existing phenomenaexisting phenomena• can provide environments for learning can provide environments for learning

and practicing interactions with various and practicing interactions with various phenomena.phenomena.

• Can be used as design tools.Can be used as design tools. But, the information gained is only as But, the information gained is only as

good as the model used.good as the model used.

A Model of Computing AgentA Model of Computing Agent

We want to construct a model for the We want to construct a model for the “computing agent” that will capture “computing agent” that will capture its fundamental properties and its fundamental properties and enable us to explore the capabilities enable us to explore the capabilities and limitations of computation and limitations of computation in the in the most general sensemost general sense. .

Properties of A Computing AgentProperties of A Computing Agent We shall require that any computing We shall require that any computing

agentagent• Can accept inputCan accept input• Can store information in and retrieve it Can store information in and retrieve it

from memoryfrom memory• Can take actions according to algorithm Can take actions according to algorithm

instructions, and that the choice of what instructions, and that the choice of what action to take may depend on the action to take may depend on the present state of the computing agent as present state of the computing agent as well as on the input item presently being well as on the input item presently being processedprocessed

• Can produce outputCan produce output

The Turing Machine: Historical The Turing Machine: Historical ProspectiveProspective

Interest in the theoretical nature of Interest in the theoretical nature of computation far predated the advent of computation far predated the advent of modern computers.modern computers.

Mathematicians are interested in formalizing Mathematicians are interested in formalizing the nature of proof, with two goals in mind:the nature of proof, with two goals in mind:• Guarantee the correctness of a proofGuarantee the correctness of a proof• Allow for mechanical theorem provingAllow for mechanical theorem proving

Leads to the study of the nature of Leads to the study of the nature of computation procedure itself.computation procedure itself.

We will look at the model proposed by Alan We will look at the model proposed by Alan Turing.Turing.

The Turing MachineThe Turing Machine

A Turing machine includes a (conceptual) A Turing machine includes a (conceptual) tape that extends infinitely in both directions.tape that extends infinitely in both directions.

The tape is divided into cells, each of which The tape is divided into cells, each of which contains one symbol.contains one symbol.

The symbols must come from a finite set of The symbols must come from a finite set of symbols called the alphabet.symbols called the alphabet.

The alphabet always contains a special The alphabet always contains a special symbol b (for “blank”), usually both of the symbol b (for “blank”), usually both of the symbols 0 and 1, and others.symbols 0 and 1, and others.

At any point in time, only a finite number of At any point in time, only a finite number of cells contain nonblank symbols.cells contain nonblank symbols.

The Turing Machine (cont’d)The Turing Machine (cont’d)

Example:Example:

The tape will be used to hold input The tape will be used to hold input to the Turing machine.to the Turing machine.

The Turing machine will write its The Turing machine will write its output on the tape.output on the tape.

The tape will also serve as memory.The tape will also serve as memory.

. b b 0 1 1 b b . .. .

The Turing Machine (cont’d)The Turing Machine (cont’d)

The rest of the Turing machine consists of The rest of the Turing machine consists of a unit that reads one cell of the tape at a a unit that reads one cell of the tape at a time and writes a symbol in that cell.time and writes a symbol in that cell.

There is a finite number k of “states” of There is a finite number k of “states” of the machine, labeled 1,2,…,k, and at any the machine, labeled 1,2,…,k, and at any moment, the unit is in one of these states.moment, the unit is in one of these states.

A Turing machine configuration:A Turing machine configuration:

. b b 0 1 1 b b . .. .

1 (current state of the machine)

The Turing Machine (cont’d)The Turing Machine (cont’d)

The Turing machine is designed to carry The Turing machine is designed to carry out only one type of primitive operation.out only one type of primitive operation.

Each time such an operation is done, three Each time such an operation is done, three actions take place:actions take place:• Write a symbol in the cellWrite a symbol in the cell• Go into a new stateGo into a new state• Move one cell left or rightMove one cell left or right

The details of the actions depend on the The details of the actions depend on the current state of the machine and on the current state of the machine and on the contents of the tape cell currently being contents of the tape cell currently being read (the input). read (the input).

Primitive Operation (Instruction)Primitive Operation (Instruction)

If you are in state IIf you are in state Iandand

you are reading symbol jyou are reading symbol jthen then

write symbol k onto the tapewrite symbol k onto the tapego into state sgo into state smove in direction dmove in direction d

Shorthand Notation for Turing Shorthand Notation for Turing Machine InstructionsMachine Instructions

Five components:Five components:• Current stateCurrent state• Current symbolCurrent symbol• Next symbolNext symbol• Next stateNext state• Direction of moveDirection of move

(current state, current symbol, next (current state, current symbol, next symbol, next state, direction of move)symbol, next state, direction of move)

ExampleExample

(1,0,1,2,R)(1,0,1,2,R)

. b b 0 1 1 b b . .. .

1 (current state of the machine)

. b b 1 1 1 b b . .. .

2 (current state of the machine)

More on Turing Machine More on Turing Machine InstructionsInstructions

What if we have (1,0,1,2,R) and (1,0,0,3,L)What if we have (1,0,1,2,R) and (1,0,0,3,L) Avoid ambiguity by requiring that a set of Avoid ambiguity by requiring that a set of

instructions for a Turing machine can never instructions for a Turing machine can never contain two different instructions of the form contain two different instructions of the form (i,j,-,-,-) and (i,j,-,-,-) (i,j,-,-,-) and (i,j,-,-,-)

If there is no instruction that applies to the If there is no instruction that applies to the current state-current symbol for the machine, current state-current symbol for the machine, then the machine halts.then the machine halts.

Conventions about initial configuration:Conventions about initial configuration:• The start-up will always be state 1The start-up will always be state 1• The machine will be reading the leftmost nonblank The machine will be reading the leftmost nonblank

cell on the tape.cell on the tape.

ExampleExample

1.1. (1,0,1,2,R)(1,0,1,2,R)

2.2. (1,1,1,2,R)(1,1,1,2,R)

3.3. (2,0,1,2,R)(2,0,1,2,R)

4.4. (2,1,0,2,R)(2,1,0,2,R)

5.5. (2,b,b,3,L)(2,b,b,3,L)

. b b 0 1 1 b b . .. .

1

Example (cont’d)Example (cont’d)

. b b 1 1 1 b b . .. .

2

. b b 1 0 1 b b . .. .

2

. b b 1 0 0 b b . .. .

2

. b b 1 0 0 b b . .. .

3

Turing Machine as a Computing Turing Machine as a Computing AgentAgent

The Turing machine can:The Turing machine can:• Read symbols on its tape.Read symbols on its tape.• Write symbols on its tape and, by Write symbols on its tape and, by

moving around over the tape, can go moving around over the tape, can go back and read those symbols at a later back and read those symbols at a later time.time.

• The present state and symbol determine The present state and symbol determine the appropriate instructionthe appropriate instruction

• If the Turing machine halts, what is If the Turing machine halts, what is written on the tape at that time can be written on the tape at that time can be considered output.considered output.

SummarySummary

A Turing machine is a general A Turing machine is a general computing machine.computing machine.

It has no limit on the amount of It has no limit on the amount of memory.memory.

A Model of An AlgorithmA Model of An Algorithm

An algorithm is a collection of instructions An algorithm is a collection of instructions intended for a computing agent to follow.intended for a computing agent to follow.

Requirements:Requirements:• Be a well-ordered collectionBe a well-ordered collection• Consists of unambiguous and effectively Consists of unambiguous and effectively

computable operationscomputable operations• Halts in a finite amount of time Halts in a finite amount of time • Produce a resultProduce a result

Does an arbitrary collection of Turing Does an arbitrary collection of Turing machine instructions satisfy all the machine instructions satisfy all the requirements?requirements?

Turing Machine ExamplesTuring Machine Examples

Bit converterBit converter• (1,0,1,1,R) and (1,1,0,1,R)(1,0,1,1,R) and (1,1,0,1,R)

A state diagram is a visual representation A state diagram is a visual representation of a Turing machine algorithm where circles of a Turing machine algorithm where circles represent states, and arrows represent represent states, and arrows represent transitions from one state to another.transitions from one state to another.

Along each transition arrow, we show three Along each transition arrow, we show three things:things:• The input symbol that causes the transitionThe input symbol that causes the transition• The corresponding outputThe corresponding output• The direction of moveThe direction of move

A Parity Bit MachineA Parity Bit Machine Odd parity bit: will be set so that the number of Odd parity bit: will be set so that the number of

1s in the whole string of bits, including the 1s in the whole string of bits, including the parity bit, is odd.parity bit, is odd.• (1,1,1,2,R): even parity state reading 1, change state(1,1,1,2,R): even parity state reading 1, change state• (1,0,0,1,R):even parity state reading 0, don’t change (1,0,0,1,R):even parity state reading 0, don’t change

statestate• (2,1,1,1,R): odd parity state reading 1, change state(2,1,1,1,R): odd parity state reading 1, change state• (2,0,0,2,R):odd parity state reading 0, don’t change (2,0,0,2,R):odd parity state reading 0, don’t change

statestate• (1,b,1,3,R): end of string in even parity state, write 1 (1,b,1,3,R): end of string in even parity state, write 1

and go to state 3and go to state 3• (2,b,0,3,R): end of string in even parity state, write 1 (2,b,0,3,R): end of string in even parity state, write 1

and go to state 3and go to state 3

Unary IncrementingUnary Incrementing

Unary representation of numbers:Unary representation of numbers:NumberNumber Turing machine tape representationTuring machine tape representation00 1111 111122 111111

Incrementing (algorithm 1):Incrementing (algorithm 1):• (1,1,1,1,R): pass to the right over 1s(1,1,1,1,R): pass to the right over 1s• (1,b,1,2,R): Add a single one at the right (1,b,1,2,R): Add a single one at the right

hand side of the stringhand side of the string

. b b 1 1 1 b b . .. .

Incrementing: Algorithm 2Incrementing: Algorithm 2

(1,1,1,1,L): pass to the left over 1s(1,1,1,1,L): pass to the left over 1s (1,b,1,2,L): Add a single 1 at the left-(1,b,1,2,L): Add a single 1 at the left-

hand end of the stringhand end of the string

2 steps vs. n+2 steps2 steps vs. n+2 steps

. b b 1 1 1 b b . .. .

Unary Addition MachineUnary Addition Machine

(1,1,b,2,R): erase the leftmost 1 and (1,1,b,2,R): erase the leftmost 1 and move rightmove right

(2,1,b,3,R): erase the second 1 and (2,1,b,3,R): erase the second 1 and move rightmove right

(3,1,1,3,R):pass over any 1s until a (3,1,1,3,R):pass over any 1s until a blank is foundblank is found

(3,b,1,4,R):write a 1 over the blank (3,b,1,4,R):write a 1 over the blank and halt.and halt.

The Church-Turing ThesisThe Church-Turing Thesis

If there is an algorithm to do a If there is an algorithm to do a symbol manipulation task, then there symbol manipulation task, then there is a Turing machine to do that task.is a Turing machine to do that task.

If we find a symbol manipulation task If we find a symbol manipulation task that no Turing machine can perform, that no Turing machine can perform, then there is no algorithm for this then there is no algorithm for this task.task.

Turing machines define the limits of Turing machines define the limits of Computability.Computability.

Unsolvable ProblemsUnsolvable Problems

Example:Example:(1,0,1,2,R)(1,0,1,2,R)(1,1,0,2,R)(1,1,0,2,R)(2,0,0,2,R)(2,0,0,2,R)(2,b,b,2,L)(2,b,b,2,L)

……b11bbb…b11bbb… 2 2

……b1bbb…b1bbb… 1 1

The Halting ProblemThe Halting Problem

Formal statement: Decide, given any collFormal statement: Decide, given any collection of Turing machine instructions toection of Turing machine instructions together with any initial tape contents, whgether with any initial tape contents, whether that Turing machine will ever halt iether that Turing machine will ever halt if started on that tape.f started on that tape.

The The halting problemhalting problem is unsolvable, and is unsolvable, and we will prove it by contradiction.we will prove it by contradiction.

ProofProof

Assume that P is a Turing machine that sAssume that P is a Turing machine that solves the halting problem. olves the halting problem.

On the initial for P we will have to put a On the initial for P we will have to put a description of a collection T of Turing mdescription of a collection T of Turing machine instructions, as well as the initial achine instructions, as well as the initial tape content t on which those instructiotape content t on which those instructions run.ns run.

……T*btbbb… (T* symbolize the binary foT*btbbb… (T* symbolize the binary form of T)rm of T)

Proof (cont’d)Proof (cont’d)

P will give us an answer P will give us an answer • Yes, halts (1)Yes, halts (1)• No, never halts (0)No, never halts (0)

When begun on a tape containing T* and t,When begun on a tape containing T* and t,• P halts with 1 on its tape exactly when T P halts with 1 on its tape exactly when T

eventually halts when begun on t eventually halts when begun on t • P halts with 0 on its tape exactly when T never P halts with 0 on its tape exactly when T never

halts when begun on t halts when begun on t Refer to Figure 10.10Refer to Figure 10.10

Proof (cont’d)Proof (cont’d)

Let’s imagine adding more instructions to P tLet’s imagine adding more instructions to P to create a new machine Q that behaves just liko create a new machine Q that behaves just like P except that when it reaches this sme confige P except that when it reaches this sme configuration, it moves forever to the right on the tauration, it moves forever to the right on the tape instead of halting. (Refer to Figure 10.11)pe instead of halting. (Refer to Figure 10.11)

Example: Pick some state not in P, say 52, and Example: Pick some state not in P, say 52, and add two new instructions to P: add two new instructions to P: • (9,1,1,52,R)(9,1,1,52,R)• (52,b,b,52,R)(52,b,b,52,R)

Proof (cont’d)Proof (cont’d) Finally, create a new machine S which first Finally, create a new machine S which first

makes a copy of what appears on its makes a copy of what appears on its input.input.

After S is finished with its copying job, it After S is finished with its copying job, it uses the same instruction as machine Q.uses the same instruction as machine Q.

What happens when machine S is run on a What happens when machine S is run on a tape that contains S*, the binary tape that contains S*, the binary representation of S?representation of S?

S first makes a copy of S* and then turn S first makes a copy of S* and then turn the computation over to Q, which is now the computation over to Q, which is now running on a tape containing S* and S*.running on a tape containing S* and S*.

Proof (cont’d)Proof (cont’d)S*

S*bS*

S

Input

outputNever halts exactly when S eventually halts on S*

Halts with 0 on tape exactly when S never halts on S*

Contradiction!

Practical Unsolvable ProblemsPractical Unsolvable Problems

No C++ program can be written to decide No C++ program can be written to decide whether any given C++ program always whether any given C++ program always stops eventually, no matter what input.stops eventually, no matter what input.

No C++ program can be written to decide No C++ program can be written to decide whether any two C++ programs are whether any two C++ programs are equivalent.equivalent.

No C++ program can be written to decide No C++ program can be written to decide whether any given C++ program on any whether any given C++ program on any given input will ever produce some given input will ever produce some specific output.specific output.