Introduction to Computer ScienceCSCI 109

Andrew GoodneyFall 2019

China – Tianhe-2

Lecture 11: Abstract Machines and Theory Nov. 18th, 2019

Schedule

1

Abstract machines and theory

uWhat is theoretical computer science?

uFinite state machinesuThe Turing machine

u The Church-Turing thesisu The halting problem

uThe analysis of algorithmsuThe analysis of problems

2

Reading: St. Amant Ch. 8

What is theoretical computer science?

uThe study of what can be computed and how efficiently it can be computed

uBased on abstract mathematical models uInsight into the behavior of a real

computer… …any real computer

3

Finite state machines

4

On

Off

UpDown

Down

Up

Light switch

Crying Asleep

Awake, not crying

Fed

HungryMorehungry

Hungry

1 day old infant10 day old infant

Finite state machines

5

M=F

M+1 M+2 M+3 M+4

F+1 F+2 F+3 F+4

Add M Add M Add M

Add F Add F Add F

Add M Add M Add M

Add M

Add F Add F Add FAdd M

Add FAdd F

St. Amant pp. 152

Add M

Add F

Finite State Machines

u Why look at FSMs?

u They “compute”v Sort-of… lots of limitations

u They are simplev Can be modeled mathematically

6

Finite state machine limitations

u If a finite state machine with n states is given n+1 symbols, it will revisit one state at least once

uNo way to tell if revisited state is being visited for the 1st time or 5th time or nth time

uThe pigeonhole principleuProblems an FSM cannot solve:

v Given a string of As and Bs, tell if the number of As and Bs are the same

v Tell if a string is a palindrome7

FSM -> TM

u FSM can’t do general computation

u Their simplicity is alluring

u Can we do better?v Combine FSM + Memory

u Several formsv Most famous is Turing Machinev Memory is a “tape”

8

The Turing machine

9

u An infinite tape, with squares marked on itu Each square can be blank or hold a symbolu A machine moves over the tapeu When positioned over a square,

the machine can v Read a symbol from the tapev Erase a symbol from the tapev Write a symbol onto the tape (erasing what was

on the square)v Do nothing

u The machine can move either one square to right or left

Simple Turing Machine with 1 symbol

10

https://vimeo.com/46913004

Alan Turing

11

Turing Machines…WTF?

u “Alan Turing and his Ph.D. advisor Church studied the universality of computation (among other things) in the 1930s. Turing formulated the Turing Machine in order to capture everything that computers can do…” Igor Markov, EECS Prof at Michigan

u “As others have correctly pointed out that the purpose ‘was to answer the halting problem, which Turing published in his 1937 paper On Computable Numbers With an Application to the Entscheidungsproblem’. The idea that a machine can perform the tasks of any other machine is at the centre of the concept of a Universal (Turing) machine…” Hector Zenil, Lab Leader, Karolinska - Senior Researcher, Oxford

12

More quotes re: Turing Machines

u “A Turing machine can perform any algorithm -- it can always be configured to give a particular output for a particular input. But this simple idealized device requires infinite memory so it's not possible to build a physical Turing machine. However, a physical device is said to be Turing complete if it can simulate any algorithm that could be run by the idealized Turing machine.” Jeff Lamattery, B.S. Physics, UCSD

13

More quotes re: Turing Machines

u “…Newman mentioned Hilbert's Entscheidungsproblem which dates back to 1928. This problem is that of finding an algorithm that can decide whether a given statement is provable within a logical system. If this decision problem were solvable, we could use algorithms to carry out mathematical research… Turing set out to understand the Entscheidungsproblem, and for this he needed to develop a mathematical model of algorithms. This model is what we now call the Turing machine model…The main theorem of the paper is that the Entscheidungsproblem is not solvable – put informally, there can be no algorithm that can "automatize" mathematics – and the proof relies on being able to express the behaviour of a Turing machine in first-order logic.” Hans Hyttel, Associate Professor

14

Turing Machine Representation: Table

15

This machine has 4 symbols: 0, 1, _ and *L means the head moves to the leftR means the head moves to the right

Turing Machine Representation: Table

16

Turing Machine Representation: Table

17

Turing Machine Representation: Table

18

write_symbol, head_move_direction, state_update

Turing Machine Representation: Table

19

write_symbol, head_move_direction, state_update

Same Machine as a State Diagram

20read_symbol,write_symbol,head_position_move

Same Machine as a State Diagram

21read_symbol,write_symbol,head_position_move

22

Example execution

u Assume the machine always begins in state 0 with the head positioned on the rightmost non-blank square

u The machine stops when it is in state 4 (called an accept state)

u The machine stops when it is in state 5 (called a reject state)

u How does the execution look if the tape has an input of 10 written on it at the beginning ?

23

Behavior of the machine on input 10

24

25

Behavior of the machine

on input 10

26

Behavior of the machine

on input 1010

What does this Turing machine do?

uChecks if a binary string has the same number of 1’s and 0’s and enters the accept state if it does, otherwise it enters the reject state.

27

Turing Machine Programming

uWriting programs for a Turing machine is very tedious

uHowever…

28

The Church-Turing thesis

Turing machines are capable of solving any effectively solvable algorithmic problem

Turing machines are thus excellent models for what (all) real computers are capable of doing

29

Problems not ‘effectively computable’

uAre there problems that are not effectively computable ?

uYes, infinite such problems

uCould these be computable someday ?uPerhaps, but not on a Turing-equivalent

computer

30

A halting question

uDeciding whether a program will halt (terminate)uDoes the following program halt ?

Input number

While number is not 0Print numbernumber = number – 1

31

The halting problem

uDeciding whether a given program will halt (terminate) on an arbitrary input

u In 1936, Turing proved that a general algorithm to the halting problem for all possible programs over all possible inputs cannot exist

The halting problem is undecidable

32

Undecidable Problem

u Undecidable Problem:v Problem requires a yes/no answer v But no program exists that will always give the correct answer

33

Proof sketch for halting problem

u Want to decide if program A halts on input au Need to write a program, lets call it halter

v Inputs: A, av Output: Yes (if program A halts on input a) or No (if program A

does not halt on input a)

halter(A,a)if(some complex computation)is trueReturn Yes

elseReturn No

34

i.e analyze A for input a

The halting problem is undecidable

uIn 1936, Turing proved that a general algorithm to the halting problem for all possible programs over all possible inputs cannot exist

uIn other words, a general version of halter (one that works for all programs and their inputs) does not exist

35

Proof by contradiction

u Assume that a general version of halter existsu Show that this assumption leads to a contradictionu Invent a new program called cleverclever(program,input)

result = halter(program,input)if result is No

return Yeselse

loop forever36

Proof by contradiction

uWhat does clever do when given itself as input?clever(program,input)

result=halter(program,input)if result is No

return Yeselse

loop forever37

Proof by contradiction

u If halter says that clever halts, then cleverloops forever (which means it doesn’t halt)

u If halter says that clever does not halt then clever halts and returns Yes 38

clever(program,input)

result=halter(program,input)

if result is No

return Yes

else

loop forever

Conclusion: The program halter cannot exist

Russell’s Paradox

u In a hypothetical small town:v There is one male barberv The barber shaves all those

and only those men in town who do not shave themselves

v All men stay clean shaven either by shaving themselves or going to the barber

u Who shaves the barber?

39

u More generally:v Let R be the set of all sets

that are not members of themselves

v If R is not a member of itself, then its definition dictates that it must contain itself, and if it contains itself, then it contradicts its own definition as the set of all sets that are not members of themselves

https://plato.stanford.edu/entries/russell-paradox/

Other Undecidable Problemsu Both of these proved by showing an implementation on a Turing machine

would just be the halting problemu Matrix mortality: given a finite set of n x n sized matrices, does a sequence of

matrix multiplications (repetitions allowed) exist that yields the zero matrix?v Undecidable for a set of six or more 3 x 3 matrices

u Post’s Correspondence Problemv Given two equal length lists of strings written in the same alphabet, (list A, list B) where

the strings are indexed 1….|A|, does a sequence of indexes exist that when applied to each list, generates the same compound word?

40

Analysis of problems

u Study of algorithms illuminates the study of classes of problems

u If a polynomial time algorithm exists to solve a problem then the problem is called tractable

u If a problem cannot be solved by a polynomial time algorithm then it is called intractable

u This divides problems into three groups:v Problems with known polynomial time algorithmsv Problems that are proven to have no polynomial-time algorithmv Problems with no known polynomial time algorithm but not yet

proven to be intractable41

Tractable and Intractable

u Tractable problems (P)v Sorting a listv Searching an unordered listv Finding a minimum spanning tree

in a graph

42

u Intractablev Listing all permutations (all

possible orderings) of n numbers

u Might be (in)tractablev Subset sum: given a set of

numbers, is there a subset that adds up to a given number?

v Travelling salesperson: n cities, n! routes, find the shortest route

These problems have no known polynomial time solution

However no one has been able to prove that such a solution does not exist

Tractability, Intractability, Undecidability

u ‘Properties of problems’ (NOT ‘properties of algorithms’)

u Tractable: problem can be solved by a polynomial time algorithm (or something more efficient)

u Intractable: problem cannot be solved by a polynomial time algorithm (all solutions are proven to be more inefficient than polynomial time)

u Unknown: not known if the problem is tractable or intractable (no known polynomial time solution, no proof that a polynomial time solution does not exist)

u Undecidable: decision problem for which there is (provably) no algorithmic solution on a Turing machinev Non-existencev Not a matter of efficiency

43