1

CDT314

FABER

Formal Languages, Automata and Models of Computation

Lecture 12

Mälardalen University

2011

2

Content

Chomsky’s Language Hierarchy Turing Machines DeterminismHaltingTM ExamplesStandard TMComputing Functions with TMCombining TMsTuring Thesis

3

The Language Hierarchy

*aRegular Languages

Context-Free Languagesnnba Rww

nnn cba ww?

**ba

?

4

Regular Languages

**ba

Context-free languages

Recursively enumerable languages}0:{ ncba nnn

}0:{ nba nn

}},{:{ bawww

}*},{:{ bawwwR

}0:{ ! nan }0:{2

nba nn

5

*aFinite Automata

Push-down Automatannba Rww

nnn cba ww

**ba

Turing Machines

6

http://www.turing.org.uk/turing/

Who was Alan Turing?

Founder of Computability Theory,

mathematician,

philosopher,

codebreaker,

visionary man before his time.

http://www.cs.usfca.edu/www.AlanTuring.net/turing_archive/index.html- Jack

Copeland and Diane Proudfoot http://www.turing.org.uk/turing/ The Alan Turing Home Page

Andrew Hodges

7

Alan Turing

1912 (23 June): Birth, London

1926-31: Sherborne School

1930: Death of friend Christopher Morcom

1931-34: Undergraduate at King's College, Cambridge University

1932-35: Quantum mechanics, probability, logic

1935: Elected fellow of King's College, Cambridge

1936: The Turing machine, computability, universal machine

1936-38: Princeton University. Ph.D. Logic, algebra, number theory

1938-39: Return to Cambridge. Introduced to German Enigma cipher machine

1939-40: The Bombe, machine for Enigma decryption

1939-42: Breaking of U-boat Enigma, saving battle of the Atlantic

8

Alan Turing

1943-45: Chief Anglo-American crypto consultant. Electronic work.

1945: National Physical Laboratory, London

1946: Computer and software design leading the world.

1947-48: Programming, neural nets, and artificial intelligence

1948: Manchester University

1949: First serious mathematical use of a computer

1950: The Turing Test for machine intelligence

1951: Elected FRS. Non-linear theory of biological growth

1952: Arrested as a homosexual, loss of security clearance

1953-54: Unfinished work in biology and physics

1954 (7 June): Death (suicide) by cyanide poisoning, Wilmslow, Cheshire.

9

Hilbert’s Program, 1900

Hilbert’s hope was that mathematics would be reducible to finding proofs (manipulating the strings of symbols) from a fixed system of axioms, axioms that everyone could agree were true.

Can all of mathematics be made algorithmic, or will there always be new problems that outstrip any given algorithm, and so require creative acts of mind to solve?

10

Turing Machines and Computability

The question Hilbert raised was whether there could be a general method or process by which one could decide whether a mathematical proposition could be proved. But what exactly was meant by a 'method' or 'process'? People had already used the concept of a 'mechanical' process, and Turing had an idea which made this quite precise: computability. Turing wrote in his first sentences:

“The "computable" numbers may be described briefly as the real numbers whose expressions as a decimal are calculable by finite means... a number is computable if its decimal can be written down by a machine. “

http://www.turing.org.uk/turing/scrapbook/machine.html

11

Turing Machines and Computability

The Turing machine concept involves specifying a very restricted set of logical operations, but Turing showed how other more complex mathematical procedures could be built out of these atomic components.

Turing argued that his formalism was sufficiently general to encompass anything that a human being could do when carrying out a definite method.

Turing's famous 1936-7 paper On computable numbers, with an application to the Entscheidungsproblem,  which worked out the theory of Turing machines and the definition of computability, is available as a PDF file on-line:

http://www.cs.ox.ac.uk/activities/ieg/e-library/sources/tp2-ie.pdf

12

Turing Machines

Turing Machines

Turing’s "Machines". These machines are humans who calculate. (Wittgenstein)

A man provided with paper, pencil, and rubber, and subject to strict discipline, is in effect a universal machine. (Turing)

13

14

............Tape

Read-Write head

Control Unit

Turing Machine

15

............

Read-Write head

No boundaries -- infinite length

The head moves Left or Right

The Tape

16

............

Read-Write head

1. Reads a symbol

2. Writes a symbol

3. Moves Left or Right

The head at each time step:

17

ExampleTime 0

............ a a cb

Time 1............ a b k c

1. Reads a2. Writes k3. Moves Left

18

Time 1............ a b k c

Time 2............ a k cf

1. Reads b2. Writes f

3. Moves Right

19

Head starts at the leftmost position

of the input string

............

Blank symbol

head

a b ca

Input string

The Input String

#####

20

1q 2qLba ,

Read WriteMove Left

1q 2qRba ,

Move Right

States & Transitions

21

Example

1q 2qRba ,

............ a b ca

Time 1

1qcurrent state

#####

22

............ # a b caTime 1

1q 2qRba ,

............ a b cbTime 2

1q

2q

# # # #

# # # # #

23

............ a b caTime 1

1q 2qLba ,

............ a b cbTime 2

1q

2q

Example

# # # # #

# # # # #

24

............ a b caTime 1

1q 2qRg,#

............ ga b cbTime 2

1q

2q

Example

# # # # #

# # # #

25

Determinism

1q

2qRba ,

Allowed Not Allowed

3qLdb ,

1q

2qRba ,

3qLda ,

No lambda transitions allowed in TM!

Turing Machines are deterministic

26

Determinism

Note the difference between state indeterminismwhen not even possible future states are known in advance

and choice indeterminismwhen possible future states are known,but we do not know which state will be taken.

27

Partial Transition Function

1q

2qRba ,

3qLdb ,

............ a b ca

1q

Example

No transition

for input symbol c

Allowed

# # # # #

28

Halting

The machine halts if there are

no possible transitions to follow

29

Example

............ a b ca

1q

1q

2qRba ,

3qLdb ,

No possible transition

HALT!

# # # # #

30

Final States

1q 2q Allowed

1q 2q Not Allowed

• Final states have no outgoing transitions

• In a final state the machine halts

31

Acceptance

Accept InputIf machine halts

in a final state

Reject Input

If machine halts

in a non-final state

or

If machine enters

an infinite loop

32

Turing Machine Example

A TM that accepts the language *aa

0q

Raa ,

L,##1q

33

Time 0 aa

0q

a

0q

Raa ,

1qL,##

# # # #

34

Time 1 aa

0q

a

0q

Raa ,

1q

# # # #

L,##

35

Time 2 aa

0q

a

0q

Raa ,

1q

# # # #

L,##

36

Time 3aa

0q

a

0q

Raa ,

1qL,##

# # # #

37

Time 4 aa

1q

a

0q

Raa ,

1q

Halt & Accept

L,##

# # # #

38

Rejection Example

Time 0 ba

0q

a

0q

Raa ,

1qL,##

# # # #

39

0q

Raa ,

1q

Time 1 ba

0q

a

No possible Transition

Halt & Reject

L,##

# # # #

40

Infinite Loop Example

0q

Raa ,

1q

Lbb ,

Another TM for language *aa

L,##

41

Time 0 ba

0q

a

0q

Raa ,

1q

Lbb ,

L,##

# # # #

42

Time 1 ba

0q

a

0q

Raa ,

1q

Lbb ,

L,##

# # # #

43

Time 2 ba

0q

a

0q

Raa ,

1q

Lbb ,

L,##

# # # #

44

Time 2 ba

0q

a

baTime 3

0q

a

baTime 4

0q

a

baTime 5

0q

a

... infinite loop…

# # # #

# # # #

# # # #

# # # #

45

Because of the infinite loop:

•The final state cannot be reached

•The machine never halts

•The input is not accepted

46

Another Turing Machine Example

Turing machine for the language }{ nnba

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

47

Time 0 ba

0q

a b

Ryy ,

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

4q

L,##

# # #}{ nnba

48

bx

1q

a bTime 1

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

49

bx

1q

a bTime 2

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

50

yx

2q

a bTime 3

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

51

yx

2q

a bTime 4

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

52

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

yx

0q

a bTime 5

L,##

# # #}{ nnba

53

yx

1q

x bTime 6

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

54

yx

1q

x bTime 7

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

55

yx x y

2q

Time 8

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

56

yx x y

2q

Time 9

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

57

Ryy ,

yx

0q

x yTime 10

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

4q

L,##

# # #}{ nnba

58

yx

3q

x yTime 11

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

59

yx

3q

x yTime 12

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

60

yx

4q

x y

0q 1q 2q3qRxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

Halt & Accept

Time 13

L,##

# # #}{ nnba

61

If we modify the

machine for the language }{ nnba

we can easily construct

a machine for the language }{ nnn cba

Observation

62

Formal Definitions for

Turing Machines

63

Transition Function

1q 2qRba ,

),,(),( 21 Rbqaq

64

1q 2qLdc ,

),,(),( 21 Ldqcq

Transition Function

65

Turing Machine

),#,,,,,( 0 FqQM

Transition

functionInitial

stateblank

Final

states

States

Input

alphabetTape

alphabet

66

Instantaneous description:

ba

1q

ac

baqca 1

Configuration

# # # #

67

Time 4 Time 5

yx

2q

a b yx

0q

a b

A move:aybqxxaybq 02

# # # # # #

68

Time 4 Time 5

yx

2q

a b yx

0q

a b

bqxxyybqxxaybqxxaybq 1102

Time 6 Time 7

yx

1q

x b yx

1q

x b

# # # # # #

# # # # # #

69

bqxxyxaybq 12

Equivalent notation:

bxxqybxxqaybxqxaybq 1102

70

Initial configuration: wq0

ba

0q

a b

w

Input string

# # #

71

For any Turing Machine M

}:{)( 210 xqxwqwML f

Initial state Final state

The Accepted Language

72

Standard Turing Machine

• Deterministic

• Infinite tape in both directions

•Tape is the input/output file

The machine we described is the standard:

73

Computing Functionswith

Turing Machines

74

A function )(wf has:

Domain

(domän):

DDw

Range

(värdemängd):

S Swf )(

75

A function may have many parameters:

yxyxf ),(

Example: Addition function

76

Integer Domain

Binary: 101

Decimal: 5

We prefer unary representation:

easier to manipulate

Unary: 11111

77

Definition

A function is computable if

there is a Turing Machine such that:

fM

Initial configuration Final configuration

Dw Domain

fq

)(wf

final state0q

w

initial state

For all

# # # #

78

)(0 wfqwq f

Initial

Configuration

Final

Configuration

In other words

Dw DomainFor all

A function is computable if

there is a Turing Machine such that

fM

79

Example (Addition)

The function yxyxf ),( is computable

Turing Machine:

Input string: yx0 unary

Output string: 0xy unary

yx, are integers

80

Start

Finish 0

fq

11

yx

11

final state

0

0q

1 11 1

x y

1

initial state

# #

# #

81

0q 1q 2q 3qL,## L,01

L,11

R,##

R,10

R,11

4q

R,11

Turing machine for function yxyxf ),(

82

Execution Example:

11x

11y

Time 0

0

0q

1 11 1x y

Final Result

0

4q

1 11 1

yx

(2)

(2)

# #

# #

83

Time 0 0

0q

1 11 1

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

84

0q

01 11 1Time 1

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

85

0

0q

1 11 1Time 2

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

86

1q

1 11 11Time 3

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

87

1q

1 11 11Time 4

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

88

1q

1 11 11Time 5

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

89

2q

1 11 11Time 6

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

90

3q

1 11 01Time 7

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

91

3q

1 11 01Time 8

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

92

3q

1 11 01Time 9

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

93

3q

1 11 01Time 10

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

94

3q

1 11 01Time 11

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

95

4q

1 11 01

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

HALT & accept

Time 12 # #

L,##

R,##

yxyxf ),(

96

Another Example (Multiplication)

The function xxf 2)( is computable

Turing Machine

Input string: x unary

Output string: xx unary

x is integer

97

Start

Finish 1

fq

11

x2

11

final state

0q

11

x

1

initial state

# #

# #

98

Turing Machine Pseudocode for xxf 2)(

• Replace every 1 with $

• Repeat:

• Find rightmost $, replace it with 1

• Go to right end, insert 1

Until no more $ remain

99

0q 1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

Turing Machine for xxf 2)(

100

0q

11

3q

1 11 1

Start Finish

# # # #

0q 1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

xxf 2)(

101

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

0q

1Time 0 # # # #

0q

1 xxf 2)(

102

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

0q

1$Time 1 # # # #

0q

xxf 2)(

103

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

0q

Time 2 # # # #

0q

$ $ xxf 2)(

104

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

1q

Time 3 # # # #

0q

$ $ xxf 2)(

105

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

2q

1Time 4 # # # #

0q

$ xxf 2)(

106

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

1q

1 1Time 5 # # #

0q

$ xxf 2)(

107

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

1q

1 1Time 6 # # #

0q

$ xxf 2)(

108

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

2q

1 1Time 7 # # #

0q

xxf 2)( 1

109

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

2q

1 1Time 8 # # #

0q

xxf 2)( 1

110

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

2q

1 1Time 9 # # #

0q

xxf 2)( 1

111

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

11Time 10 # #

0q

1q

1 1 xxf 2)(

112

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

11Time 11 # #

0q

1q

1 1 xxf 2)(

113

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

11Time 12 # #

0q

1q

1 1 xxf 2)(

114

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

11Time 13 # #

0q

1q

1 1 xxf 2)(

115

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

11Time 14 # #

0q

3q

1 1 xxf 2)(

116

Another Example

The function is computable

),( yxf0

1 yx

yx

if

if

Turing Machine for

Output: 1 0or

Input: yx0

117

Turing Machine Pseudocode

Match a 1 from with a 1 from x y

• Repeat

Until all of or are matchedx y

• If a 1 from is not matched

erase tape, write 1

else

erase tape, write 0

x

118

Combining Turing Machines

119

Block Diagram

Turing

Machineinput output

120

Example

),( yxf0

yx yx

yx

if

if

Comparer

Adder

Eraser

yx, yx

yx

yx

0

121

Turing’s Thesis

122

Do Turing machines have

the same power with

a digital computer?

Intuitive answer: Yes

There is no formal answer!

Question:

123

Turing’s thesis

Any computation carried out

by mechanical means

can be performed by a Turing Machine

(1930)

http://www.engr.uconn.edu/~dqg/papers/myth.pdf The Origins of the Turing Thesis Myth Goldin & Wegner

124

And, the other way round:

A computation is mechanical

if and only if

it can be performed by a Turing Machine

Strong Turing Thesis: There is no known model of computation

more powerful than Turing Machines

Wrong! Asynchronous parallel computation is more powerful than TM model, and cannot be reduced to TM.

125

Definition of Algorithm

An algorithm for function

is a Turing Machine which computes

)(wf)(wf

126

When we say

There exists an algorithm

Algorithms are Turing Machines

It means

There exists a Turing Machine

127

Language Grammar Machine

Chomsky Hierarchy

Non-computable ??

Recursively enumerable (RE)

Unrestricted Turing machines 0

Recursive

Turing machines that always halt 0

Context-sensitive

Context-sensitive grammar

Linear-bounded automata 1

Context-free

Context-free grammar

Non-deterministic pushdown automata 2

Regular

Regular expressions Finite state automata 3

properly inclusive

Languages, Grammars, Machines

128

Regular Languages

FINITE AUTOMATA

**ba

Context-free languages

PUSH-DOWN AUTOMATA

Unrestricted grammar languages

TURING MACHINES

}0:{ ncba nnn

}0:{ nba nn

}},{:{ bawww

}*},{:{ bawwwR

}0:{ ! nan }0:{2

nba nn

Interesting further reading

Computational and evolutionary aspects of language

http://www.nature.com/nature/journal/v417/n6889/full/nature00771.html

Turing’s thesis

http://www.ams.org/notices/200610/fea-feferman.pdf

http://www.mathcomp.leeds.ac.uk/turing2012/give-page.php?408

2012 The Alan Turing World

https://sites.google.com/site/naturalcomputingaisbiacap2012 Symposium on Natural/Unconventional Computing AISB/IACAP World Congress

129