Function problemFrom Wikipedia, the free encyclopedia

Contents

1 Alphabet (formal languages) 11.1 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3 Literature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2 ChurchTuring thesis 22.1 Statement in Churchs and Turings words . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.2.1 Circa 19301952 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2.2 Later developments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2.3 The thesis as a denition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.3 Success of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.4 Informal usage in proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.5 Variations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.6 Philosophical implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.7 Non-computable functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.8 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.9 Footnotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.10 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.11 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3 Computability theory 143.1 Computable and uncomputable sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.2 Turing computability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.3 Areas of research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.3.1 Relative computability and the Turing degrees . . . . . . . . . . . . . . . . . . . . . . . . 163.3.2 Other reducibilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.3.3 Rices theorem and the arithmetical hierarchy . . . . . . . . . . . . . . . . . . . . . . . . 173.3.4 Reverse mathematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.3.5 Numberings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.3.6 The priority method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.3.7 The lattice of recursively enumerable sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.3.8 Automorphism problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

i

ii CONTENTS

3.3.9 Kolmogorov complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.3.10 Frequency computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.3.11 Inductive inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.3.12 Generalizations of Turing computability . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.3.13 Continuous computability theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.4 Relationships between denability, proof and computability . . . . . . . . . . . . . . . . . . . . . 203.5 Name of the subject . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.6 Professional organizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.7 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.8 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.10 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4 Computable function 244.1 Denition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244.2 Characteristics of computable functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254.3 Computable sets and relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264.4 Formal languages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264.6 ChurchTuring thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274.7 Uncomputable functions and unsolvable problems . . . . . . . . . . . . . . . . . . . . . . . . . . 274.8 Extensions of computability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

4.8.1 Relative computability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284.8.2 Higher recursion theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284.8.3 Hyper-computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

4.9 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284.10 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

5 Computational complexity theory 305.1 Computational problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

5.1.1 Problem instances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305.1.2 Representing problem instances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315.1.3 Decision problems as formal languages . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315.1.4 Function problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335.1.5 Measuring the size of an instance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

5.2 Machine models and complexity measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335.2.1 Turing machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335.2.2 Other machine models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345.2.3 Complexity measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345.2.4 Best, worst and average case complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . 355.2.5 Upper and lower bounds on the complexity of problems . . . . . . . . . . . . . . . . . . . 35

5.3 Complexity classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

CONTENTS iii

5.3.1 Dening complexity classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365.3.2 Important complexity classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365.3.3 Hierarchy theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375.3.4 Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

5.4 Important open problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385.4.1 P versus NP problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385.4.2 Problems in NP not known to be in P or NP-complete . . . . . . . . . . . . . . . . . . . . 395.4.3 Separations between other complexity classes . . . . . . . . . . . . . . . . . . . . . . . . 39

5.5 Intractability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405.6 History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405.7 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

5.8.1 Textbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425.8.2 Surveys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5.9 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

6 Context-free grammar 436.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436.2 Formal denitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

6.2.1 Production rule notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446.2.2 Rule application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446.2.3 Repetitive rule application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456.2.4 Context-free language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456.2.5 Proper CFGs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456.2.6 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

6.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456.3.1 Well-formed parentheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456.3.2 Well-formed nested parentheses and square brackets . . . . . . . . . . . . . . . . . . . . 466.3.3 A regular grammar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466.3.4 Matching pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476.3.5 Algebraic expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476.3.6 Further examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486.3.7 Derivations and syntax trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

6.4 Normal forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 506.5 Closure properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 506.6 Decidable problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516.7 Undecidable problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

6.7.1 Universality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516.7.2 Language equality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516.7.3 Language inclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516.7.4 Being in a lower or higher level of the Chomsky hierarchy . . . . . . . . . . . . . . . . . . 516.7.5 Grammar ambiguity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

iv CONTENTS

6.7.6 Language disjointness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516.8 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 526.9 Subclasses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 526.10 Linguistic applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536.11 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

6.11.1 Parsing algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536.12 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536.13 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

7 Counting problem (complexity) 557.1 Counting complexity class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 557.2 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

8 Decision problem 568.1 Denition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 568.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 568.3 Decidability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578.4 Complete problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 588.5 Equivalence with function problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 588.6 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 588.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

9 Emil Leon Post 609.1 Early work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 609.2 Recursion theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 609.3 Polyadic groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 619.4 Selected papers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 619.5 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 619.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 619.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 619.8 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 629.9 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

10 Exponential growth 6310.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6410.2 Basic formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6610.3 Reformulation as log-linear growth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6710.4 Dierential equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6710.5 Dierence equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6810.6 Other growth rates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6810.7 Limitations of models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6810.8 Exponential stories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

10.8.1 Rice on a chessboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

CONTENTS v

10.8.2 Water lily . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6910.9 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6910.10References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

10.10.1 Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7010.11External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

11 Function problem 7111.1 Formal denition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7111.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7111.3 Relationship to other complexity classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7111.4 Self-reducibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7211.5 Reductions and complete problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7211.6 Total function problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7211.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7211.8 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

12 Injective function 7312.1 Denition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7412.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7512.3 Injections can be undone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7812.4 Injections may be made invertible . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7812.5 Other properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7812.6 Proving that functions are injective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7912.7 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7912.8 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8012.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8012.10External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

13 Model of computation 8113.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8113.2 Uses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8113.3 Categories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8113.4 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8113.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8213.6 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

14 Norman Shapiro 8314.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8414.2 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

15 Optimization problem 8515.1 Continuous optimization problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8515.2 Combinatorial optimization problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

vi CONTENTS

15.2.1 NP optimization problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8615.3 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8715.4 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

16 Register machine 8816.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8816.2 Formal denition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8816.3 Historical development of the register machine model . . . . . . . . . . . . . . . . . . . . . . . . 90

16.3.1 (1954, 1957) Wangs model: Post-Turing machine . . . . . . . . . . . . . . . . . . . . . 9016.3.2 Minsky, Melzak-Lambek and Shepherdson-Sturgis models cut the tape into many . . . . 9116.3.3 (1961) Melzaks model is dierent: clumps of pebbles go into and out of holes . . . . . . . 9116.3.4 Lambek (1961) atomizes Melzaks model into the Minsky (1961) model: INC and DEC-

with-test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9216.3.5 Elgot-Robinson (1964) and the problem of the RASP without indirect addressing . . . . . 9216.3.6 Hartmanis (1971) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9316.3.7 Cook and Reckhow (1973) describe the RAM . . . . . . . . . . . . . . . . . . . . . . . . 93

16.4 Precedence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9316.5 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9416.6 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9416.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9516.8 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

17 Search problem 9817.1 Denition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9817.2 Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9917.3 Search method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9917.4 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9917.5 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

18 Tetration 10018.1 Denition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10318.2 Iterated powers vs. iterated bases/exponentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . 10318.3 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10318.4 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10418.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10418.6 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

18.6.1 Extension of domain for bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10418.6.2 Extensions of the domain for (iteration) heights . . . . . . . . . . . . . . . . . . . . . . 105

18.7 Open questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11018.8 Inverse relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

18.8.1 Super-root . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11018.8.2 Super-logarithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

18.9 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

CONTENTS vii

18.10References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11218.11External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

19 Turing machine 11619.1 Informal description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11619.2 Formal denition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11819.3 Additional details required to visualize or implement Turing machines . . . . . . . . . . . . . . . . 118

19.3.1 Alternative denitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11919.3.2 The state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11919.3.3 Turing machine state diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

19.4 Models equivalent to the Turing machine model . . . . . . . . . . . . . . . . . . . . . . . . . . . 12119.5 Choice c-machines, Oracle o-machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12219.6 Universal Turing machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12219.7 Comparison with real machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

19.7.1 Limitations of Turing machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12519.8 History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

19.8.1 Historical background: computational machinery . . . . . . . . . . . . . . . . . . . . . . 12519.8.2 The Entscheidungsproblem (the decision problem): Hilberts tenth question of 1900 . . . 12619.8.3 Alan Turings a- (automatic-)machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12719.8.4 19371970: The digital computer, the birth of computer science . . . . . . . . . . . . 12819.8.5 1970present: the Turing machine as a model of computation . . . . . . . . . . . . . . . 128

19.9 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12919.10Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12919.11References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

19.11.1 Primary literature, reprints, and compilations . . . . . . . . . . . . . . . . . . . . . . . . 12919.11.2 Computability theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12919.11.3 Churchs thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13019.11.4 Small Turing machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13119.11.5 Other . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

19.12External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

20 Turing reduction 13320.1 Denition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

20.1.1 Relation of Turing completeness to computational universality . . . . . . . . . . . . . . . . 13320.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13420.3 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13420.4 The use of a reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13420.5 Stronger reductions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13420.6 Weaker reductions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13520.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13520.8 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

viii CONTENTS

21 -recursive function 13621.1 Denition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13621.2 Equivalence with other models of computability . . . . . . . . . . . . . . . . . . . . . . . . . . . 13721.3 Normal form theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13721.4 Symbolism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13821.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13921.6 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13921.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13921.8 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13921.9 Text and image sources, contributors, and licenses . . . . . . . . . . . . . . . . . . . . . . . . . . 140

21.9.1 Text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14021.9.2 Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14321.9.3 Content license . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

Chapter 1

Alphabet (formal languages)

In formal language theory, a non-empty set is called alphabet when its intended use in string operations shall beindicated.[1][2] Its members are then commonly called symbols or letters, e.g. characters or digits.[1][2] For example acommon alphabet is {0,1}, the binary alphabet. A nite string is a nite sequence of letters from an alphabet; forinstance a binary string is a string drawn from the alphabet {0,1}. An innite sequence of letters may be constructedfrom elements of an alphabet as well.Given an alphabet , we write to denote the set of all nite strings over the alphabet . Here, the denotes theKleene star operator, so is also called the Kleene closure of . We write 1 (or occasionally, N or ! ) todenote the set of all innite sequences over the alphabet .For example, using the binary alphabet {0,1}, the strings , 0, 1, 00, 01, 10, 11, 000, etc. are all in the Kleene closureof the alphabet (where represents the empty string).Alphabets are important in the use of formal languages, automata and semiautomata. In most cases, for deninginstances of automata, such as deterministic nite automata (DFAs), it is required to specify an alphabet from whichthe input strings for the automaton are built.If L is a formal language, i.e. a (possibly innite) set of nite-length strings, the alphabet of L is the set of all symbolsthat may occur in any string in L. For example, if L is the set of all variable identiers in the programming languageC, Ls alphabet is the set { a, b, c, ..., x, y, z, A, B, C, ..., X, Y, Z, 0, 1, 2, ..., 7, 8, 9, _ }.

1.1 See also Combinatorics on words

1.2 References[1] Aho, Alfred V.; Sethi, Ravi; Ullman, Jerey D. (1985). Compilers: Principles, Techniques, and Tools (March 1988 reprint

ed.). Addison-Wesley. p. 92. ISBN 0-201-10088-6. The term alphabet or character class denotes any nite set of symbols.

[2] Ebbinghaus, H.-D.; Flum, J.; Thomas, W. (1994). Mathematical Logic (2nd ed.). New York: Springer. p. 11. ISBN0-387-94258-0. By an alphabet A we mean a nonempty set of symbols.

1.3 Literature John E. Hopcroft and Jerey D. Ullman, Introduction to Automata Theory, Languages, and Computation,

Addison-Wesley Publishing, Reading Massachusetts, 1979. ISBN 0-201-02988-X.

1

Chapter 2

ChurchTuring thesis

Churchs thesis redirects here. For the axiom CT in constructive mathematics, see Churchs thesis (constructivemathematics).

In computability theory, the ChurchTuring thesis (also known as the TuringChurch thesis,[1] the ChurchTuring conjecture, Churchs thesis, Churchs conjecture, and Turings thesis) is a hypothesis (thesis) aboutthe nature of computable functions. In simple terms, the ChurchTuring thesis states that a function on the naturalnumbers is computable in an informal sense (i.e., computable by a human being using a pencil-and-paper method,ignoring resource limitations) if and only if it is computable by a Turing machine. The thesis is named after Americanmathematician Alonzo Church and his Ph.D. student, the British mathematician Alan Turing.Before the precise denition of computable function, mathematicians often used the informal term eectively cal-culable to describe functions that are computable by paper-and-pencil methods. In the 1930s, several independentattempts were made to formalize the notion of computability:

In 1933, Austrian-American mathematician Kurt Gdel, with Jacques Herbrand, created a formal denitionof a class called general recursive functions. The class of general recursive functions is the smallest classof functions (possibly with more than one argument) which includes all constant functions, projections, thesuccessor function, and which is closed under function composition and recursion.

In 1936, Alonzo Church created a method for dening functions called the -calculus. Within -calculus, hedened an encoding of the natural numbers called the Church numerals. A function on the natural numbersis called -computable if the corresponding function on the Church numerals can be represented by a term ofthe -calculus.

Also in 1936, before learning of Churchs work, Alan Turing created a theoretical model for machines, nowcalled Turing machines, that could carry out calculations from inputs by manipulating symbols on a tape. Givena suitable encoding of the natural numbers as sequences of symbols, a function on the natural numbers is calledTuring computable if some Turing machine computes the corresponding function on encoded natural numbers.

Church [2] and Turing [3] proved that these three formally dened classes of computable functions coincide: a functionis -computable if and only if it is Turing computable if and only if it is general recursive. This has led mathemati-cians and computer scientists to believe that the concept of computability is accurately characterized by these threeequivalent processes. Other formal attempts to characterize computability have subsequently strengthened this belief(see below).On the other hand, the ChurchTuring thesis states that the above three formally-dened classes of computable func-tions coincide with the informal notion of an eectively calculable function. Since, as an informal notion, the conceptof eective calculability does not have a formal denition, the thesis, although it has near-universal acceptance, can-not be formally proven.Since its inception, variations on the original thesis have arisen, including statements about what can physicallybe realized by a computer in our universe (Physical Church-Turing Thesis) and what can be eciently computed(Complexity-Theoretic ChurchTuring Thesis). These variations are not due to Church or Turing, but arise fromlater work in complexity theory and digital physics. The thesis also has implications for the philosophy of mind.

2

2.1. STATEMENT IN CHURCHS AND TURINGS WORDS 3

2.1 Statement in Churchs and Turings wordsSee also: Eectively calculable

J. B. Rosser (1939) addresses the notion of eective computability as follows: Clearly the existence of CC andRC (Churchs and Rossers proofs) presupposes a precise denition of 'eective'. 'Eective method' is here used inthe rather special sense of a method each step of which is precisely predetermined and which is certain to producethe answer in a nite number of steps.[4] Thus the adverb-adjective eective is used in a sense of 1a: producinga decided, decisive, or desired eect, and capable of producing a result.[5][6]

In the following, the words eectively calculable will mean produced by any intuitively 'eective' means what-soever and eectively computable will mean produced by a Turing-machine or equivalent mechanical device.Turings denitions given in a footnote in his 1939 Ph.D. thesis Systems of Logic Based on Ordinals, supervised byChurch, are virtually the same:

" We shall use the expression 'computable function' to mean a function calculable by a machine, and let'eectively calculable' refer to the intuitive idea without particular identication with any one of thesedenitions.[7]

The thesis can be stated as follows:

Every eectively calculable function is a computable function.[8]

Turing stated it this way:

It was stated ... that 'a function is eectively calculable if its values can be found by some purelymechanical process.' We may take this literally, understanding that by a purely mechanical processone which could be carried out by a machine. The development ... leads to ... an identication ofcomputability with eective calculability. ( is the footnote above, ibid.)

2.2 HistoryMain article: History of the ChurchTuring thesis

One of the important problems for logicians in the 1930s was David Hilberts Entscheidungsproblem, which askedwhether there was a mechanical procedure for separating mathematical truths from mathematical falsehoods. Thisquest required that the notion of algorithm or eective calculability be pinned down, at least well enough for thequest to begin.[9] But from the very outset Alonzo Church's attempts began with a debate that continues to this day.[10]Was the notion of eective calculability to be (i) an axiom or axioms in an axiomatic system, or (ii) merely adenition that identied two or more propositions, or (iii) an empirical hypothesis to be veried by observation ofnatural events, or (iv) or just a proposal for the sake of argument (i.e. a thesis).

2.2.1 Circa 19301952In the course of studying the problem, Church and his student Stephen Kleene introduced the notion of -denablefunctions, and they were able to prove that several large classes of functions frequently encountered in number the-ory were -denable.[11] The debate began when Church proposed to Gdel that one should dene the eectivelycomputable functions as the -denable functions. Gdel, however, was not convinced and called the proposalthoroughly unsatisfactory.[12] Rather, in correspondence with Church (ca 19345), Gdel proposed axiomatizingthe notion of eective calculability"; indeed, in a 1935 letter to Kleene, Church reported that:

His [Gdels] only idea at the time was that it might be possible, in terms of eective calculability as anundened notion, to state a set of axioms which would embody the generally accepted properties of thisnotion, and to do something on that basis.[13]

4 CHAPTER 2. CHURCHTURING THESIS

But Gdel oered no further guidance. Eventually, he would suggest his (primitive) recursion, modied by Herbrandssuggestion, that Gdel had detailed in his 1934 lectures in Princeton NJ (Kleene and Rosser transcribed the notes).But he did not think that the two ideas could be satisfactorily identied except heuristically.[14]

Next, it was necessary to identify and prove the equivalence of two notions of eective calculability. Equippedwith the -calculus and general recursion, Stephen Kleene with help of Church and J. B. Rosser produced proofs(1933, 1935) to show that the two calculi are equivalent. Church subsequently modied his methods to includeuse of HerbrandGdel recursion and then proved (1936) that the Entscheidungsproblem is unsolvable: There is nogeneralized eective calculation (method, algorithm) that can determine whether or not a formula in either therecursive- or -calculus is valid (more precisely: no method to show that a well formed formula has a normalform).[15]

Many years later in a letter to Davis (ca 1965), Gdel would confess that he was, at the time of these [1934] lectures,not at all convinced that his concept of recursion comprised all possible recursions.[16] By 19634 Gdel woulddisavow HerbrandGdel recursion and the -calculus in favor of the Turing machine as the denition of algorithmor mechanical procedure or formal system.[17]

Ahypothesis leading to a natural law?: In late 1936 Alan Turing's paper (also proving that the Entscheidungsproblemis unsolvable) was delivered orally, but had not yet appeared in print.[18] On the other hand, Emil Post's 1936 paperhad appeared and was certied independent of Turings work.[19] Post strongly disagreed with Churchs identica-tion of eective computability with the -calculus and recursion, stating:

Actually the work already done by Church and others carries this identication considerably beyond theworking hypothesis stage. But to mask this identication under a denition ... blinds us to the need ofits continual verication.[20]

Rather, he regarded the notion of eective calculability as merely a working hypothesis that might lead byinductive reasoning to a "natural law" rather than by a denition or an axiom.[21] This idea was sharply criti-cized by Church.[22]

Thus Post in his 1936[13] paper was also discounting Kurt Gdel's suggestion to Church in 19345 that the thesismight be expressed as an axiom or set of axioms.[13]

Turing adds another denition, Rosser equates all three: Within just a short time, Turings 193637 paper OnComputable Numbers, with an Application to the Entscheidungsproblem[18] appeared. In it he stated another notionof eective computability with the introduction of his a-machines (now known as the Turing machine abstractcomputational model). And in a proof-sketch added as an Appendix to his 193637 paper, Turing showed thatthe classes of functions dened by -calculus and Turing machines coincided.[23] Church was quick to recognise howcompelling Turings analysis was. In his review of Turings paper[24] he made clear that Turings notion made theidentication with eectiveness in the ordinary (not explicitly dened) sense evident immediately.In a few years (1939) Turing would propose, like Church and Kleene before him, that his formal denition of mechan-ical computing agent was the correct one.[25] Thus, by 1939, both Church (1934) and Turing (1939) had individuallyproposed that their formal systems should be denitions of eective calculability";[26] neither framed their state-ments as theses.Rosser (1939) formally identied the three notions-as-denitions:

All three denitions are equivalent, so it does not matter which one is used.[27]

Kleene proposes Churchs Thesis: This left the overt expression of a thesis to Kleene. In his 1943 paper RecursivePredicates and Quantiers Kleene proposed his THESIS I":

This heuristic fact [general recursive functions are eectively calculable] ... led Church to state thefollowing thesis(22). The same thesis is implicit in Turings description of computing machines(23).

THESIS I. Every eectively calculable function (eectively decidable predicate) is general[28]recursive [Kleenes italics]

Since a precise mathematical denition of the term eectively calculable (eectively decidable) hasbeen wanting, we can take this thesis ... as a denition of it ...[29]

2.2. HISTORY 5

"(22) references Church 1936"(23) references Turing 19367

Kleene goes on to note that:

"... the thesis has the character of an hypothesisa point emphasized by Post and by Church(24). If weconsider the thesis and its converse as denition, then the hypothesis is an hypothesis about the applicationof the mathematical theory developed from the denition. For the acceptance of the hypothesis, thereare, as we have suggested, quite compelling grounds.[29]

"(24) references Post 1936 of Post and Churchs Formal denitions in the theoryof ordinal numbers, Fund. Math. vol 28 (1936) pp.1121 (see ref. #2, Davis1965:286).

Kleenes ChurchTuring Thesis: A few years later (1952) Kleene would overtly name, defend, and express the twotheses and then identify them (show equivalence) by use of his Theorem XXX:

Heuristic evidence and other considerations led Church 1936 to propose the following thesis.Thesis I. Every eectively calculable function (eectively decidable predicate) is general re-cursive.[30]

Theorem XXX: The following classes of partial functions are coextensive, i.e. have the same members:(a) the partial recursive functions, (b) the computable functions ....[31]Turings thesis: Turings thesis that every function which would naturally be regarded as computable iscomputable under his denition, i.e. by one of his machines, is equivalent to Churchs thesis by TheoremXXX.[31]

2.2.2 Later developmentsAn attempt to understand the notion of eective computability better led Robin Gandy (Turings student and friend)in 1980 to analyze machine computation (as opposed to human-computation acted out by a Turing machine). Gandyscuriosity about, and analysis of, "cellular automata", "Conways game of life", parallelism and crystalline automataled him to propose four principles (or constraints) ... which it is argued, any machine must satisfy.[32] His most-important fourth, the principle of causality is based on the nite velocity of propagation of eects and signals;contemporary physics rejects the possibility of instantaneous action at a distance.[33] From these principles andsome additional constraints(1a) a lower bound on the linear dimensions of any of the parts, (1b) an upper bound onspeed of propagation (the velocity of light), (2) discrete progress of the machine, and (3) deterministic behaviorheproduces a theorem that What can be calculated by a device satisfying principles IIV is computable.[34] ".In the late 1990s Wilfried Sieg analyzed Turings and Gandys notions of eective calculability with the intentof sharpening the informal notion, formulating its general features axiomatically, and investigating the axiomaticframework.[35] In his 1997 and 2002 Sieg presents a series of constraints on the behavior of a computor"a humancomputing agent who proceeds mechanically"; these constraints reduce to:

"(B.1) (Boundedness) There is a xed bound on the number of symbolic congurations a computor can imme-diately recognize.

"(B.2) (Boundedness) There is a xed bound on the number of internal states a computor can be in. "(L.1) (Locality) A computor can change only elements of an observed symbolic conguration. "(L.2) (Locality) A computor can shift attention from one symbolic conguration to another one, but the newobserved congurations must be within a bounded distance of the immediately previously observed conguration.

"(D) (Determinacy) The immediately recognizable (sub-)conguration determines uniquely the next computationstep (and id [instantaneous description] )"; stated another way: "A computors internal state together with theobserved conguration xes uniquely the next computation step and the next internal state.[36]

The matter remains in active discussion within the academic community.[37][38]

6 CHAPTER 2. CHURCHTURING THESIS

2.2.3 The thesis as a denition

The thesis can be viewed as nothing but an ordinary mathematical denition. Comments by Gdel on the subjectsuggest this view, e.g. the correct denition of mechanical computability was established beyond any doubt byTuring.[39] The case for viewing the thesis as nothing more than a denition is made explicitly by Robert I. Soare in[40] where it is also argued that Turings denition of computability is no less likely to be correct than the epsilon-deltadenition of a continuous function.

2.3 Success of the thesisOther formalisms (besides recursion, the -calculus, and the Turing machine) have been proposed for describingeective calculability/computability. Stephen Kleene (1952) adds to the list the functions "reckonable in the systemS1" of Kurt Gdel 1936, and Emil Post's (1943, 1946) "canonical [also called normal] systems".[41] In the 1950s HaoWang and Martin Davis greatly simplied the one-tape Turing-machine model (see PostTuring machine). MarvinMinsky expanded the model to two or more tapes and greatly simplied the tapes into up-down counters, whichMelzak and Lambek further evolved into what is now known as the counter machine model. In the late 1960s andearly 1970s researchers expanded the counter machine model into the register machine, a close cousin to the modernnotion of the computer. Other models include combinatory logic and Markov algorithms. Gurevich adds the pointermachine model of Kolmogorov and Uspensky (1953, 1958): "... they just wanted to ... convince themselves thatthere is no way to extend the notion of computable function.[42]

All these contributions involve proofs that the models are computationally equivalent to the Turing machine; suchmodels are said to be Turing complete. Because all these dierent attempts at formalizing the concept of eectivecalculability/computability have yielded equivalent results, it is now generally assumed that the ChurchTuring the-sis is correct. In fact, Gdel (1936) proposed something stronger than this; he observed that there was somethingabsolute about the concept of reckonable in S1":

It may also be shown that a function which is computable ['reckonable'] in one of the systems S, or evenin a system of transnite type, is already computable [reckonable] in S1. Thus the concept 'computable'['reckonable'] is in a certain denite sense 'absolute', while practically all other familiar metamathemat-ical concepts (e.g. provable, denable, etc.) depend quite essentially on the system to which they aredened[43]

2.4 Informal usage in proofsProofs in computability theory often invoke[44] the ChurchTuring thesis in an informal way to establish the com-putability of functions while avoiding the (often very long) details which would be involved in a rigorous, formalproof. To establish that a function is computable by Turing machine, it is usually considered sucient to give aninformal English description of how the function can be eectively computed, and then conclude By the ChurchTuring thesis that the function is Turing computable (equivalently partial recursive).Dirk van Dalen (in Gabbay 2001:284[45]) gives the following example for the sake of illustrating this informal use ofthe ChurchTuring thesis:

EXAMPLE: Each innite RE set contains an innite recursive set.

Proof: Let A be innite RE. We list the elements of A eectively, n0, n1, n2, n3, ...From this list we extract an increasing sublist: put m0=n0, after nitely many steps we nd an n suchthat n > m0, put m1=n. We repeat this procedure to nd m2 > m1, etc. this yields an eective listingof the subset B={m0,m1,m2,...} of A, with the property m < m.Claim. B is decidable. For, in order to test k in B we must check if k=m for some i. Since the sequenceof m's is increasing we have to produce at most k+1 elements of the list and compare them with k. Ifnone of them is equal to k, then k not in B. Since this test is eective, B is decidable and, by Churchsthesis, recursive.

2.5. VARIATIONS 7

(Emphasis added). In order to make the above example completely rigorous, one would have to carefully construct aTuring Machine, or -function, or carefully invoke recursion axioms, or at best, cleverly invoke various theorems ofcomputability theory. But because the computability theorist believes that Turing computability correctly captureswhat can be computed eectively, and because an eective procedure is spelled out in English for deciding the set B,the computability theorist accepts this as proof that the set is indeed recursive.As a rule of thumb, the ChurchTuring thesis should only be invoked to simplify proofs in cases where the writer wouldbe capable of, and expects the readers also to be capable of, easily (but not necessarily without tedium) producing arigorous proof if one were demanded.

2.5 VariationsThe success of the ChurchTuring thesis prompted variations of the thesis to be proposed. For example, the PhysicalChurchTuring thesis (PCTT) states:

All physically computable functions are Turing-computable[46]

The ChurchTuring thesis says nothing about the eciency with which one model of computation can simulateanother. It has been proved for instance that a (multi-tape) universal Turing machine only suers a logarithmicslowdown factor in simulating any Turing machine.[47] No such result has been proved in general for an arbitrary butreasonable model of computation. A variation of the ChurchTuring thesis that addresses this issue is the FeasibilityThesis[48] or (Classical) Complexity-Theoretic ChurchTuring Thesis (SCTT), which is not due to Church orTuring, but rather was realized gradually in the development of complexity theory. It states:[49]

"A probabilistic Turing machine can eciently simulate any realistic model of computation."

The word 'eciently' here means up to polynomial-time reductions. This thesis was originally called Computa-tional Complexity-Theoretic ChurchTuring Thesis by Ethan Bernstein and Umesh Vazirani (1997). The Complexity-Theoretic ChurchTuring Thesis, then, posits that all 'reasonable' models of computation yield the same class ofproblems that can be computed in polynomial time. Assuming the conjecture that probabilistic polynomial time(BPP) equals deterministic polynomial time (P), the word 'probabilistic' is optional in the Complexity-TheoreticChurchTuring Thesis. A similar thesis, called the Invariance Thesis, was introduced by Cees F. Slot and Peter vanEmde Boas. It states: Reasonable machines can simulate each other within a polynomially bounded overhead intime and a constant-factor overhead in space.[50] The thesis originally appeared in a paper at STOC'84, which wasthe rst paper to show that polynomial-time overhead and constant-space overhead could be simultaneously achievedfor a simulation of a Random Access Machine on a Turing machine.[51]

If BQP is shown to be a strict superset of BPP, it would invalidate the Complexity-Theoretic ChurchTuring Thesis.In other words, there would be ecient quantum algorithms that perform tasks that do not have ecient probabilisticalgorithms. This would not however invalidate the original ChurchTuring thesis, since a quantum computer canalways be simulated by a Turing machine, but it would invalidate the classical Complexity-Theoretic ChurchTuringthesis for eciency reasons. Consequently, the Quantum Complexity-Theoretic ChurchTuring thesis states:[49]

A quantum Turing machine can eciently simulate any realistic model of computation.

Eugene Eberbach and Peter Wegner[52] claim that the ChurchTuring thesis is sometimes interpreted too broadly,stating the broader assertion that algorithms precisely capture what can be computed is invalid. They claim thatforms of computation not captured by the thesis are relevant today, terms which they call super-Turing computation.

2.6 Philosophical implicationsPhilosophers have interpreted the ChurchTuring thesis as having implications for the philosophy of mind; however,many of the philosophical interpretations of the Thesis involve basic misunderstandings of the thesis statement.[53]B. Jack Copeland states that its an open empirical question whether there are actual deterministic physical processesthat, in the long run, elude simulation by a Turing machine; furthermore, he states that it is an open empirical ques-tion whether any such processes are involved in the working of the human brain.[54] There are also some important

8 CHAPTER 2. CHURCHTURING THESIS

open questions which cover the relationship between the ChurchTuring thesis and physics, and the possibility ofhypercomputation. When applied to physics, the thesis has several possible meanings:

1. The universe is equivalent to a Turing machine; thus, computing non-recursive functions is physically impos-sible. This has been termed the Strong ChurchTuring thesis and is a foundation of digital physics.

2. The universe is not equivalent to a Turing machine (i.e., the laws of physics are not Turing-computable), butincomputable physical events are not harnessable for the construction of a hypercomputer. For example, auniverse in which physics involves real numbers, as opposed to computable reals, might fall into this category.The assumption that incomputable physical events are not harnessable has been challenged, however,[55] bya proposed computational process that uses quantum randomness together with a computational machine tohide the computational steps of a Universal Turing Machine with Turing-incomputable ring patterns.

3. The universe is a hypercomputer, and it is possible to build physical devices to harness this property and cal-culate non-recursive functions. For example, it is an open question whether all quantum mechanical events areTuring-computable, although it is known that rigorous models such as quantum Turing machines are equiva-lent to deterministic Turing machines. (They are not necessarily eciently equivalent; see above.) John Lucasand Roger Penrose[56] have suggested that the human mind might be the result of some kind of quantum-mechanically enhanced, non-algorithmic computation, although there is no scientic evidence for this pro-posal.

There are many other technical possibilities which fall outside or between these three categories, but these serve toillustrate the range of the concept.

2.7 Non-computable functionsOne can formally dene functions that are not computable. A well-known example of such a function is the BusyBeaver function. This function takes an input n and returns the largest number of symbols that a Turing machinewith n states can print before halting, when run with no input. Finding an upper bound on the busy beaver functionis equivalent to solving the halting problem, a problem known to be unsolvable by Turing machines. Since the busybeaver function cannot be computed by Turing machines, the ChurchTuring thesis states that this function cannotbe eectively computed by any method.Several computational models allow for the computation of (Church-Turing) non-computable functions. These areknown as hypercomputers. Mark Burgin[57] argues that super-recursive algorithms such as inductive Turing machinesdisprove the ChurchTuring thesis. His argument relies on a denition of algorithm broader than the ordinary one,so that non-computable functions obtained from some inductive Turing machines are called computable. This inter-pretation of the ChurchTuring thesis diers from the interpretation commonly accepted in computability theory,discussed above. The argument that super-recursive algorithms are indeed algorithms in the sense of the ChurchTuring thesis has not found broad acceptance within the computability research community.

2.8 See also Abstract machine Churchs thesis in constructive mathematics Computability logic Computability theory Decidability History of the ChurchTuring thesis Hypercomputer Model of computation

2.9. FOOTNOTES 9

Super-recursive algorithm ChurchTuringDeutsch principle, which states that every physical process can be simulated by a universal

computing device

stochastics Monte Carlo method

2.9 Footnotes[1] Rabin, Michael O. (June 2012). Turing, Church, Gdel, Computability, Complexity and Randomization: A Personal View.

[2] Church 1934:90 footnote in Davis 1952

[3] Turing 19367 in Davis 1952:149

[4] Rosser 1939 in Davis 1965:225

[5] Merriam Websters Ninth New Collegiate Dictionary

[6] See also Merriam-Websters Online Dictionary, 11th Edition (accessed July 26, 2014), which also gives these denitionsfor eective -- the rst ["producing a decided, decisive, or desired eect"] as the denition for sense 1a of the wordeective, and the second ["capable of producing a result"] as part of the Synonym Discussion of EFFECTIVE there,(in the introductory part, where it summarizes the similarities between the meanings of the words eective, eectual,ecient, and ecacious).

[7] A. M. Turing (1939), Systems of Logic Based on Ordinals (Ph.D. thesis). Princeton University. p. 8.

[8] Gandy (Gandy 1980 in Barwise 1980:123) states it this way: What is eectively calculable is computable. He calls thisChurchs Thesis, a peculiar choice of moniker.

[9] Daviss commentary before Church 1936 An Unsolvable Problem of Elementary Number Theory in Davis 1965:88. Churchuses the words eective calculability on page 100.

[10] In his review of Churchs Thesis after 70 Years edited by Adam Olszewski et al. 2006, Peter Smiths criticism of a paper byMuraswski and Wolenski suggests 4 lines re the status of the ChurchTuring Thesis: (1) empirical hypothesis (2) axiomor theorem, (3) denition, (4) explication. But Smith opines that (4) is indistinguishable from (3), cf Smith (July 11, 2007)Churchs Thesis after 70 Years at http://www.logicmatters.net/resources/pdfs/CTT.pdf

[11] cf footnote 3 in Church 1936 An Unsolvable Problem of Elementary Number Theory in Davis 1965:89

[12] Dawson 1997:99

[13] Sieg 1997:160

[14] Sieg 1997:160 quoting from the 1935 letter written by Church to Kleene, cf Footnote 3 in Gdel 1934 in Davis 1965:44

[15] cf Church 1936 in Davis 1965:105

[16] Daviss commentary before Gdel 1934 in Davis 1965:40

[17] For a detailed discussion of Gdels adoption of Turings machines as models of computation, see Shagrir date TBD athttp://edelstein.huji.ac.il/staff/shagrir/papers/Goedel_on_Turing_on_Computability.pdf

[18] Turing 1937

[19] cf. Editors footnote to Post 1936 Finite Combinatory Process. Formulation I. at Davis 1965:289.

[20] Post 1936 in Davis 1965:291 footnote 8

[21] Post 1936 in Davis 1952:291

[22] Sieg 1997:171 and 1767

[23] Turing 19367 in Davis 1965:263

[24] Church 1937

10 CHAPTER 2. CHURCHTURING THESIS

[25] Turing 1939 in Davis:160

[26] cf. Church 1934 in Davis 1965:100, also Turing 1939 in Davis 1965:160

[27] italics added, Rosser 1939 in Davis 1965:226

[28] An archaic usage of Kleene et al. to distinguish Gdels (1931) rekursiv (a few years later named primitive recursionby Rzsa Pter (cf Gandy 1994 in Herken 19945:68)) from HerbrandGdels recursion of 1934 i.e. primitive recursionequipped with the additional mu operator; nowadays mu-recursion is called, simply, "recursion".

[29] Kleene 1943 in Davis 1965:274

[30] Kleene 1952:300

[31] Kleene 1952:376

[32] Gandy 1980 in Barwise 1980:123)

[33] Gandy 1980 in Barwise 1980:135

[34] Gandy 1980 in Barwise:126

[35] (Sieg 19989 in SiegSomnerTalcott 2002:390; also Sieg 1997:154)

[36] In a footnote Sieg breaks Posts 1936 (B) into (B.1) and (B.2) and (L) into (L.1) and (L.2) and describes (D) dierently.With respect to his proposed Gandy machine he later adds LC.1, LC.2, GA.1 and GA.2. These are complicated; see Sieg19989 in SiegSomnerTalcott 2002:390.

[37] A collection of papers can be found at Churchs Thesis after 70 Years edited by Adam Olszewski et al. 2006. Also a reviewof this collection by Peter Smith (July 11, 2007) Churchs Thesis after 70 Years at http://www.logicmatters.net/resources/pdfs/CTT.pdf

[38] See also Hodges, Andrew (2005). Did Church and Turing Have a Thesis about Machines?" (PDF). Archived from theoriginal on July 27, 2014. Retrieved July 27, 2014.

[39] Gdel, K. [193?], Undecidable Diophantine Propositions, in Collected Works, III, p. 168.

[40] R. I. Soare, 1996, Computability and Recursion, Bulletin of Symbolic Logic v. 2 pp. 284321.

[41] Kleene 1952:320

[42] Gurevich 1988:2

[43] translation of Gdel (1936) by Davis in TheUndecidable p. 83, diering in the use of the word 'reckonable' in the translationin Kleene (1952) p. 321

[44] Horsten in Olszewski 2006:256

[45] Gabbay 2001:284

[46] Piccinini 2007:101 Computationalism, the ChurchTuring Thesis, and the ChurchTuring Fallacy. doi:10.1007/s11229-005-0194-z. in Synthese (2007) 154:97120.

[47] Arora, Sanjeev; Barak, Boaz, Complexity Theory: A Modern Approach, Cambridge University Press, 2009, ISBN 978-0-521-42426-4, section 1.4, Machines as strings and the universal Turing machine and 1.7, Proof of theorem 1.9

[48] http://www.claymath.org/millennium/P_vs_NP/Official_Problem_Description.pdf

[49] Phillip Kaye, Raymond Laamme, Michele Mosca, An introduction to quantum computing, Oxford University Press, 2007,ISBN 0-19-857049-X, pp. 56

[50] Peter van Emde Boass, Machine Models and Simulations, in Handbook of Theoretical Computer Science A, Elsevier, 1990,p. 5

[51] C. Slot, P. van Emde Boas, On tape versus core: an application of space ecient perfect hash functions to the invariance ofspace, STOC, December 1984

[52] Eberbach and Wegner, 2003

[53] In particular, see the numerous examples (of errors, of misappropriation of the thesis) at the entry in the Stanford Encyclo-pedia of Philosophy. For a good place to encounter original papers see David J. Chalmers, ed. 2002, Philosophy of Mind:Classical and Contemporary Readings, Oxford University Press, New York.

2.10. REFERENCES 11

[54] B. Jack Copeland,Computation in Luciano Floridi (ed.), The Blackwell guide to the philosophy of computing and information,Wiley-Blackwell, 2004, ISBN 0-631-22919-1, p. 15

[55] Michael Fiske, Turing Incomputable Computation in Turing-100 proceedings, The Alan Turing Centenary. http://www.easychair.org/publications/?page=1303694832.

[56] cf his subchapter The ChurchTuring Thesis (p. 4749) in his chapter Algorithms and Turing machines in his 1990(2nd edition) Emperors New Mind: Concerning Computers, Minds, and the Laws of Physics, Oxford University Press,Oxford UK. Also his a nal chapter titled Where lies the physics of mind?" where, in a subsection he describes Thenon-algorithmic nature of mathematical insight (p. 4168).

[57] Super-Recursive Algorithms (Monographs in Computer Science), Springer, 2005. ISBN 0-387-95569-0

2.10 References Barwise, Jon, H. J. Keisler, and K. Kunen, Editors, 1980, The Kleene Symposium, 426 pages, North-Holland

Publishing Company, Amsterdam, ISBN 0-444-85345-6

Ben-Amram, A.M. (2005). The Church-Turing Thesis and its Look-Alikes. SIGACT News 36 (3): 113116.doi:10.1145/1086649.1086651.

Bernstein, E; Vazirani, U. (1997). Quantum complexity theory. SIAM Journal on Computing 26 (5): 14111473. doi:10.1137/S0097539796300921.

Blass, Andreas; Yuri Gurevich (2003). Algorithms: A Quest for Absolute Denitions (PDF). Bulletin ofEuropean Association for Theoretical Computer Science (81).

Burgin, Mark (2005). Super-recursive algorithms. Monographs in computer science. Springer. ISBN 0-387-95569-0.

Church, Alonzo (1932). A set of Postulates for the Foundation of Logic. Annals of Mathematics 33 (2):346366. doi:10.2307/1968337. JSTOR 1968337.

Church, Alonzo (1936). An Unsolvable Problem of Elementary Number Theory. American Journal ofMathematics 58 (58): 345363. doi:10.2307/2371045. JSTOR 2371045.

Church, Alonzo (1936). A Note on the Entscheidungsproblem. Journal of Symbolic Logic (1): 4041. Church, Alonzo (1937). Review: A. M. Turing, On Computable Numbers, with an Application to the

Entscheidungsproblem. Journal of Symbolic Logic (2): 4243.

Church, Alonzo (1941). The Calculi of Lambda-Conversion. Princeton: Princeton University Press. Cooper, S. B.; Odifreddi, P. (2003). Incomputability in Nature. In S. B. Cooper & S. S. Goncharov. Com-putability and Models: Perspectives East and West. Kluwer Academic/Plenum Publishers. pp. 137160.

Davis, Martin, ed. (1965). The Undecidable, Basic Papers on Undecidable Propositions, Unsolvable ProblemsAnd Computable Functions. New York: Raven Press. Includes original papers by Gdel, Church, Turing,Rosser, Kleene, and Post mentioned in this section.

Eberbach, E.; Wegner, P. (October 2003). Beyond Turing Machines. Bulletin of the European Associationfor Theoretical Computer Science (81): 279304.

Gabbay, D.M. (2001). Handbook of Philosophical Logic 1 (2nd ed.). Gandy, Robin (1980). Churchs Thesis and the Principles for Mechanisms. In H.J. Barwise, H.J. Keisler,

and K. Kunen. The Kleene Symposium. North-Holland Publishing Company. pp. 123148.

Gandy, Robin (19945). Rolf Herken, ed. The universal Turing Machine: A Half-Century Survey. New York:Wien SpringerVerlag. pp. 51. ISBN 3-211-82637-8. Check date values in: |date= (help)

Gdel, Kurt (1965) [1934]. On Undecidable Propositions of Formal Mathematical Systems. In Davis, M.The Undecidable. Kleene and Rosser (lecture note-takers); Institute for Advanced Study (lecture sponsor).New York: Raven Press.

12 CHAPTER 2. CHURCHTURING THESIS

Gdel, Kurt (1936). On The Length of Proofs. Ergenbnisse eines mathematishen Kolloquiums (in German)(Heft) (7): 2324. Cited by Kleene (1952) as "ber die Lange von Beweisen, in Ergebnisse eines math. Koll,etc.

Gurevich, Yuri (June 1988). On Kolmogorov Machines and Related Issues. Bulletin of European Associationfor Theoretical Computer Science (35): 7182.

Gurevich, Yuri (July 2000). Sequential Abstract State Machines Capture Sequential Algorithms (PDF).ACMTransactions on Computational Logic 1 (1): 77111. doi:10.1145/343369.343384.

Herbrand, Jacques (1932). Sur la non-contradiction de l'arithmtique. Journal fur die reine und angewandteMathematik (166): 18.

Hofstadter, Douglas R.. Chapter XVII: Church, Turing, Tarski, and Others. Gdel, Escher, Bach: an EternalGolden Braid.

Kleene, Stephen Cole (1935). A Theory of Positive Integers in Formal Logic. American Journal of Mathe-matics 57 (57): 153173 & 219244. doi:10.2307/2372027. JSTOR 2372027.

Kleene, Stephen Cole (1936). Lambda-Denability and Recursiveness. Duke Mathematical Journal (2):340353.

Kleene, Stephen Cole (1943). Recursive Predicates and Quantiers. American Mathematical Society Trans-actions (Transactions of the American Mathematical Society, Vol. 53, No. 1) 54 (1): 4173. doi:10.2307/1990131.JSTOR 1990131. Reprinted in The Undecidable, p. 255. Kleene rened his denition of general recursionand proceeded in his chapter 12. Algorithmic theories to posit Thesis I (p. 274); he would later repeat thisthesis (in Kleene 1952:300) and name it Churchs Thesis (Kleene 1952:317) (i.e., the Church thesis).

Kleene, Stephen Cole (1952). Introduction to Metamathematics. North-Holland. OCLC 523942. Knuth, Donald (1973). The Art of Computer Programming. 1/Fundamental Algorithms (2nd ed.). Addison

Wesley. Kugel, Peter (November 2005). Communications of the ACM. Its time to think outside the computationalbox 48 (11).

Lewis, H.R.; Papadimitriou, C.H. (1998). Elements of the Theory of Computation. Upper Saddle River, NJ,USA: Prentice-Hall.

Manna, Zohar (1974) [2003]. Mathematical Theory of Computation. Dover. ISBN 978-0-486-43238-0. Markov, A.A. (1960) [1954]. The Theory of Algorithms. American Mathematical Society Translations 2

(15): 114. Olszewski, Adam (2006). Churchs Thesis After 70 Years. Pour-El, M.B.; Richards, J.I. (1989). Computability in Analysis and Physics. Springer Verlag. Rosser, J. B. (1939). An Informal Exposition of Proofs of Godels Theorem and Churchs Theorem. TheJournal of Symbolic Logic (The Journal of Symbolic Logic, Vol. 4, No. 2) 4 (2): 5360. doi:10.2307/2269059.JSTOR 2269059.

Sieg, Wilfried, Richard Sommer, and Carolyn Talcott (eds.), 2002, Reections on the Foundations of Mathe-matics: Essays in Honor of Solomon Feferman, Lecture Notes in Logic 15, 444 pages, A K Peters, Ltd., ISBN1-56881-169-1

Soare, Robert (1996). Computability and Recursion. Bulletin of Symbolic Logic (2): 284321. Syropoulos, Apostolos (2008). Hypercomputation: Computing Beyond the ChurchTuring Barrier. Springer.

ISBN 978-0-387-30886-9. Turing, A. M. (1937) [Delivered to the Society November 1936], On Computable Numbers, with an Appli-

cation to the Entscheidungsproblem (PDF), Proceedings of the London Mathematical Society, 2 42: 23065,doi:10.1112/plms/s2-42.1.230 and Turing, A.M. (1938). On Computable Numbers, with an Application tothe Entscheidungsproblem: A correction. Proceedings of the London Mathematical Society. 2 43 (1937). pp.5446. doi:10.1112/plms/s2-43.6.544. (See also: Davis 1965:115)

2.11. EXTERNAL LINKS 13

2.11 External links The ChurchTuring Thesis entry by B. Jack Copeland in the Stanford Encyclopedia of Philosophy. Computation in Physical Systems a comprehensive philosophical treatment of relevant issues Transcendental idealism and Posts variant of the Church-Turing thesis EGTheory blog

Chapter 3

Computability theory

For the concept of computability, see Computability.

Computability theory, also called recursion theory, is a branch of mathematical logic, of computer science, andof the theory of computation that originated in the 1930s with the study of computable functions and Turing degrees.The basic questions addressed by recursion theory are What does it mean for a function on the natural numbersto be computable?" and How can noncomputable functions be classied into a hierarchy based on their level ofnoncomputability?". The answers to these questions have led to a rich theory that is still being actively researched.The eld has since grown to include the study of generalized computability and denability. Invention of the cen-tral combinatorial object of recursion theory, namely the Universal Turing Machine, predates and predetermines theinvention of modern computers. Historically, the study of algorithmically undecidable sets and functions was moti-vated by various problems in mathematics that turned to be undecidable; for example, word problem for groups andthe like. There are several applications of the theory to other branches of mathematics that do not necessarily con-centrate on undecidability. The early applications include the celebrated Higmans embedding theorem that providesa link between recursion theory and group theory, results of Michael O. Rabin and Anatoly Maltsev on algorithmicpresentations of algebras, and the negative solution to Hilberts Tenth Problem. The more recent applications in-clude algorithmic randomness, results of Soare et al. who applied recursion-theoretic methods to solve a problem inalgebraic geometry,[1] and the very recent work of Slaman et al. on normal numbers that solves a problem in analyticnumber theory.Recursion theory overlaps with proof theory, eective descriptive set theory, model theory, and abstract algebra. Ar-guably, computational complexity theory is a child of recursion theory; both theories share the same technical tool,namely the Turing Machine. Recursion theorists in mathematical logic often study the theory of relative computabil-ity, reducibility notions and degree structures described in this article. This contrasts with the theory of subrecursivehierarchies, formal methods and formal languages that is common in the study of computability theory in computerscience. There is a considerable overlap in knowledge and methods between these two research communities; how-ever, no rm line can be drawn between them. For instance, parametrized complexity was invented by a complexitytheorist Michael Fellows and a recursion theorist Rod Downey.

3.1 Computable and uncomputable setsRecursion theory originated in the 1930s, with work of Kurt Gdel, Alonzo Church, Alan Turing, Stephen Kleeneand Emil Post.[2]

The fundamental results the researchers obtained established Turing computability as the correct formalization ofthe informal idea of eective calculation. These results led Stephen Kleene (1952) to coin the two names Churchsthesis (Kleene 1952:300) and Turings Thesis (Kleene 1952:376). Nowadays these are often considered as asingle hypothesis, the ChurchTuring thesis, which states that any function that is computable by an algorithm is acomputable function. Although initially skeptical, by 1946 Gdel argued in favor of this thesis:

Tarski has stressed in his lecture (and I think justly) the great importance of the concept of generalrecursiveness (or Turings computability). It seems to me that this importance is largely due to the fact

14

3.2. TURING COMPUTABILITY 15

that with this concept one has for the rst time succeeded in giving an absolute notion to an interest-ing epistemological notion, i.e., one not depending on the formalism chosen.*"(Gdel 1946 in Davis1965:84).[3]

With a denition of eective calculation came the rst proofs that there are problems in mathematics that cannot beeectively decided. Church (1936a, 1936b) and Turing (1936), inspired by techniques used by Gdel (1931) to provehis incompleteness theorems, independently demonstrated that the Entscheidungsproblem is not eectively decidable.This result showed that there is no algorithmic procedure that can correctly decide whether arbitrary mathematicalpropositions are true or false.Many problems of mathematics have been shown to be undecidable after these initial examples were established.In 1947, Markov and Post published independent papers showing that the word problem for semigroups cannot beeectively decided. Extending this result, Pyotr Novikov and William Boone showed independently in the 1950sthat the word problem for groups is not eectively solvable: there is no eective procedure that, given a word ina nitely presented group, will decide whether the element represented by the word is the identity element of thegroup. In 1970, Yuri Matiyasevich proved (using results of Julia Robinson) Matiyasevichs theorem, which impliesthat Hilberts tenth problem has no eective solution; this problem asked whether there is an eective procedureto decide whether a Diophantine equation over the integers has a solution in the integers. The list of undecidableproblems gives additional examples of problems with no computable solution.The study of which mathematical constructions can be eectively performed is sometimes called recursive math-ematics; the Handbook of Recursive Mathematics (Ershov et al. 1998) covers many of the known results in thiseld.

3.2 Turing computabilityThe main form of computability studied in recursion theory was introduced by Turing (1936). A set of naturalnumbers is said to be a computable set (also called a decidable, recursive, or Turing computable set) if there is aTuring machine that, given a number n, halts with output 1 if n is in the set and halts with output 0 if n is not in theset. A function f from the natural numbers to themselves is a recursive or (Turing) computable function if there is aTuring machine that, on input n, halts and returns output f(n). The use of Turing machines here is not necessary;there are many other models of computation that have the same computing power as Turing machines; for examplethe -recursive functions obtained from primitive recursion and the operator.The terminology for recursive functions and sets is not completely standardized. The denition in terms of -recursivefunctions as well as a dierent denition of rekursiv functions by Gdel led to the traditional name recursive for sets andfunctions computable by a Turing machine. The word decidable stems from the German word Entscheidungsproblemwhich was used in the original papers of Turing and others. In contemporary use, the term computable functionhas various denitions: according to Cutland (1980), it is a partial recursive function (which can be undened forsome inputs), while according to Soare (1987) it is a total recursive (equivalently, general recursive) function. Thisarticle follows the second of these conventions. Soare (1996) gives additional comments about the terminology.Not every set of natural numbers is computable. The halting problem, which is the set of (descriptions of) Turingmachines that halt on input 0, is a well-known example of a noncomputable set. The existence of many noncom-putable sets follows from the facts that there are only countably many Turing machines, and thus only countably manycomputable sets, but there are uncountably many sets of natural numbers.Although the halting problem is not computable, it is possible to simulate program execution and produce an innitelist of the programs that do halt. Thus the halting problem is an example of a recursively enumerable set, which is a setthat can be enumerated by a Turing machine (other terms for recursively enumerable include computably enumerableand semidecidable). Equivalently, a set is recursively enumerable if and only if it is the range of some computablefunction. The recursively enumerable sets, although not decidable in general, have been studied in detail in recursiontheory.

3.3 Areas of researchBeginning with the theory of recursive sets and functions described above, the eld of recursion theory has grown toinclude the study of many closely related topics. These are not independent areas of research: each of these areas

16 CHAPTER 3. COMPUTABILITY THEORY

draws ideas and results from the others, and most recursion theorists are familiar with the majority of them.

3.3.1 Relative computability and the Turing degreesMain articles: Turing reduction and Turing degree

Recursion theory in mathematical logic has traditionally focused on relative computability, a generalization of Turingcomputability dened using oracle Turing machines, introduced by Turing (1939). An oracle Turing machine is ahypothetical device which, in addition to performing the actions of a regular Turing machine, is able to ask questionsof an oracle, which is a particular set of natural numbers. The oracle machine may only ask questions of the form Isn in the oracle set?". Each question will be immediately answered correctly, even if the oracle set is not computable.Thus an oracle machine with a noncomputable oracle will be able to compute sets that a Turing machine without anoracle cannot.Informally, a set of natural numbers A is Turing reducible to a set B if there is an oracle machine that correctly tellswhether numbers are in A when run with B as the oracle set (in this case, the set A is also said to be (relatively)computable from B and recursive in B). If a set A is Turing reducible to a set B and B is Turing reducible to A thenthe sets are said to have the same Turing degree (also called degree of unsolvability). The Turing degree of a set givesa precise measure of how uncomputable the set is.The natural examples of sets that are not computable, including many dierent sets that encode variants of the haltingproblem, have two properties in common:

1. They are recursively enumerable, and

2. Each can be translated into any other via a many-one reduction. That is, given such sets A and B, there is atotal computable function f such that A = {x : f(x) B}. These sets are said to be many-one equivalent (orm-equivalent).

Many-one reductions are stronger than Turing reductions: if a set A is many-one reducible to a set B, then A isTuring reducible to B, but the converse does not always hold. Although the natural examples of noncomputable setsare all many-one equivalent, it is possible to construct recursively enumerable sets A and B such that A is Turingreducible to B but not many-one reducible to B. It can be shown that every recursively enumerable set is many-one reducible to the halting problem, and thus the halting problem is the most complicated recursively enumerableset with respect to many-one reducibility and with respect to Turing reducibility. Post (1944) asked whether everyrecursively enumerable set is either computable or Turing equivalent to the halting problem, that is, whether there isno recursively enumerable set with a Turing degree intermediate between those two.As intermediate results, Post dened natural types of recursively enumerable sets like the simple, hypersimple andhyperhypersimple sets. Post showed that these sets are strictly between the computable sets and the halting problemwith respect to many-one reducibility. Post also showed that some of them are strictly intermediate under otherreducibility notions stronger than Turing reducibility. But Post left open the main problem of the existence of re-cursively enumerable sets of intermediate Turing degree; this problem became known as Posts problem. After tenyears, Kleene and Post showed in 1954 that there are intermediate Turing degrees between those of the computablesets and the halting problem, but they failed to show that any of these degrees contains a recursively enumerable set.Very soon after this, Friedberg and Muchnik independently solved Posts problem by establishing the existence ofrecursively enumerable sets of intermediate degree. This groundbreaking result opened a wide study of the Turingdegrees of the recursively enumerable sets which turned out to possess a very complicated and non-trivial structure.There are uncountably many sets that are not recursively enumerable, and the investigation of the Turing degreesof all sets is as central in recursion theory as the investigation of the recursively enumerable Turing degrees. Manydegrees with special properties were constructed: hyperimmune-free degreeswhere every function computable relativeto that degree is majorized by a (unrelativized) computable function; high degrees relative to which one can computea function f which dominates every computable function g in the sense that there is a constant c depending on g suchthat g(x) < f(x) for all x > c; random degrees containing algorithmically random sets; 1-generic degrees of 1-genericsets; and the degrees below the halting problem of limit-recursive sets.The study of arbitrary (not necessarily recursively enumerable) Turing degrees involves the study of the Turing jump.Given a set A, the Turing jump of A is a set of natural numbers encoding a solution to the halting problem for oracleTuring machines running with oracle A. The Turing jump of any set is always of higher Turing degree than the

3.3. AREAS OF RESEARCH 17

original set, and a theorem of Friedburg shows that any set that computes the Halting problem can be obtained as theTuring jump of another set. Posts theorem establishes a close relationship between the Turing jump operation andthe arithmetical hierarchy, which is a classication of certain subsets of the natural numbers based on their denabilityin arithmetic.Much recent research on Turing degrees has focused on the overall structure of the set of Turing degrees and the setof Turing degrees containing recursively enumerable sets. A deep theorem of Shore and Slaman (1999) states that thefunction mapping a degree x to the degree of its Turing jump is denable in the partial order of the Turing degrees.A recent survey by Ambos-Spies and Fejer (2006) gives an overview of this research and its historical progression.

3.3.2 Other reducibilitiesMain article: Reduction (recursion theory)

An ongoing area of research in recursion theory studies reducibility relations other than Turing reducibility. Post(1944) introduced several strong reducibilities, so named because they imply truth-table reducibility. A Turing ma-chine implementing a strong reducibility will compute a total function regardless of which oracle it is presented with.Weak reducibilities are those where a reduction process may not terminate for all oracles; Turing reducibility is oneexample.The strong reducibilities include:

One-one reducibility A is one-one reducible (or 1-reducible) to B if there is a total computable injective function fsuch that each n is in A if and only if f(n) is in B.

Many-one reducibility This is essentially one-one reducibility without the constraint that f be injective. A is many-one reducible (or m-reducible) to B if there is a total computable function f such that each n is in A if and onlyif f(n) is in B.

Truth-table reducibility A is truth-table reducible to B if A is Turing reducible to B via an oracle Turing machinethat computes a total function regardless of the oracle it is given. Because of compactness of Cantor space, thisis equivalent to saying that the reduction presents a single list of questions (depending only on the input) to theoracle simultaneously, and then having seen their answers is able to produce an output without asking additionalquestions regardless of the oracles answer to the initial queries. Many variants of truth-table reducibility havealso been studied.

Further reducibilities (positive, disjunctive, conjunctive, linear and their weak and bounded versions) are discussedin the article Reduction (recursion theory).The major research on strong reducibilities has been to compare their theories, both for the class of all recursivelyenumerable sets as well as for the class of all subsets of the natural numbers. Furthermore, the relations between thereducibilities has been studied. For example, it is known that every Turing degree is either a truth-table degree or isthe union of innitely many truth-table degrees.Reducibilities weaker than Turing reducibility (that is, reducibilities that are implied by Turing reducibility) havealso been studied. The most well known are arithmetical reducibility and hyperarithmetical reducibility. Thesereducibilities are closely connected to denability over the standard model of arithmetic.

3.3.3 Rices theorem and the arithmetical hierarchyRice showed that for every nontrivial class C (which contains some but not all r.e. sets) the index set E = {e: the ethr.e. set We is in C} has the property that either the halting problem or its complement is many-one reducible to E, thatis, can be mapped using a many-one reduction to E (