warm-up problemcbruni/cs245resources/lectures/...the halting problem is undecidable theorem: the...
TRANSCRIPT
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Warm-Up Problem
Please fill out your Teaching Evaluation Survey! Please comment on thewarm-up problems if you havenβt filled in your survey yet.
Warm up: Given a program π that accepts input, is there an input wherethe program runs forever?
Theorem: The Exists-Input-Runs-Forever Problem is undecidable.
1/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
DecidabilityIntroduction and Reductions to the Halting
Problem
Carmen BruniLecture 23
Based on slides by Jonathan Buss, Lila Kari, Anna Lubiw and SteveWolfman with thanks to B. Bonakdarpour, A. Gao, D. Maftuleac, C.
Roberts, R. Trefler, and P. Van Beek
2/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Last Time
β’ Define a decidable problem.β’ Describe the Halting Problem.β’ Show that problems are decidable.β’ Give reductions to prove undecidability
3/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Learning Goals
β’ Prove that the Halting Problem is Undecidableβ’ Prove other problems are undecidable based on reductions to the
Halting Problem.
4/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
The Halting problem
The decision problem: Given a program π and an input πΌ , will π haltwhen run with input πΌ?
β’ βHaltsβ means βterminatesβ or β does not get stuck.ββ’ One of the first known undecidable problems
The Halting problem is undecidable.
There does not exist an algorithm π», which gives the correct answer forthe Halting problem for every program π and every input πΌ .
Exercise: Translate the above statement into a Predicate formula.
5/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
The Halting Problem is Undecidable
A proof by video
https://www.youtube.com/watch?v=92WHN-pAFCs
6/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Common questions about the video
β’ Why can we feed a program as an input to itself? That is, why doesπ»(π , π) make sense?We can convert any program to a string, then we can feed the stringof the program to itself as input.
β’ What does the negator do?It negates the behaviour of the machine. If H predicts that theprogram halts, then the negator goes into an infinite loop and doesnot halt. If H predicts that the program does not halt, then thenegator halts.The negator is designed to make H fail at its prediction task.
β’ Why do we need the photocopier?In the video, H takes two inputs. We need to make two copies of theinput. In code, we do not need the photocopier. We simply need tocall π»(π , π).
7/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
The Halting Problem is Undecidable
Theorem: The Halting problem is undecidable.
Proof by contradiction.
Assume that there exists an algorithm π», which decides the Haltingproblem for every program and every input.
We will construct an algorithm π, which takes program π as input.
We will show that π» gives the wrong answer when predicting whether theprogram π halts when run with input π. This contradicts the fact that π»decides the Halting problem for every program and every input. Therefore,π» does not exist.
8/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
The Halting Problem is Undecidable.
The algorithm π:
β’ Takes a program π as inputβ’ Makes two copies of πβ’ Runs π»(π, π):
β’ If π»(π, π) outputs βyesβ, then π goes into an infinite loop and runsforever (ie. it doesnβt halt)
β’ If π»(π, π) outputs βnoβ, then π halts and returns 0
Now, we ask βWhat happens if we try π(π)?β
9/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Proof that π(π) Does Halt Is Impossible
Suppose that π(π) halted. Then π»(π, π) by construction must haveoutputted βnoβ, that is, π does not halt when given itself as input. Thiscontradicts our assumption that π did halt when given itself as input.
10/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Proof that π(π) Does Not Halt Is Impossible
Suppose that π(π) did not halt. Then π»(π, π) by construction musthave outputted βyesβ, that is, π does halt when given itself as input. Thiscontradicts our assumption that π did not halt when given itself as input.
The combination of this slide and the next shows that this machine πcannot exist. However, every part of the machine π does exist except forpossibly the machine π». Hence π» cannot exist and we have acontradiction.
11/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Diagonalization
The above method is related to Cantorβs diagonalization method which willdiscuss at the end of todayβs lecture and the start of tomorrowβs lecture.
12/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Specific Input Run Forever Problem
Problem: Given a program π , does it run forever on input π = 0?
Theorem: The Specific Input Run Forever Problem is undecidable.
Proof: Assume towards a contradiction that the specific input problem isdecidable. That is, there is an algorithm π΅ such that when I give it aprogram π β², it returns βyesβ if and only if it runs forever on the inputπ = 0.Given a program π and an input πΌ , construct a program π that:
β’ Ignores the input and runs π on πΌ .β’ If π when run on πΌ halts, then halt and return 0.
Note that this π halts at 0 [in fact it halts on all inputs!] if and only if πhalts on πΌ . We now solve the Halting Problem. (Continued on next slide)
13/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Specific Input Run Forever Problem
Problem: Given a program π , does it run forever on input π = 0?
Theorem: The Specific Input Run Forever Problem is undecidable.
Proof: Assume towards a contradiction that the specific input problem isdecidable. That is, there is an algorithm π΅ such that when I give it aprogram π β², it returns βyesβ if and only if it runs forever on the inputπ = 0.Given a program π and an input πΌ , construct a program π that:
β’ Ignores the input and runs π on πΌ .β’ If π when run on πΌ halts, then halt and return 0.
Note that this π halts at 0 [in fact it halts on all inputs!] if and only if πhalts on πΌ . We now solve the Halting Problem. (Continued on next slide)
13/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Specific Input ProblemProblem: Given a program π , does it run forever on input π = 0?
Theorem: The Specific Input Problem is undecidable.
Proof: (Continued...) We now solve the Halting Problem. Consider analgorithm π΄:
β’ It consumes π and πΌβ’ It creates πβ’ It runs algorithm π΅ on π and then negates the output.
Now, algorithm π΅ on π returns βyesβ if and only if π runs forever oninput π = 0. But, this can happen by construction if and only if π doesnot halt on πΌ . So negating the final answer means that if we get βnoβfrom the above algorithm, then we have that π does not halt on input πΌ ,a contradiction. Hence algorithm π΅ cannot exist, that is, the SpecificInput Run Forever Problem is undecidable.
14/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Partial Correctness Problem
Problem: Given a Hoare triple β¦ π1 β¦ πΆ β¦ π1 β¦, does πΆ satisfy the tripleunder partial correctness?
Theorem: The Partial Correctness Problem is undecidable.
Proof: Assume towards a contradiction that there is an algorithm π΅ suchthat when β¦ π1 β¦ πΆ β¦ π1 β¦, is satisfied under partial correctness it returnsβyesβ. We claim that we can construct an algorithm π΄ to decide theHalting-No-Input Problem. Let π be a program. Algorithm π΄ is given by
β’ Run algorithm π΅ on program β¦ true β¦ π β¦ false β¦ and return thenegated answer.
(continued on next slide)
15/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Partial Correctness Problem
Problem: Given a Hoare triple β¦ π1 β¦ πΆ β¦ π1 β¦, does πΆ satisfy the tripleunder partial correctness?
Theorem: The Partial Correctness Problem is undecidable.
Proof: Assume towards a contradiction that there is an algorithm π΅ suchthat when β¦ π1 β¦ πΆ β¦ π1 β¦, is satisfied under partial correctness it returnsβyesβ. We claim that we can construct an algorithm π΄ to decide theHalting-No-Input Problem. Let π be a program. Algorithm π΄ is given by
β’ Run algorithm π΅ on program β¦ true β¦ π β¦ false β¦ and return thenegated answer.
(continued on next slide)
15/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Partial Correctness Problem
Problem: Given a Hoare triple β¦ π1 β¦ πΆ β¦ π1 β¦, does πΆ satisfy the tripleunder partial correctness?Theorem: The Partial Correctness Problem is undecidable.Proof: (Continued...) Now, by construction, π΄ halts and returns βyesβ ifand only if program π halts. This is true since the Hoare triple β¦ true β¦ πβ¦ false β¦ is never satisfied and so by the definition of partial correctness,this triple satisfies partial correctness if and only if π does not halt. Thusalgorithm π΅ will return βnoβ if and only if program π halts. This is thenegation of what we want our program do so flip the answer. Thus ifalgorithm π΅ exists, then the Halting-No-Input Problem is decidable. Thisis a contradiction. Thus the Partial Correctness problem is undecidable.
16/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Another Undecidable Problem
Provability of a formula in Predicate Logic:
Given a formula π, does π have a proof?(Or, equivalently, is π valid?)
No algorithm exists to solve provability:
Theorem Provability is undecidable.
17/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Provability In Predicate Logic Is Undecidable
Outline of proof: Suppose that some algorithm π decides provability.
1. Devise an algorithm to solve the following problem:Given a program π and input πΌ , produce a formula ππ,πΌsuch that ππ,πΌ has a proof iff π halts on input πΌ .
2. Combine algorithm π with the above algorithm to get an algorithmthat decides the Halting Problem: On input (π , πΌ), give the formulaππ,πΌ as input to π.
But no algorithm decides the Halting problem.
Thus no algorithm decides provability.
18/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
The Technique: βDiagonalizationβ
The technique used in the proof of the undecidability of the haltingproblem is called diagonalization.
It was originally devised by Georg Cantor (in 1873) for a different purpose.
Cantor was concerned with the problem of measuring the sizes of infinitesets. Are some infinite sets larger than others?
Example. The set of even natural numbers is the same size (!!) as the setof all natural numbers.
Example. The set of all infinite sequences over {0, 1} is larger than theset of natural numbers.
(Idea of proof: (A) Assume that each sequence has a number. (B) find asequence that is different from every numbered sequence.)
βDiagonalizationβ Arguments 19/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Countable sets
A set π is countable if there is a one-to-one correspondence between πand the set of natural numbers (β).
How do we prove that a set is uncountable?
Countability 20/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Cantorβs diagonal argument
Consider an infinite sequence π = (π 1, π 2, β¦), where each element π π is aninfinite sequence of 1s or 0s (π π is a binary string of infinite length).
π 1 = (0, 0, 0, 0, 0, 0, β¦)π 2 = (1, 1, 1, 1, 1, 1, β¦)π 3 = (0, 1, 0, 1, 0, 1, β¦)π 4 = (1, 0, 1, 0, 1, 0, β¦)π 5 = (1, 1, 0, 0, 1, 1, β¦)π 6 = (0, 0, 1, 1, 0, 0, β¦)β¦
Countability 21/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Cantorβs diagonal argument
There is a sequence π such that if π π,π = 1, then π π = 0, and otherwiseπ π = 1.
π 1 = (0, 0, 0, 0, 0, 0, β¦)π 2 = (1, 1, 1, 1, 1, 1, β¦)π 3 = (0, 1, 0, 1, 0, 1, β¦)π 4 = (1, 0, 1, 0, 1, 0, β¦)π 5 = (1, 1, 0, 0, 1, 1, β¦)π 6 = (0, 0, 1, 1, 0, 0, β¦)β¦
Define π = (1, 0, 1, 1, 0, 1, β¦).Then π is not any of the sequences π π on the list π.
Countability 22/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Cantorβs diagonal argument
By construction, π is not contained in the countable sequence π.
Let π be a set consisting of all infinite sequences of 0s and 1s. Bydefinition, π must contain π and π .
Since π is not in π, the set π cannot coincide with π.
Therefore, π is uncountable; it cannot be placed in one-to-onecorrespondence with β.
Countability 23/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Cantorβs diagonal argument
Therefore, the set of even integers is smaller than the set of all infinitesequences over {0, 1}.
Countability 24/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
β is Uncountable
We will show that a subset of β, per example (0, 1), is uncountable.Let π be the following countable sequence of elements from (0, 1):π1 = 0.3245890172245 β¦π2 = 0.1478562003640 β¦π3 = 0.7129601400235 β¦π4 = 0.0024533889974 β¦π5 = 0.5364578912235 β¦π6 = 0.8581488004222 β¦β¦
Countability 25/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
β is Uncountable
It is possible to build a new number π = (0.π1π2π3 β¦) in such a way thatif ππ,π = π, where π is one digit natural number, then ππ is an one-digitnatural number different than π.π1 = 0.3245890172245 β¦π2 = 0.1478562003640 β¦π3 = 0.7129601400235 β¦π4 = 0.0024533889974 β¦π5 = 0.5364578912235 β¦π6 = 0.8581488004222 β¦β¦π = (0.580135 β¦)
Countability 26/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
β is Uncountable
By the construction, π β (0, 1) is not contained in the countable set π.Therefore the countable set π cannot coincide with (0, 1).We obtain that (0, 1) is uncountable.
Countability 27/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
β is Uncountable (long proof)
We build an one-to-one correspondence between the set π and a subset ofβ.Let function π(π‘) = 0.π‘, where π‘ is a string in π .For example, π(0111 β¦) = 0.0111 β¦.Observe that π(1000 β¦) = 0.1000 β¦ = 1/2, andπ(0111 β¦) = 0.0111 β¦ = 1/4 + 1/8 + 1/16 + β¦ = ββ
π=11π = 1/2.
Hence, π is not a bijection.
Countability 28/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Cantorβs diagonal argument (long proof)
To produce a bijection from π to the interval (0, 1) β β:β’ From (0, 1) remove the numbers having two binary expansions and
form π = (1/2, 1/4, 3/4, 1/8, 3/8, 5/8, 7/8, β¦).β’ From π , remove the strings appearing after the binary point in the
binary expansions of 0, 1, and the numbers in sequence π and formπ = (000 β¦ , 111 β¦ , 1000 β¦ , 0111 β¦ , 01000 β¦ , 00111 β¦ , β¦).
π(π‘) from π to (0, 1) is defined by: If π‘ is the πth string in sequence π, letπ(π‘) be the πth number in sequence π; otherwise, let π(π‘) = 0.π‘.
Countability 29/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Cantorβs diagonal argument (long proof)
To build a bijection from π to β, we use π‘ππ(π₯), a bijection from(π/2, π/2) to β.The linear function β(π₯) = π.π₯ β π/2 provides a bijection from (0, 1) to(βπ/2, π/2).The composite function π‘ππ(β(π₯)) provides a bijection from (0, 1) to β.The function π‘ππ(β(π(π‘))) is a bijection from π to β.
Countability 30/31
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Cantorβs diagonal argument
Using diagonalization, one can show that (for example):
β’ |β| = |β| = |β€|β’ |β| < |2β|β’ The set of all functions from β to β is uncountable.
Countability 31/31