computability, turing machines and lambda calculus
TRANSCRIPT
Programming Languages and Paradigms
Functional Programming Background(computability, turing machines, lambda calculus)
Edward (Ned) Blurock
Functional Programming Functional Programming
COMPUTABILITY
Will an algorithm be able to compute the task?Is the task I want impossible?Is it possible to formulate an impossible task?What is the consequences for other algorithms
Functional Programming Computability
Computability
Computability vs Complexity
• Computability refers to whether of not in principle it is possible to evaluate f(n) by following a set of instructions.
• We are not for the moment worried if this computation requires 1,000,000 or more consecutive steps.
• The latter refers to complexity which we will return to later.
Functional Programming
Computability
EnsheidungsproblemDecision problem
Hilbert 1928
The Entscheidungsproblem asks for an algorithm
that takes as input a statement of a first-order logic and
answers "Yes" or "No" according to whether the statement is universally valid
(valid in every structure satisfying the axioms).
An algorithm to ask whether something is trueIs it computable
Functional Programming
Computability
EnsheidungsproblemDecision problem
Hilbert 1928
• Before the question could be answered, the notion of "algorithm" had to be formally defined. – This was done by Alonzo Church in 1936 with the
concept of "effective calculability" based on his λ calculus
– and by Alan Turing in the same year with his concept of Turing machines.
– It was recognized immediately by Turing that these are equivalent models of computation.
Functional Programming
Computability
Alan Turing (1912 – 1954)
Alonzo Church (1903-1995)
Turing Machine Lambda calculusTwo mathematical ways to ask questions about
“computability”Functional Programming
Computability
Equivalent Computers
z z zz z z z
1
Start
HALT
), X, L
2: look for (
#, 1, -
), #, R
(, #, L
(, X, R
#, 0, -
Finite State Machine
...
Turing Machine
term = variable | term term | (term) | variable . term
y. M v. (M [y v]) where v does not occur in M.
(x. M)N M [ x N ]
Lambda Calculus
Functional Programming
Computability
Alan Turing, 1936
• "On computable numbers, with an application to the Entscheidungsproblem [decision problem]”
• addressed a previously unsolved mathematical problem posed by the German mathematician David Hilbert in 1928:
Functional Programming
Computability
Turing, 1936
Is there, in principal, any definite mechanical method or process
by which all mathematical questions
could be decided?
The Decision ProblemFunctional Programming
Computability
Turing, 1936
• To answer the question (in the negative), • Turing proposed a simple abstract computing
machine, • modeled after a mathematician with a pencil,
an eraser, and a stack of pieces of paper• He asserted that any function is a computable
function if it is computable by one of these abstract machines.
Functional Programming
Computability
Turing, 1936
• He then investigated whether there were some mathematical problems which could not be solved by any such abstract machine.
• Such abstract computing machines are now called "Turing machines".
• One particular Turing machine, called a "Universal Turing Machine", served as the model for designing actual programmable computers.
Functional Programming
Computability
Why a Turing Machine
Because Turing machines are very simple compared with computers in practical use,
conceptually easier to prove impossibility results
for Turing machines. These impossibility results
apply as well to all known computersFunctional Programming
Computability
Turing, 1936
• Startling result of Turing's 1936 paper• assertion that there are well-defined
problems that cannot be solved by any computational procedure.
Functional Programming
Computability
What is Computable?Computation is usually modelled as a mapping from inputs to outputs, carried out by a formal “machine,” or program, which processes its input in a sequence of steps.
An “effectively computable” function is one that can be computed in a finite amount of time using finite resources.
………………………
input
yes
no
outputEffectively
computable function
Functional Programming
Computability
Unsolvable
• problems are formulated as functions– unsolvable functions: noncomputable;
• Problems formulated as predicates– Undecidable.
Using Turing's concept of the abstract machine, we would say that a function is noncomputable
if there exists no Turing machine that could compute it.
Functional Programming
Turing Machine
TURING MACHINEComputability
Functional Programming
............Tape
Read-Write headControl Unit
Functional Programming Turing Machine
Turing Machine
............
Read-Write head
No boundaries -- infinite length
The head moves Left or Right
The tape
OR
Functional Programming
Turing Machine
............
Read-Write head
The head at each time step:
1. Reads a symbol 2. Writes a symbol 3. Moves Left or Right
Functional Programming
Turing Machine
............
Example: Time 0
............Time 1
a a cb
a b k c
1. Reads a2. Writes k 3. Moves Left
a
Functional Programming
Turing Machine
............Time 1
a b k c
............Time 2
a k cf
1. Reads b2. Writes f3. Moves Right
b
Functional Programming
Turing Machine
Add 1 to a sequence of ones
If 1, then move right If 1, then move left
Loop
If 0, replace with 1
Start: 0 0 0 1 1 1 1 0 0 0 4 ones in a row
End: 0 0 0 1 1 1 1 1 0 0 5 ones in a row
Functional Programming
Turing Machine
So what?
You can add a 1 to a sequence of ones
From a set of a few (a manageable number) of primitive operations
If a number is represented by a sequence of onesthen you have the x+1 operator, i.e addition
This is a/(the most) basic mathematical operator
Powerful enough to represent complex mathematics
You can derive other (all) mathematic operations and objects
Functional Programming
Turing Machine
Powerful “language”From a set of a few (a manageable number) of
primitive operations
Easier enough To formulate Proofs about algorithms
Powerful enough To represent complex mathematics
All the ingredients are there Only have to deal with a few basic operations
Not worried about complexity…. Worried about, for example, “can” we compute?Functional Programming
Turing Machine (pt 2)
TURING MACHINE (PT 2)Computability
Functional Programming
Turing Machine (pt 2)
The Input String
............
Blank symbol
head
a b ca
Head starts at the leftmost positionof the input string
Input string
Functional Programming
Turing Machine (pt 2)
States & Transitions
1q 2qLba ,
Read Write Move Left
1q 2qRba ,
Move Right
Functional Programming
Turing Machine (pt 2)
Turing machine for the language }{ nnba
0q 1q 2q3q Rxa ,
Raa ,Ryy ,
Lyb ,
Laa ,Lyy ,
Rxx ,
Ryy ,
Ryy ,4q
L,
Functional Programming
Turing Machine (pt 2)
0q 1q 2q3q Rxa ,
Raa ,Ryy ,
Lyb ,
Laa ,Lyy ,
Rxx ,
Ryy ,
Ryy ,4q
L,
ba
0q
a bTime 0
Functional Programming
Turing Machine (pt 2)
0q 1q 2q3q Rxa ,
Raa ,Ryy ,
Lyb ,
Laa ,Lyy ,
Rxx ,
Ryy ,
Ryy ,4q
L,
bx
1q
a b Time 1
Functional Programming
Turing Machine (pt 2)
0q 1q 2q3q Rxa ,
Raa ,Ryy ,
Lyb ,
Laa ,Lyy ,
Rxx ,
Ryy ,
Ryy ,4q
L,
x
2q
a b Time 2 b
Functional Programming
Turing Machine (pt 2)
0q 1q 2q3q Rxa ,
Raa ,Ryy ,
Lyb ,
Laa ,Lyy ,
Rxx ,
Ryy ,
Ryy ,4q
L,
yx
2q
a b Time 3
Functional Programming
Turing Machine (pt 2)
0q 1q 2q3q Rxa ,
Raa ,Ryy ,
Lyb ,
Laa ,Lyy ,
Rxx ,
Ryy ,
Ryy ,4q
L,
yx
2q
a b Time 4
Functional Programming
Turing Machine (pt 2)
0q 1q 2q3q Rxa ,
Raa ,Ryy ,
Lyb ,
Laa ,Lyy ,
Rxx ,
Ryy ,
Ryy ,4q
L,
yx
0q
a b Time 5
Functional Programming
Turing Machine (pt 2)
0q 1q 2q3q Rxa ,
Raa ,Ryy ,
Lyb ,
Laa ,Lyy ,
Rxx ,
Ryy ,
Ryy ,4q
L,
yx
1q
x b Time 6
Functional Programming
Turing Machine (pt 2)
0q 1q 2q3q Rxa ,
Raa ,Ryy ,
Lyb ,
Laa ,Lyy ,
Rxx ,
Ryy ,
Ryy ,4q
L,
yx
1q
x b Time 7
Functional Programming
Turing Machine (pt 2)
0q 1q 2q3q Rxa ,
Raa ,Ryy ,
Lyb ,
Laa ,Lyy ,
Rxx ,
Ryy ,
Ryy ,4q
L,
yx x y
2q
Time 8
Functional Programming
Turing Machine (pt 2)
0q 1q 2q3q Rxa ,
Raa ,Ryy ,
Lyb ,
Laa ,Lyy ,
Rxx ,
Ryy ,
Ryy ,4q
L,
yx x y
2q
Time 9
Functional Programming
Turing Machine (pt 2)
0q 1q 2q3q Rxa ,
Raa ,Ryy ,
Lyb ,
Laa ,Lyy ,
Rxx ,
Ryy ,
Ryy ,4q
L,
yx
0q
x y Time 10
Functional Programming
Turing Machine (pt 2)
0q 1q 2q3q Rxa ,
Raa ,Ryy ,
Lyb ,
Laa ,Lyy ,
Rxx ,
Ryy ,
Ryy ,4q
L,
yx
3q
x y Time 11
Functional Programming
Turing Machine (pt 2)
0q 1q 2q3q Rxa ,
Raa ,Ryy ,
Lyb ,
Laa ,Lyy ,
Rxx ,
Ryy ,
Ryy ,4q
L,
yx
3q
x y Time 12
Functional Programming
Turing Machine (pt 2)
0q 1q 2q3q Rxa ,
Raa ,Ryy ,
Lyb ,
Laa ,Lyy ,
Rxx ,
Ryy ,
Ryy ,4q
L,
yx
4q
x y
Halt & Accept
Time 13
Functional Programming
LAMBDA CALCULUSComputability
Functional Programming Lambda Calculus
Lambda Calculus
Lambda Calculus
Alonzo Church (1903-1995)
Another formulationofcomputability
Functional Programming
Lambda Calculus
Equivalent Computers
z z zz z z z ...
Turing Machine
term = variable | term term | (term) | variable . term
y. M v. (M [y v]) where v does not occur in M.
(x. M)N M [ x N ]
Lambda Calculus
Functional Programming
Lambda Calculus
What is Calculus?• In High School:
d/dx xn = nxn-1 [Power Rule]d/dx (f + g) = d/dx f + d/dx g [Sum Rule]
Calculus is a branch of mathematics that deals with limits and the differentiation and integration of functions of one or more variables...
Functional Programming
Lambda Calculus
Real Definition• A calculus is just a bunch of rules for
manipulating symbols.• People can give meaning to those symbols,
but that’s not part of the calculus.• Differential calculus is a bunch of rules for
manipulating symbols. There is an interpretation of those symbols corresponds with physics, slopes, etc.
Functional Programming
Lambda Calculus
-calculusAlonzo Church, 1940
term = variable | term term | (term) | variable . term
Functional Programming
Lambda Calculus
-calculus
variable . term
F(variable) = term
Functional Programming
Lambda Calculus
Why?
• Once we have precise and formal rules for manipulating symbols, we can use it to reason with.
• Since we can interpret the symbols as representing computations, we can use it to reason about programs.
Functional Programming
Lambda Calculus
Universal Computer• Lambda Calculus can simulate a Turing Machine
– Everything a Turing Machine can compute, Lambda Calculus can compute also
• Turing Machine can simulate Lambda Calculus– Everything Lambda Calculus can compute, a Turing
Machine can compute also• Church-Turing Thesis: this is true for any other
mechanical computer also
Functional Programming
Lambda Calculus
Normal Steps• Turing machine:
– Read one square on tape, follow one FSM transition rule, write one square on tape, move tape head one square
• Lambda calculus:– One beta reduction
• Your PC:– Execute one instruction (?)
• What one instruction does varies
Functional Programming
Lambda Calculus
-Reduction (the source of all computation)
(x. M)N
This is the function definitionM is an expression with the variable x
N is used at the value of x
N substitutes x in the expression M
Functional Programming
Lambda Calculus
(x. x)a = a
-Reduction (the source of all computation)
Identity operationFunction gives the argument as output
(x. xx)a = aa(y.(x. xy)a)b =(y.ay)b=ab (y.(x. xy) z.zy)b =(y.(z.zy)y)b = (y. yy)b =bb
Functional Programming
Lambda Calculus
-calculusUseful and expression enough that a programming language was
modeled after it
LISP was developed from -calculus
not the other way round.)
This was the “first” general language ofArtificial intelligence
AlsoThe “first” functional programming language
Functional Programming
Lambda Calculus
Expression Reductions
Functional Programming
Reductions are used in the lambda calculus to reduce a statement to its simplest possible form.
This is, essentially, what a computation in lambda calculus does.
If you were to give a lambda calculus expression to "lambda calculus" calculator, it would use a combination of alpha, beta, and eta reductions to find the simplest reduction of your expression.
https://classes.soe.ucsc.edu/cmps112/Spring03/readings/lambdacalculus/reductions.html
Lambda Calculus
Expression Reduction
Functional Programming
( x. x x) ( y. y)𝛌 𝛌--> x x [ y. 𝛌
y / x]== ( y. y) ( y. y)𝛌 𝛌
--> y [ y. y / 𝛌y]
== y. y𝛌
How the original expression isevaluated
Is determined the
operational semanticsof lambda calculus (there are many ways to evaluate expressions)
Operational Semantics: how the substitutions are made:Key concepts:
specification of free variablesspecification of bound variables
Β-reduction a formalization of simple substitution
Name Capture
Functional Programming Lambda Calculus
Care must be taken to avoid “name capture”
of bound variables.
For example, this reduction would be “wrong”:
(λx. (λy. x)) y λy. y⇒because the outer y is presumably different
from the inner y bound by the lambda. In cases like these, the bound variable is renamed to obtain the “correct” behavior:
(λx. (λy. x)) y λ⇒ z. y
http://www.cs.yale.edu/homes/hudak/CS201S08/lambda.pdf
Α-reduction is a formalization of renaming of variables (without this, β-reduction can give the wrong answer)
Examples
EXAMPLES OF LAMBDA CALCULUSComputability
Functional Programming
Examples
syntax
• the identity function:– 𝛌x.x
• 2 notational conventions:• applications associate to the left (like in ML): • “y z x” is “(y z) x”• the body of a lambda extends as far as possible to the
right:• “𝛌x.x z.x z x𝛌 ” is “ x.(x z.(x z x))𝛌 𝛌 ”
Functional Programming
Examples
terminology
• 𝛌x.t
• 𝛌x.x y
the scope of x is the term t(this is the part of the expression
where x can be substituted)
x is bound in the term x.x y𝛌(x is one of the variables that can be substituted)
y is free in the term x.x y𝛌(y is free to be substituted
into the expression .. In this case x)
Functional Programming
Examples
Example
(λx. x x) ( y. y)𝛌--> x x [ y. y 𝛌 /
x]== ( y. 𝛌 y) ( y. y𝛌 )--> y [ y. y 𝛌 / y]== y. y𝛌
Substitute ( y. y) into x𝛌Substituted
Substitute ( y. y𝛌 ) into y
Substituted
Functional Programming
Examples
Another example
( x. 𝛌 x x) ( x. x x𝛌 )--> x x [ x. x x𝛌 /x]== ( x. x x) ( x. x x)𝛌 𝛌
In other words, it is simple to write non terminating computations
in the lambda calculuswhat else can we do?
Substitute x. x x 𝛌 into every x
The result is the same as the beginning
Functional Programming