general background on computing

General Background on Computing

Mika Hirvensalo

Department of Mathematics and StatisticsUniversity of Turku

[email protected]

Thessaloniki, May 2016

Mika Hirvensalo General Background on Computing 1 of 68

About Turku

Established 13rd century, capital of Finland until 1812Population 186 000 (city area) 320 000 (subregion area)

About Turku

Established 13rd century

, capital of Finland until 1812Population 186 000 (city area) 320 000 (subregion area)

About Turku

Established 13rd century, capital of Finland until 1812

Population 186 000 (city area) 320 000 (subregion area)

About Turku

Established 13rd century, capital of Finland until 1812Population 186 000 (city area) 320 000 (subregion area)

About Turku

University of Turku

First established 1640

Current one 192020 000 students in seven faculties

University of Turku

First established 1640Current one 1920

20 000 students in seven faculties

University of Turku

First established 1640Current one 1920

20 000 students in seven faculties

Department of Mathematics and Statistics

Discrete mathematics:

Coding Theory and CryptographyComplex Systems and ComputingNumber TheoryWords and Automata

Analysis

Functional analysisGeometric complex analysis

Applied Mathematics

BiomathematicsOptimization

Statistics

Analysis methods for e.g. cancer cell data

Terminology

Counting: Recognizing, estimating or determining the number ofobjects

Computing: Rewriting data represented with finitely manysymbols, using a set of finitely specified local rules

Elementary computational step: One application of a local rule.

Terminology

Counting

Computing

Multiplication7 4 3 2 1 1

× 4 5 7 3 2 1

5 2 0 13 7 1 5

2 9 7 23 3 9 5 5 1

Addition1 1

2 9 9+ 1

Elementary steps: Eg. Single-digit multiplication and addition

Computing

× 4 5 7 3 2 1

5 2 0 13 7 1 5

2 9 7 23 3 9 5 5 1

Addition1 1

2 9 9+ 1

Computing

× 4 5 7 3 2 1

5 2 0 13 7 1 5

2 9 7 23 3 9 5 5 1

Addition1 1

2 9 9+ 1

Uses of counting

Abstraction

Early counting

Age c. 44 000 years, found in Lembobo, Swaziland.

Age c. 20 000 years, found in Ishango, Kongo.

Early counting

Age c. 44 000 years, found in Lembobo, Swaziland.

Age c. 20 000 years, found in Ishango, Kongo.

From counting to computing

Babylonian multiplication table, approx. 3700–3900 years ago.

Uses of computing: Trading

Beyond counting and computing

Rhind papyrus, approx. 3600 years ago.

Mechanical computing

Antikythera machine, approx. 2200 years ago.

A modern reconstruction of the Antikythera machine: Solar systemcalender

Roman abacus (replica), approx. 1800 years ago.

Mechanical computing: bibliography

Oxford University Press (2013)

Mechanical computing with electronic aid

Friden calculator (1949 –1966)

(Alan Turing, 1939–1940)

Electronic computing

Colossus Mark 2 (1944) (to analyze the Lorenz chiper)

Semiconductor technology

Commodore 64 (1982–1993) 1 MHz, 64 kB RAM

Semiconductor technology

ASUS ZenBook UX305FA (2015–) 0.8–2 GHz Dual core, 8 GBRAM + 256 GB SSD

Moore’s law

Programmable computers

Hardware + software

Hardware provides a physical platform for processing data(“body”)

Software encodes the rules of data processing (“soul”)

Hardware + software

What next?

Hardware: Molecular computing?

What next?

Software: Intelligence?

What next?

Awareness, human intelligence?

What next?

Superhuman intelligence?

Encyclopædia Britannica:

Computer, device for processing, storing, and displayinginformation.Computer once meant a person who did computations, but now theterm almost universally refers to automated electronic machinery.

Computational problems

Integer multiplication:552453422346237439542353× 932473424326247469571361?

Obvious, easy to compute: 20× 20 = 400 single-digitmultiplications +additions (computational steps)

Optimal method? Not the obvious one!

Primality test. Not so obvious but an efficient solution known

Factorization

515148134515950669863549241095573754248915352433

= 552453422346237439542353

× 932473424326247469571361

No efficient classical algorithm known.

Integer multiplication:552453422346237439542353× 932473424326247469571361?Obvious, easy to compute: 20× 20 = 400 single-digitmultiplications +additions (computational steps)

Factorization

515148134515950669863549241095573754248915352433

= 552453422346237439542353

× 932473424326247469571361

Optimal method?

Not the obvious one!

Factorization

515148134515950669863549241095573754248915352433

= 552453422346237439542353

× 932473424326247469571361

Factorization

515148134515950669863549241095573754248915352433

= 552453422346237439542353

× 932473424326247469571361

Factorization

515148134515950669863549241095573754248915352433

= 552453422346237439542353

× 932473424326247469571361

Factorization

515148134515950669863549241095573754248915352433

= 552453422346237439542353

× 932473424326247469571361

Factorization

515148134515950669863549241095573754248915352433

= 552453422346237439542353

× 932473424326247469571361

Factorization

515148134515950669863549241095573754248915352433

= 552453422346237439542353

× 932473424326247469571361

Productm, n 7→ mn

An instance of Product: Input (3, 5) (output 15)

Factorization

m 7→ p (smallest prime factor of m)

An instance of Factorization: Input 15 (output 3)

Primality

n 7→ 0/1. 1 if n prime, 0 otherwise

An instance of Primality: Input 7 (output 1)

Another instance of Primality: Input 8 (output 0)

Factorization seems harder than Product

Primality appears hard

Factorization is at least as hard as Primality (reduction)

Productm, n 7→ mn

Factorization

Primality

Productm, n 7→ mn

Factorization

Primality

Productm, n 7→ mn

Factorization

Primality

Productm, n 7→ mn

Factorization

Primality

Productm, n 7→ mn

Factorization

Primality

Productm, n 7→ mn

Factorization

Primality

Productm, n 7→ mn

Factorization

Primality

Productm, n 7→ mn

Factorization

Primality

Productm, n 7→ mn

Factorization

Primality

Productm, n 7→ mn

Factorization

Primality

Input and output are strings over some alphabetΣ = {a1, . . . , ak}

Encoding ai (and hence input and output) in binary is alwayspossible

Decision problems: Output ∈ {0, 1}

A general problem can be presented as a sequence of decisionproblems: 1:st bit of the output? 2:nd bit of the output? etc.

Computation: InputA−→ Output

What does it take to compute A? How much time? Howmuch space?

Input and output are strings over some alphabetΣ = {a1, . . . , ak}Encoding ai (and hence input and output) in binary is alwayspossible

What does it take to compute A?

How much time? Howmuch space?

What does it take to compute A? How much time?

Howmuch space?

Computational Complexity – Preliminaries

What is computation?

How to measure the complexity of computation?

Time / Space ? Physical time (in seconds) not useful

Time / Space ?

Physical time (in seconds) not useful

Gottfried Wilhelm Leibniz (1646–1716)

Scientia Universalis:

Characteristica universalis

Calculus ratiocinator

”I think that some selected men could finish the matter in fiveyears”

Kurt Godel (1906–1978)

Incompleteness theorems⇔ Algorithmic undecidability

Incompleteness theorems

⇔ Algorithmic undecidability

Incompleteness theorems⇔ Algorithmic undecidability

Recommended reading

Scott Aaronson: Quantum Computing since Democritus

(Cambride University Press 2013)

Alan Turing (1912–1954)

Theoretical model of computer,Turing Machine (1937)

Tape → I N P U T

6← Read-write head� �

p, q, r , . . .State set →(program)

Turing Machine

Tape → I N P U T

State p:

Reading a, write b (b depends on p and a)

More read-write head (direction depends on p and a)

Move to state q (q depends on p and a)

Transition function δ(p, a) = (q, b,D) (computational step)

Turing Machine

Tape → I N P U T

State p:

Reading a, write b (b depends on p and a)

More read-write head (direction depends on p and a)

Move to state q (q depends on p and a)

Transition function δ(p, a) = (q, b,D) (computational step)

Turing Machine

INPUTT−→ OUTPUT

Turing machine has a starting state q0 and final state(s) qf

In the beginning, INPUT is written on the tape, read-writehead set to read the first symbol, and the state is q0

Computation is carried on by applying the transition functionδ again and again until a final state is reached

When qf is reached, the computation stops and the tapecontent is interpreted as OUTPUT

On decision problems, it is enough to have two ending statesqy (yes) and qn (no), and tape content can be ignored

Notation: T (INPUT ) = OUTPUT

Turing Machine

INPUTT−→ OUTPUT

Turing Machine

INPUTT−→ OUTPUT

Turing Machine

INPUTT−→ OUTPUT

Turing Machine

INPUTT−→ OUTPUT

Turing Machine

INPUTT−→ OUTPUT

Turing Machine

INPUTT−→ OUTPUT

Church-Turing Thesis

Church-Turing Thesis:

Turing machine is the exact mathematical counterpart of theintuitive notion of algorithm

Not provable

Not seriously challenged

All known algorithms can be converted into Turing machineformalism

Not provable

Turing machines

Computational time exact (number of steps)

Space needed for computation exact (number of cells)

Useful constructions

Concatenation

Parallel computation

Subroutines

Encoding

TMs are too elementary for practical algorithm design

Very useful theoretically

Turing machines

Concatenation

Subroutines

Encoding

Turing machines

Concatenation

Subroutines

Encoding

Turing machines

Concatenation

Subroutines

Encoding

Turing machines

Concatenation

Subroutines

Encoding

Turing machines

Concatenation

Subroutines

Encoding

Turing machines

Concatenation

Subroutines

Encoding

Turing machines

Concatenation

Subroutines

Encoding

Turing machines

Concatenation

Subroutines

Encoding

Turing machines

Concatenation

Subroutines

Encoding

Programmability

A description of a Turing machine is a finite table oftransitions ⇒ finite string.

The state set describes the program

Part of the input can be interpreted as program

Universal Turing Machine UOn input (T ,w), U simulates the computation of T on input w .

The input w of T must be encoded into the alphabet of UQuite small universal Turing machines exists:

15 states, 2-element alphabet9 states, 3-element alphabet2 states, 18-element alphabet

Programmability

The input w of T must be encoded into the alphabet of U

Quite small universal Turing machines exists:

Programmability

Algorithmic undecidability

Program STUCK

Finds out if program P with input x gets stuck (runs forever):STUCK(P,x)= yes/no.

Program TEASE

TEASE(input) =

{Stop, if STUCK(input,input)=yes

Get stuck, if STUCK(input,input)=no

TEASE(TEASE)?

Stops if, jos STUCK(TEASE,TEASE)=yes (meaning thatTEASE(TEASE) does not stop)

Gets stuck, if STUCK(TEASE,TEASE)=no (meaning thatTEASE(TEASE) stops) Contradiction! ⇒ no program STUCK

exists

Program STUCK

Program TEASE

TEASE(input) =

TEASE(TEASE)?

exists

Program STUCK

Program TEASE

TEASE(input) =

TEASE(TEASE)?

exists

Program STUCK

Program TEASE

TEASE(input) =

TEASE(TEASE)?

exists

Program STUCK

Program TEASE

TEASE(input) =

TEASE(TEASE)?

exists

Program STUCK

Program TEASE

TEASE(input) =

TEASE(TEASE)?

Gets stuck, if STUCK(TEASE,TEASE)=no (meaning thatTEASE(TEASE) stops)

Contradiction! ⇒ no program STUCK

exists

Program STUCK

Program TEASE

TEASE(input) =

TEASE(TEASE)?

exists

Halting problem is algorithmically undecidable

There is no such program as STUCK

Strings can be encoded in numbers INPUT . . . →N = 7378808584.

Turing Machine operations are interpreted as calculations:N1 = 73 78 808584 → N2 = 72 79 808584

Implies algorithmic undecidability for many mathematicalproblems

Hilbert’s 10th problem

Given an polynomial p(x1, x2, . . . , xn) over integers, does it haveany integer zero? (Undecidable: Yury Matiyasevich 1970)

Matrix problems: Given set {M1, . . . ,Mk} of integer matrices,can we get the zero matrix multiplicatetively?

Measuring time/space

Gluing alphabet symbols together results in a bigger alphabetand more δ-rules, but smaller computations time (constantimprovement)

“Increasing hardware makes computation faster”(constant improvement)

Time / Space -resources should be measured only up tomultiplicative constant.

Ordo-notation

Measuring computational resources

Computational resources (time/space) should be measuredignoring the multiplicative constants

Example

Algorithms A1, A2, A3, and A4 consume respectively 2n6, 600n3,40000n, and 2n

1010 steps to accomplish their tasks. Ignoring themultiplicative constants, their running times are around n6, n3, n,and 2n. Hence A3 is the fastest and A4 the slowest.

Ordo-notation

Example

Ordo-notation

Example

Ordo-notation

Definition

f (x) = O(g(x)), if there are constants K > 0 and M > 0 so that

|f (x)| ≤ K |g(x)| ,

whenever x ≥ M.

Example

x5 − 20x4 − 9x3 − 6x2 − 12x − 4 = O(x5)

Example

xn = O(mx)

for each n ∈ N and m > 1.

Ordo-notation

Definition

|f (x)| ≤ K |g(x)| ,

whenever x ≥ M.

Example

x5 − 20x4 − 9x3 − 6x2 − 12x − 4 = O(x5)

Example

xn = O(mx)

Ordo-notation

Definition

|f (x)| ≤ K |g(x)| ,

whenever x ≥ M.

Example

x5 − 20x4 − 9x3 − 6x2 − 12x − 4 = O(x5)

Example

xn = O(mx)

Ordo-notation

Definition

|f (x)| ≤ K |g(x)| ,

whenever x ≥ M.

Example

x5 − 20x4 − 9x3 − 6x2 − 12x − 4 = O(x5)

Example

xn = O(mx)

Notations

Definition

A finite set Σ = {a1, . . . , an} is called an alphabet. The set of allstrings (words) over alphabet Σ is denoted by Σ∗. A formallanguage over alphabet Σ is a subset of Σ∗.

Example

Any Turing machine T defines a formal languageL(T ) = {w ∈ Σ∗ | The computation of T on input w stops}

Definition

A formal language L ⊆ Σ∗ is recursively enumerable if it can beaccepted by a Turing machine, meaning that w ∈ L ⇐⇒ T haltson input w . The set of recursively enumerable languages isdenoted by RE.

Notations

Definition

Example

Definition

Notations

Definition

Example

Definition

Notations

Definition

Example

Definition

Notations

Definition

A formal language L ⊆ Σ∗ is recursive if its membership problem issolvable by a Turing machine that halts on every input: w ∈ L⇐⇒ the computation of T on w halts on an accepting state. Theset of recursive languages is denoted by R

Example

The halting language

H = {(P, x) | Turing machine P halts on input x }

is recursively enumerable but not recursive (There is no programSTUCK).

Notations

Definition

Example

Notations

Definition

Example

Polynomial Time Computation

Straightforward to design: On input w = a1 . . . an (denoten = |w |), machine counts nk steps and then stops.

Can run in parallel with any other TM

Polynomial Time Turing Machines

On any input w , the machine stops in at most nk steps.

The decision (yes/no) depends on the halting state:

If machine stops because the “time” is up, the answer is “no”(reject)If machine finishes its computation in time bound, the answercan be yes (accept) or (no) depending on the input

k is a parameter that can be chosen freely

Example

Product is solvable in O(n2) steps (normal multiplication)

Example

Primality is solvable in O(n6+ε) steps (highly nontrivial)

Definition

P is the set of formal languages accepted by polynomial-timeTuring Machines (Practical computation with no errors)

Example

Definition

Example

Definition

Example

Definition

Turing Machine generalizations

Nondeterministic

Instead of single transition δ(p, a) = (q, b,D) the machine canchoose its action from a finite set (p, a) −→ (qi , bi ,Di ) (transitionrelation).

A nondeterministic Turing machine does not define a function

but a relation INPUTT−→ OUTPUT

Probabilistic

Same as nondeterministic, but all transitions occur with a givenprobability (p, a)

pi−→ (qi , bi ,Di )

A probabilistic Turing machine does not define a function buta probability distribution over OUTPUT s, depending on theINPUT .

Nondeterministic

Probabilistic

Nondeterministic

Probabilistic

Nondeterministic

Probabilistic

Nondeterministic

Probabilistic

Definition (Nondeterministic acceptance)

An input word w L is accepted By a nondeterministic Turingmachine N if there is at least one accepting computation.Otherwise w is rejected by N .

Tree of computations

Tree of computationsMika Hirvensalo General Background on Computing 58 of 68

Definition (Monte Carlo Model)

An input word w is accepted by a probabilistic Turing Machine P,if the acceptance probability for w is at least 2

3 A word is rejectedis its acceptance probability is at most 1

Definition (Las Vegas Model)

An input word w is accepted by a probabilistic Turing machine, ifits acceptance probability is 1. A word is rejected, if it acceptanceprobability is at most 1

Definition (Monte Carlo Model)

An input word w is accepted by a probabilistic Turing Machine P,if the acceptance probability for w is at least 2

3 A word is rejectedis its acceptance probability is at most 1

Definition (Las Vegas Model)

An input word w is accepted by a probabilistic Turing machine, ifits acceptance probability is 1. A word is rejected, if it acceptanceprobability is at most 1

Example (Nondeterministic Factorization Algorithm)

1 Input m = n1 . . .mn in binary

2 For i = 1 to n2 do: guess the i :th digit of a factor f = f1 . . . f n

3 Perform division m/f deterministically

4 If f divides m, the run is successful, otherwise not

5 In a succesful case, the algorithm can be re-run on f

For any composide number m, there exists a factor f , andhence there is at least one succesful run of the algorithm.

In general, running a nondeterministic algorithm correspondsto guessing and verification

Important complexity classes

Definition

P is the class of languages that can be accepted bypolynomial time determinstic Turing machines

NP is the class of languages that can be accepted bynondeterministic polynomial time Turing machines

BPP is the class of languages that can be accepted byprobabilistic polynomial time Turing machines with MonteCarlo acceptance model (practical computability)

Clearly P ⊆ NP and P ⊆ BPP.

Important complexity classes

Definition

P is the class of languages that can be accepted bypolynomial time determinstic Turing machines

NP is the class of languages that can be accepted bynondeterministic polynomial time Turing machines

BPP is the class of languages that can be accepted byprobabilistic polynomial time Turing machines with MonteCarlo acceptance model (practical computability)

Clearly P ⊆ NP and P ⊆ BPP.

P vs. NP

For any nondeteministic Turing machine machine, the numberof computational choices can be assumed two

Machine “tosses coin” on each step

Guiding string: s = s1s2 . . . snk ; si tells which nondeterministicoption to take (outcomes of the “coin tosses”)

Computing with a guiding string is deterministic: a guidingstring determines a path in the tree of computations

w ∈ L ⇔ if there is a guiding string s leading to an acceptingfinal state. Hence:

Nondeterministic computing= guessing the guiding string + computing deterministically

P vs. NP

Definition (3-Sat)

Given a Boolean expression φ(x1, . . . , xn) in Conjunctive NormalForm, three literals in each clause, determine if there is a satisfyingassignment.

Example

φ(x1, x2, x3, x4, x5) = (¬x1∨¬x2∨x3)∧(x2∨¬x3∨¬x4)∧(¬x3∨x4∨x5)has a satisfying assignment (x1, x2, x3, x4, x5) = (0, 1, 0, 1, 1).

Nondeterministic algorithm for 3-Sat

1 For i = 1 to n do: Guess value vi for xi2 Check whether φ(v1, . . . , vn) has truth value 1.

P vs. NP

Definition (3-Sat)

Example

P vs. NP

Definition (3-Sat)

Example

P vs. NP

Definition (3-Sat)

Example

P vs. NP

Deterministic algorithm for 3-Sat

1 Go through all values v1 . . . vn ∈ {0, 1}n;

2 For each valuation, check whether φ(v1, . . . , vn) has truthvalue 1

The deterministic version requires 2n rounds, whereas thenondeterministic version has only one round.

Can you do better (deterministically)?

Uwe Schoning:O(( 4

3 )n) algorithm

P vs. NP

Uwe Schoning:O(( 4

3 )n) algorithm

P vs. NP

Uwe Schoning:O(( 4

3 )n) algorithm

P vs. NP

Uwe Schoning:O(( 4

3 )n) algorithm

P vs. NP

Can you solve 3-Sat deterministically in nk time?

No-one knows

3-Sat is NP-complete

All other NP problems reduce to it

P algorithm to 3-Sat will give a P algorithm for all NPproblems

A polynomial algorithm for 3-Sat would imply that “guessing” isequally difficult to “verifying”; NP would be equal to P.

P vs. NP

No-one knows

P vs. NP

No-one knows

P vs. NP

No-one knows

P vs. NP

No-one knows

P vs. NP

An NP algorithm for finding the proof of Riemann Hypothesis

1 Guess a proof consisting of n symbols (There are kn such,where k is the number of alphabet symbols)

2 Verify that it is correct

If P = NP there is also a deterministic polynomial-time algorithmfor the same task

Probably P 6= NP, but how to prove it?

P vs. NP

Clay Mathematics Institute:

The solver of the P vs. NP problem will get

P vs. NP

Clay Mathematics Institute:

The solver of the P vs. NP problem will get

Further complexity classes

Complexity Zoo athttps://complexityzoo.uwaterloo.ca/Complexity Zoo

496 complexity classes by now

Further complexity classes

Complexity Zoo athttps://complexityzoo.uwaterloo.ca/Complexity Zoo

496 complexity classes by now

general background on computing

Documents