gentle introduction to computational...

Complexity classes defined with deterministic algorithmsAbstracting the combinatorics

Proving hardnessPspace

Big theorems in computational complexity

Gentle Introduction to Computational Complexity

Francois Schwarzentruber

ENS Rennes, France

April 4, 2019

1 / 87




Motivation: naive methodology

Define a problem

e.g. coverage by a connected multi-drone system

Directly design an algo

Issues

you may design an overly complicated algorithm......whereas the problem is intrinsically simpler

you may try to design a simple algorithm......whereas the problem is intrinsically harder;

2 / 87




Motivation: computational complexity comes in

Define a problem Study its intrinsic complexity

Theorem (Charrier et al., 2017)

The coverage by a connected multi-dronesystem is Pspace-complete.

Design an algoWe learned:

No polynomial-time algorithm exists,unless P = Pspace;

Algorithmic techniques correspondingto Pspace are suitable.

3 / 87




Motivation: algorithmic techniques

Complexityclass

Algorithmic techniques

P Greedy algorithm, dynamic programming, algo-rithms on graphs

NP Backtracking, Backjumping, Branch-and-bound,SAT, SAT modulo theories, Linear programming

Pspace Model checkers, planners, Monte-Carlo treesearch, QBF solvers

4 / 87




ProblemsDefinition of an algorithmTime and space (memory) as resourcesDefinition of complexity classes

Outline

1 Complexity classes defined with deterministic algorithmsProblemsDefinition of an algorithmTime and space (memory) as resourcesDefinition of complexity classes

2 Abstracting the combinatorics

3 Proving hardness

4 Pspace

5 Big theorems in computational complexity5 / 87





Outline



3 Proving hardness

4 Pspace






Example: traveler salesman problem (TSP)

Definition (Problem)

Defined in terms of input/output.

Example

TSP

input: a weighted graphG = (V ,E ,w);G described by an adjacency list and weights are written in binary

output: a minimal tour;

7 / 87





Outline



3 Proving hardness

4 Pspace






Algorithm

Definition (Deterministic algorithm)

Standard algorithm, but no random choices.

Example

function tsp(G )bestWeight := +∞bestTour = −for all tours t do

if weight(t) < bestWeight thenbestWeight := weight(t)bestTour := t

return bestTour

9 / 87





Outline



3 Proving hardness

4 Pspace






Cobham’s thesis

Cobham’s thesis

Polynomial in the size n of the input = efficient.

Reasons:

Many real algorithms are O(n3) at most;

algo efficient ⇒ for i := 1..n do algo efficient;

Do not depend so much on a computation model.

Very liberal concerning the complexity analysis

O(log n) logarithmicO(poly(n)) polynomial

2O(poly(n)) exponential

22O(poly(n))double-exponential

11 / 87





Membership in Exptime

Proposition

TSP is in Exptime.

Proof.

tsp solves TSP.

tsp runs in 2poly(|G |).

function tsp(G )bestWeight := +∞bestTour = −for all tours t do

if weight(t) < bestWeight thenbestWeight := weight(t)bestTour := t

return bestTour

12 / 87





Outline



3 Proving hardness

4 Pspace






Definition of complexity classes

Definition

Exptime is the class of problems for which there is an algorithmthat solves it in exponential-time.

LogspaceP PspaceExptime Expspace2Exptime 2Expspace

14 / 87




Decision problemsOne-player algoDefinition of NP

Outline

1 Complexity classes defined with deterministic algorithms

2 Abstracting the combinatoricsDecision problemsOne-player algoDefinition of NP

3 Proving hardness

4 Pspace

5 Big theorems in computational complexity

15 / 87





New fine-grained complexity classes

TSP is in Exptime...

Methodology

Define new fine-grained complexity classes by means of games.

16 / 87





Outline



3 Proving hardness

4 Pspace


17 / 87





Decision problems

Definition (decision problem)

Problem whose output is yes/no.

Example ( )

TSP

input: a weighted graphG = (V ,E ,w);G described by an adjacency list and weights are written in binary

output: a minimal tour.

18 / 87





TSP reformulated as a decision problem

Example

TSP

input:- a weighted graph G = (V ,E ,w);- a threshold c ∈ N;

output:- yes, if there is a tour in G of weight ≤ c ;- no otherwise.

19 / 87





Examples

Example

GRAPH COLORING

Input: an undirected graph G = (V ,E );

Output: yes if there is a coloring of vertices using , thanassigns different colors to adjacent vertices; no otherwise.

6 1

7

2

83

94 10

5

6 1

7

2

83

94 10

5

20 / 87





Examples

Example

GRAPH COLORING

Input: an undirected graph G = (V ,E );

Output: yes if there is a coloring of vertices using , thanassigns different colors to adjacent vertices; no otherwise.

6 1

7

2

83

94 10

5

6 1

7

2

83

94 10

5

21 / 87





Examples

Example

SAT

Input: a Boolean formula ϕ;

Output: yes if there are values for Boolean variables thatmake ϕ true; no otherwise.

(p ∨ q) ∧ (r → ¬p) ∧ r ∧ (r → ¬s) ∧ (s → ¬q)

22 / 87





Examples

Example

HALT

input: a program π;

output:- yes, if the execution of π halts;- no otherwise.

23 / 87





Outline



3 Proving hardness

4 Pspace


24 / 87





One-player algo

Definition (One-player algo)

A one-player algo is an algorithm that may use special instructions

choose b ∈ {0, 1}

and that ends with instruction win or instruction loose.

one-player algox

win/loose

25 / 87





One-player algo for graph coloringone-player-algo graphColoring (G )

for all vertices v of G dochoose a color among for v

if no adjacent vertices have the same color then

winelse

loose

one-player algoGwin/loose

26 / 87





One-player game for graph coloring

one-player-algo graphColoring (G )for all vertices v of G do

choose a color among for vif no adjacent vertices have the same color then

winelse

loose

Proposition

G is a positive instance of GRAPH COLORINGiff

the player has a winning strategy in graphColoring (G )

→ we say that graphColoring decides GRAPH COLORING.

27 / 87





One-player game

one-player-algo tsp(G , c)t := empty tourfor i = 1..nb vertices in G do

choose a non-already chosen successor for textend t with that successor

if t is a tour and weight(t) ≤ c then win else loose

Proposition

(G , c) is a positive instance of TSPiff

the player has a winning strategy at the game tsp(G , c)

→ we say that tsp decides TSP.

28 / 87





Outline



3 Proving hardness

4 Pspace


29 / 87





Execution-time of a one-player algo

Definition (poly-time one-player algo)

A one-player algo algo(x) is in polynomial-time in |x | if the lengthof all runs is poly(|x |).

one-player-algo graphColoring (G )for all vertices v of G do

choose a color among for vif no adjacent vertices have the same color then

winelse

loose

30 / 87





Definition of NP

Definition

NP = class of decision problems such that there is a one-playergame that decides it in polynomial-time.

game for a NP-problemx

win/loose

Example (of decision problems that are in NP)

TSP Graph coloring SAT Shortest path (is in P)Real linear programming (is in P) Clique in graphsInteger linear programming

31 / 87





Terminology

one player game = non-deterministic algorithm

N = non-deterministic;

choice of a move = a non-deterministic choice/guess

the list of moves = a certificate

LogspaceP PspaceExptime Expspace2Exptime 2Expspace

NLogspaceNP NPspaceNExptime NExpspaceNExptime NExpspace

32 / 87




Easier thanIntrinsic hardnessNP-complete problems

Outline



3 Proving hardnessEasier thanIntrinsic hardnessNP-complete problems

4 Pspace


33 / 87





Outline




4 Pspace


34 / 87





Easier than

Definition

Problem Pb1 is easier than problem Pb2 if there is a poly-timedeterministic algorithm tr such that:

x is positive instance of Pb1 iff tr(x) is a positive instance of Pb2

reduction tr

Pb1

tr(x) Pb2x

Terminology

Pb1 reduces to Pb2 in poly-time

tr is called a poly-time reduction from Pb1 to Pb235 / 87





Example: Graph coloring is easier than SAT

reduction tr

GRAPH COLORING

tr(G ) SATG

tr(G ) := a Boolean formula expressing that G is colorable.

http://people.irisa.fr/Francois.Schwarzentruber/

reductioncatalog/

36 / 87

http://people.irisa.fr/Francois.Schwarzentruber/reductioncatalog/

http://people.irisa.fr/Francois.Schwarzentruber/reductioncatalog/





Outline




4 Pspace


37 / 87





SAT is NP-hard

Terminology

A problem is NP-hard if any NP-problem is easier than it.

Theorem

Cook’s theorem SAT is NP-hard.

reduction tr

Any NP-problem

tr(x) SATx

38 / 87





Any NP-problem is easier than SAT

one-player poly-timealgo for that NP-problem

x

win/loose

tr(x) := Boolean formula saying ‘the game run on x is winning’

39 / 87





Any NP-problem is easier than SAT

one-player poly-timealgo for that NP-problem

x

win/loose

tr(x) := Boolean formula saying ‘the game run on x is winning’

40 / 87





NP-hard problems

Theorem

Pb1 is NP-hard

Pb1 easier than Pb2

}implies Pb2 is NP-hard.

any NP-problem SAT 3SAT

COLORING

HAMILTONIAN CYCLE TSP

easier than easier than

easier than

easier than

easier than

41 / 87





Outline




4 Pspace


42 / 87





NP-complete problems

Definition

A problem is NP-complete if:

it is in NP;

it is NP-hard.

SAT 3SAT

COLORING

HAMILTONIAN CYCLE TSP

easier than

easier than

easier than

easier than

43 / 87





Understand where the ‘complexity’ is

Complexity of TSP restrictions

NP-complete even for 2D grid graphs

[Itai, Papadimitriou, Szwarcfiter, 1982]

in P for solid grid graphs

[Arkin, Bender, Demaine, Fekete, Mitchell, Sethia, 2001]

Parameterized complexity

identify a parameter (e.g. diameter of graphs, etc.) on inputs thatsums up the complexity.

44 / 87




Two-player poly-time gamesOne-player poly-space gamesPSPACE-complete problems

Outline



3 Proving hardness

4 PspaceTwo-player poly-time gamesOne-player poly-space gamesPSPACE-complete problems


45 / 87





Outline



3 Proving hardness



46 / 87





Motivation

Bounded planning versus the environment

querying (∃∀)∗-properties (e.g. SQL, etc.)

∃x , ∀y , (R(x , y)→ ∃z , p(f (x , y , z))

→ winning strategy for player one in a two-player poly-time game

[Arora, Barak, chap. 4.2.2 (� The essence of Pspace: optimum strategies for

game-playing �)]

47 / 87





Generalized Hex

Example (Even, Tarjan, 1976)

HEX

Input: A graph G , a source s, a target t;

Output: yes if player 1 has a winning strategy to the game:

by turn, player i select a non-selected vertex in G \ {s, t};player 1 wins if there is a s − t-path made up of 1-vertices.

48 / 87





Two-player algo

Definition (Two-player algo)

A two-player algo is an algorithm that may use special instructions

player one chooses b ∈ {0, 1}player two chooses b ∈ {0, 1}

and that ends with player one wins or player one looses.

Two-player algox

1 wins/1 looses

49 / 87





Two-player poly-time algos

two-player-algo hex(G , s, t)while some non-selected vertex in G do

player one chooses a non-selected vertex in G \ {s, t}player two chooses a non-selected vertex in G \ {s, t}if there is a s − t-path made up of 1-vertices then

player one winsplayer one looses

Definition

A strategy for a player tells her/him which move to take ateach time.

A winning strategy for player one makes player one win,whatever the moves of the other player.

50 / 87





A two-player algo decides a decision problem

Proposition

(G , s, t) is a positive instance of HEXiff

the first player has a winning strategy in hex(G , s, t)

→ We say that hex decides HEX!

51 / 87





Alternating poly-time: AP

Definition

AP is the class of decision problems such that there is a two-playeralgo that decides it in poly-time.

‘

Theorem

HEX in AP.

Proof.

hex decides HEX in poly-time.

52 / 87





Quantified binary formulas

Example

QBF-SAT

Input: a closed quantified binary formula ϕ;

Output: yes if ϕ is true; no otherwise.

∃p,∀q, ∀r ,∃s(p → (q ∧ r → s))

Theorem

QBF-SAT in AP.

53 / 87





In Pspace!

Theorem

AP ⊆ Pspace.

Deterministic backtracking algorithm (min-max style)!

?

54 / 87





In Pspace!

Theorem

AP ⊆ Pspace.


?

?

55 / 87





In Pspace!

Theorem

AP ⊆ Pspace.


?

?

?

56 / 87





In Pspace!

Theorem

AP ⊆ Pspace.


?

?

?

x

57 / 87





In Pspace!

Theorem

AP ⊆ Pspace.


?

?

?

X

58 / 87





In Pspace!

Theorem

AP ⊆ Pspace.


?

?

X

59 / 87





In Pspace!

Theorem

AP ⊆ Pspace.


?

?

?

60 / 87





In Pspace!

Theorem

AP ⊆ Pspace.


?

?

?

x

61 / 87





In Pspace!

Theorem

AP ⊆ Pspace.


?

?

?

x

62 / 87





In Pspace!

Theorem

AP ⊆ Pspace.


?

?

x

63 / 87





In Pspace!

Theorem

AP ⊆ Pspace.


?

x

64 / 87





Outline



3 Proving hardness



65 / 87





Definition

Definition

NPspace is the class of decision problems such that there is aone-player algo that decides it in polynomial-space.

Example

SOKOBAN:

Input: a Sokoban position;

Output: yes if the player can win from thatposition; no otherwise.

66 / 87





One-player algo deciding SOKOBAN in poly-space

one-player-algo sokoban(position)while position is not winning do

choose position := one of the next possible positionsfrom position

win

67 / 87





Savitch’s theorem

Theorem

AP = Pspace = NPspace.

68 / 87





Savitch’s theorem

Theorem

AP = Pspace = NPspace.⊆ ⊆

Proof of NPspace ⊆ AP

One playerpoly-space algo

Two playerpoly-time algo

reformulation

69 / 87





Two-player poly-time algo equivalent to Sokoban

Player one chooses a mid-position of Sokoban

Player two chooses which part to check

≤ 2poly(n) steps?

n = size of the Sokoban position given in input

70 / 87








≤ 2poly(n)−1? ≤ 2poly(n)−1?


71 / 87








≤ 2poly(n)−1?≤ 2poly(n)−1?


72 / 87








≤ 2poly(n)−1?

≤ 2poly(n)−2? ≤ 2poly(n)−2?


73 / 87








≤ 2poly(n)−1?

≤ 2poly(n)−2? ≤ 2poly(n)−2?


74 / 87





Outline



3 Proving hardness



75 / 87





PSPACE-complete problems: some two-player poly-time games

QBF-SAT

Input: a closed quantified binary formula ϕ;

Output: yes if ϕ is true; no otherwise.

∃p,∀q, ∀r ,∃s(p → (q ∧ r → s))

First-order query on a finite model

input: a finite model M, a first-order formula ϕ;

output: yes if M satisfies ϕ, no otherwise.

∃x , ∀y , (R(x , y)→ ∃z , p(f (x , y , z))

also HEX!76 / 87





PSPACE-complete problems: some one-player poly-space games

Universality of a regular expression

input: a regular expression e;

output: yes if the language denoted by e is Σ∗, no otherwise.

Classical planning

input: an initial state ι, a final state γ, description of actions;

output: yes if γ is reachable from ι by executing some actions,no otherwise.

Also Sokoban!

77 / 87




About gamesAbout Logspace and NLogspaceAbout non-uniform poly-time algorithm

Outline



3 Proving hardness

4 Pspace

5 Big theorems in computational complexityAbout gamesAbout Logspace and NLogspaceAbout non-uniform poly-time algorithm

78 / 87





Outline



3 Proving hardness

4 Pspace


79 / 87





Correspondence between alternating and usual classes

[Chandra, Stockmeyer, 1980, Alternations]

Theorem

AP = PspaceAexptime = Expspace

......

ALogspace = PAPspace = Exptime

......

80 / 87





Outline



3 Proving hardness

4 Pspace


81 / 87





Definitions

Definition

Logspace is the class of decision problems decided by analgorithm in logarithmic space.

The input is read-only

∼ a constant number of pointers

Definition

NLogspace is the class of decision problems decided by aone-player algo in logarithmic space.

82 / 87





Important results

Theorem

Reachability in a directed graph is NLogspace-complete.

s

t

Theorem (Immerman-Szelepcsenyi, 1988)

NLogspace = coNLogspace

Theorem (Reingold, 2005)

Reachability in an undirected graph is in Logspace.

s

t

83 / 87





Outline



3 Proving hardness

4 Pspace


84 / 87





We believe P 6= NP, even we believe P/poly 6= NP

Definition

P/poly = class of decision problems s.th. there are deterministicalgorithms A1,A2, . . . such that

An decides inputs of size n in poly(n).

Theorem (Karp and Lipton, 1980)

If NP ⊆ P/poly , the polynomial hierarchy collapses at level 2:

Two-playerpoly-time algo

with a fixed number of alternations

Two-playerpoly-time algo

with 2 alternations

transformation

85 / 87





This tutorial did not present...

Formal definitions of complexity classes with Turing machines

The obscure terminology (non-determinism, alternation,certificates, reduction, etc.)

Other types of reduction: log-space, FO, etc.

Probabilistic complexity classes

Quantum complexity classes

Counting and Toda’s theorem

Descriptive complexity

Circuit complexity

Other computation models: RAM, etc.

Interactive proofs

Parametrized complexity

...86 / 87





Thank you for your attention!

87 / 87

gentle introduction to computational...

Documents