![Page 1: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/1.jpg)
Computational Complexity
Dr. Colin CampbellCourse: EMAT20531
![Page 2: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/2.jpg)
Order Notation for Functions
The order notation is used to give a bound on the limiting behaviour of a function.
We write f(x) is O(g(x)) iff
xgmxfmx that such 00
![Page 3: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/3.jpg)
Order example f=100x3+50x2+10
x+100 g=x3
f-mg
m=110
m=120
m=130
m=140
m=150
![Page 4: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/4.jpg)
The big O Notation for Algorithms
We can represent the complexity of an algorithm A by a function
fA(n)=number of steps (or time) it takes for A to complete given an input of size n.
The size of input may be measured in a number of different ways.
![Page 5: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/5.jpg)
Big O NotationA=Do i=1,..,nPrint “hello world”ClearRepeatPrint “End” f(n)=2n+1 O(f(n))=n A is linear in n
![Page 6: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/6.jpg)
Big O Notation: 2
A=Do i=1,…,nDo j=1,…,nPrint “Hello World”ClearRepeatRepeatPrint “End” f(n)=2n2+1 O(f(n) is n2
![Page 7: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/7.jpg)
The Quicksort Algorithm
Input: a list L=(a1,…,an) of numbers Choose an number p at random from L Determine L1 all numbers ai less than p
and L2 all numbers ai greater than or equal to p
Recursively sort L1 and L2
Solution=concatenate L1 and L2
![Page 8: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/8.jpg)
Quicksort Example
L=(1,5,2,7,6,4) sort p=4 [(1,2)(4,5,6,7)=(1,2,4,5,6,7)]L1=(1,2) L2=(5,7,6,4) sort L=(5,7,6,4) p=6 [(4,5)(6,7)=(4,5,6,7)] L1=(5,4), L2=(7,6) sort L=(5,4) [(4,5)] sort L=(7,6) [(6,7)]
Sort the list (1,5,2,7,6,4)
![Page 9: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/9.jpg)
Quicksort: The Worst Case
Given p partitioning into L1 and L2 is O(n) The worst possible case would be that
every time we partition the list we have L1 length n-1 and L2 length 1
For the i’th partition we have a computation O(n-i)
Now (n-i)=n2. Hence the entire computation is O(n2).
![Page 10: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/10.jpg)
Quicksort: On Average On average the split between L1
and L2 will result in two lists of n/2. This is based on the assumption that
the elements of the list are selected from an interval [x,y] according to a uniform distribution
Consider P(pai), the expected number of elements in L2 is nP(pai)
![Page 11: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/11.jpg)
Quicksort: On Average (2) p is a uniformly distributed random
variable taking values in [x,y] Therefore
xy
xaapP ii
ai is also a uniformly distributed random variable from [x,y] with expected value (x+y)/2.
Therefore, the expected value of P(pai) is ½ and the expected number of elements in L2 is n/2
![Page 12: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/12.jpg)
Quicksort: On Average (3)
Consider, the number of such partitions before we end up with strings of length 1. i.e. find k for which n/2k=1
Log(n/2k)=0 log(n)=log(2k)=k Hence, there are on average log(n)
partitions each requiring O(n) computations
This gives O(nlog(n))
![Page 13: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/13.jpg)
Polynomial Time
An algorithm runs in polynomial time if fA(n) is O(nk) for some positive integer k.
E.g. Quicksort always runs in polynomial time even in the worst case when it is O(n2)
The class P refers to those algorithms which run in polynomial time on a Turing Machine as described in the earlier section.
![Page 14: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/14.jpg)
Non-deterministic Turing Machines
A Non-deterministic Turing machine is a Turing machine where for some state qi and symbol S there is more than one instruction beginning qiS.
q1
q2
q3
1:R
0:R
![Page 15: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/15.jpg)
Computation Paths A deterministic Turing machine follows
a computation path. A Non-Deterministic generates a tree of
computations. We can either think of a NDTM as
following all paths simultaneously until it finds an accepting state
Or being a very lucky guesser between when choosing between states
![Page 16: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/16.jpg)
Accepting States and Paths
For NDTM we usually think of binary yes no responses from a computation.
We define a set of accepting state AQ. An accepting path is a path which
terminates in an accepting state. A NDTM accepts an input x if there is
an accepting path for that input.
![Page 17: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/17.jpg)
NDTM Example Let TMP be a deterministic Turing machine
that given a string of 1s and 0s terminates in state q+ if there are an even number of 1 and q- otherwise. AQ={q+}
Further suppose that the start state of TMP is q3
Given a string of 1s we mean substrings to be a string of the same length where some or all of the 1s have been switched to 0.
Eg 111 has substrings 111,110,101,100, 011,010,001,000
![Page 18: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/18.jpg)
NDTM Example: 2
q1 q2
q3
0:R
1:0
1:RTMP
1111
111
10211
011
11111
102
1
11131103 1011
Several deterministic steps1013
1003
0111
0011
0113
0103
0013
0003
q- q+
q+ q- q+ q- q- q+
![Page 19: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/19.jpg)
Non-Deterministic Polynomial
Intuitively, we see that a NDTM is more efficient than a DTM since it can explore many computational paths simultaneously.
NP refer to the class of algorithms which run in polynomial time on a NDTM.
Clearly PNP but does P=NP?
![Page 20: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/20.jpg)
NP Complete
NP-Complete are a collection of decision problems for which there is an algorithm in NP but no known algorithm in P.
Formally, a problem in NP is NP-Complete if any other problem in NP can be reduced to it.
Problem C is reducible to NP-Complete problem D if there is an algorithm A which solves C with a subroutine A+ that solves D, such if A+ ran in polynomial time so would A.
![Page 21: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/21.jpg)
SAT: An NP Complete Problem
SAT: Given a sentence of Propositional Logic S in CNF, is there an allocation of truth values for which S is true?
There is no known algorithm in P which solves SAT
This is basically because all (known) approaches require us (in the worst case) to check all possible truth value allocations. If S has n propositional variables there are 2n possible allocations of truth values.
![Page 22: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/22.jpg)
A NDTP algorithm for SAT Simplify things by assuming that S is in
CNF and each clause has exactly k literals for k>2. (k-SAT)
Notice that for any given allocation of truth values we can check if satisfies S in O(nk) – n is the number of variables in S.
There are 2n(2n-1)..(2n-k+1)/3! Different clauses.
Assume we can check if each is satisfied in 1 step then the computation is O(nk)
![Page 23: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/23.jpg)
SAT Algorithm: 2
Algorithm• For each variable guess either t or f. [O(n) on NDTM]• Check if allocation of truth values satisfies S [O(nk)]
S=(PR Q)(PRQ)
Pt f
Q Qt f t f
R R R Rt t t tf f f f
acc
ept
acc
ept
reje
ct
acc
ept
acc
ept
reje
ct
acc
ept
acc
ept
![Page 24: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/24.jpg)
Proving NP-Completeness
Usually a problem D is shown to be NP-complete by proving that a known NP-complete problem C reduces to it.All NP problems
reduce
NP complete C
D
All NP problems
reduce
Dreduce
![Page 25: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/25.jpg)
CLIQUE
In graph theory a CLIQUE is a set of vertices (nodes) where each is connected to the others.
6
4
3
5
2
1
5
2
1
Clique of size 3
![Page 26: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/26.jpg)
The CLIQUE Problem
For a given graph decide whether or not there is a clique of size k.
This has application in chip design where for example nodes are wires on a chip and edges signify that two wires can overlap.
CLIQUE tells the designer something about how much space is required on the chip.
We shall show that SAT reduces to CLIQUE So CLIQUE is NP-Complete.
![Page 27: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/27.jpg)
CLIQUE and SAT
Let S be a sentence in CNF with k clauses. S=C1C2…Ck We construct a graph G where Vertices={Li:LiCi} i.e. every literal from
every clause Edges={<Li,Lj> LiLj and ij} i.e. edges
between literals from different clauses provides they do not conflict.
S is satisfiable if and only if G has a clique of size k.
![Page 28: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/28.jpg)
CLIQUE and SAT: 2
Example S=(PQ)(PQ)(PR) L1=P, L2=Q, L3=P, L4=Q, L5=P, L6=R
L2
L5
L3
L6
L4
L1Graph G
![Page 29: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/29.jpg)
CLIQUE and SAT: 3 (If) Suppose there is a clique of size k then there
are k non-conflicting literals, one from each clause in S.
Since they are non-conflicting there is a truth allocation that makes each true and hence each clause in S true.
(Only If) Suppose there is an allocation of truth values for which S is true.
At least one literal from each clause must be true
These cannot be conflicting, hence they must form a clique of size k in G
![Page 30: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/30.jpg)
CLIQUE and SAT: 4
L2
L5
L3
L6
L4
L1Graph G
L2
L5L4
L2L6
L4
L6
L4
L1
L5L4
L1
L2
L3
L6
P=t,Q=t,R={t,f}
P=t,Q=t,R=t
P=t,Q=t,R={t,f}P={t,f},Q=t,R=tP=f,Q=t,R=t
![Page 31: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/31.jpg)
Other NP-Complete Problems: The Knapsack Problem
500g100
Weight: wi
Utility: ui
1kg200
300g400
750g80
Maximize 1i
iiux
Quantity: xi
subject to Cwx ii
i
Decision problem: is there a solution for which Vuxi
ii 1and Cwx i
ii ?
![Page 32: Computational Complexity Dr. Colin Campbell Course: EMAT20531](https://reader035.vdocument.in/reader035/viewer/2022081420/5515de04550346d46f8b4baf/html5/thumbnails/32.jpg)
Other NP-Complete Problems: The Travelling Salesman Problem
A B
C D
E
F
10
8 718
12
16 25
4
9
Starting at A find the shortest route which visits all towns and then returns to A.