1 complexity lecture 17-19 ref. handout p 66 - 77
TRANSCRIPT
1
Complexity
Lecture 17-19
Ref. Handout p 66 - 77
2
Measuring Program Performance
How can one algorithm be ‘better’ than another?
Time taken to execute ? Size of code ? Space used when running ? Scaling properties ?
3
Scaling
Amount of data
Time
4
Time and Counting
initialisefor (i = 1; i<=n; i++) {
do something}finish
Time needed:
t1set up loop t2each iteration t4
t5
The total ‘cost’ is t1+t2+n∗t4+t5 Instead of actual time, ti can be treated
as how many instructions needed
5
The computation cost can be estimatedas a function f(x) In the above example,
cost is f(x) = t1+t2+x∗t4+t5
= c + m∗x ( c = t1+t2+t5, m = t4 )
f(x) =c+mx
x (size of data)
c
m = slope
a linear function
6
The computation cost can be estimatedas a function f(x) – more examples
cost functions can be any type depending on the algorithm, e.g.f(x) = c ( c is a constant ), h(x) = log(x), or g(x) = x2
f(x) =c
x (size of data)
c
cost stays same
h(x) = log(x)
cost growsvery slowg(x) =x2
quadratic growth
7
The Time Complexity
A cost function for an algorithm indicates how computation time grows when the amount of data increases
A special term for it – TIME COMPLEXITYcomplexity = time complexity/space complexitySpace complexity studies the memory usage (but mostly we pay our attention to time complexity)
“Algorithm A’s time complexity is linear” = A’s computation time is proportional to the size of A’s input data
8
The Big-O Notation
How to compare (evaluate) two cost functions
Is n+100*n2 better than 1000*n2
It has been agreed to use a standard measurement – an order of magnitude notation – the Big-O notation
9
The Big-O Notation - definition
A function f(x) is said to be the big-O of a function g(x) if there is an integer N and a constant C such that
f(x) ≤ C∗g(x) for all x ≥ N
We write
f(x) = O(g(x))
10
The Big-O Notation - Examples
f(x) = 3∗xg(x) = xC = 3, N =0f(x) ≤ 3∗g(x)
f(x) = 4+7∗xg(x) = xC = 8, N =4f(x) ≤ 8∗g(x)
g(x) = x
f(x) = 3x = 3g(x)f(x) = 4+7x
g(x) = x
8*g(x)
3 x∗ = O(x) 4+7 x∗ = O(x)N
11
The Big-O Notation - Examples
f(x) = x+100∗x2
g(x) = x2
C = 101, N = 1f(x) ≤ 101∗g(x)
f(x) = 20+x+5∗x2+8∗x3
g(x) = x3
C = 15, N =20f(x) ≤ 9∗g(x)
when x > 1x+100∗x2 < x2+100∗x2
x2+100∗x2 = 101*g(x)
x+100∗x2= O(x2)
when x > 2020+x+5∗x2+8∗x3 <x3+x3+5∗x3+8x3
= 15∗x3 = 15*g(x)20+x+5∗x2+8∗x3= O(x3)
12
Power of n
A smaller power is always better eventually.for larger enough n
The biggest power counts most.for larger enough n
1000000000 * n < n2
10 *n4 + 23*n3 + 99*n2 + 200*n + 77 < 11*n4
13
Using Big-O Notation for classifyingalgorithms
23*n3 + 11
1000000000000000*n3 +100000000*n2 + ...
or even 100*n2 + n
All cost functions above is O(n3)
Performance is no worse than C*n3 eventually
14
Working Out Big-O
Ignore constants in multiplication - 10*n is O(n)
Leave products – n*log(n) is O( n*log(n) )
Use worst case in sums - n2 + 100*n + 2 is O(n2)
15
Memo for In-class test 17 [ /5]
questions my answers
correct answers
comments
1
2
3
4
5
16
Some Typical Algorithms - Search
Search an array of size n for a particular item
Sequential search
Time taken is O( )
Binary search
Time taken is O( )
17
Some Typical Algorithms - Sorting
Insertion sort Use 2 lists – unsorted and sorted
insertionSort(input_list) {
unsorted = input_list (assume size = n)
sorted = an empty list
loop from i =0 to n-1 do {
insert (unsorted[i], sorted) }
return sorted
}
Time taken is O( )
18
Some Typical Algorithms - Sorting
Bubble sort
Swap if xi < xi-1 for all pairs
Do this n times (n = length of the list)
x0 3
x1 5
x2 1
x3 2
x4 6
Time taken is O( )
19
Some Typical Algorithms - Sorting
Quick sort
quickSort(list) {
If List has zero or one number return list
Randomly pick a pivot xi to create two partitions larger than xi and smaller than xi
return quickSort(SmallerList)
+ xi + quickSort(LargerList)
}
// ‘+’ means appending two lists into one
Time taken is O( )
20
Worst Cases, Best Cases, and Average Cases
Complexity:performance on the average cases
Worst/Best cases:The worst (best) performance which an algorithm can get
E.g. 1 - bubble sort and insertion sort can get O(n) in the best case (when the list is already ordered)
E.g. 2 – quick sort can get O(n2) in the worst case (when partition doesn’t work well)
21
Sorting a sorted list
123456789
22
A Summary on Typical Algorithms
Sequential search O(n) Binary search (sorted list) O(log(n))
Bubble sort O(n2) best O(n) worst O(n2) Insertion sort O(n2) best O(n) worst O(n2)
Quick sort O(log(n)*n) best O(log(n)*n)
worst O(n2)
23
Can you order these functions ?smaller comes first
log(n)
log(n)*n n4/3
√n
n 2n
(log(n))2
n2
24
Memo for In-class test 18 [ /5]
questions my answers
correct answers
comments
1
2
3
4
5
25
Speeding up the Machine
can solve problem of size N in one day machine speed
up
Complexity
X 1000 X 1000000
O( log(n) ) N1000 N1000000
O( n ) 1000 * N 1000000 * N
O( n2 ) 32 * N 1000 * N
O( n3 ) 10 * N 100 * N
O( 2n ) N + 10 N + 20
size
26
Code Breaking
A number is the product of two primes
How to work out prime factors?
i.e. 35 = 7*5
5959 = 59*101
48560209712519 = 6850049 * 7089031
Complexity is O(10n) where n is the number of digits ( or O(s) where s is the number)
Typically n is about 150
27
Cost of Speeding Up
For each additional digit multiply time by 10( as the complexity is O(10n) )
For 10 extra digitsincrease time from 1 second to 317 years
For 18 extra digitincrease time from 1 second totwice the age of the universe
28
Where Different Formulas Appear
O(log(n)) find an item in an ordered collection
O( n ) look at every item
O( n2 ) for every item look at every other item
O( 2n ) look at every subset of the items
29
Do Big-O Times Matter?
Is n3 worse than 100000000000000000*n ?
Algorithm A is O(n2) for 99.999% of casesand
O(2n) for 0.001% cases
Algorithm B is always O(n10)
Which one is better?
30
Degree of Difficulty
log(n)
n*log(n)
nn, n (2^n)
n, n2, n3, ......
2n, 3n, ......
polynomial time
‘tractable’
exponential time
‘intractable’
31
A Simple O(2n) Problem
Given n objects of different weights,
split them into two equally heavy piles
(or say if this isn’t possible)
32
Hard Problem Make Easy
Magic Machine(for problem A)
Possible Datafor problem A
CorrectAnswer? Answer
checker
33
NP Problems
A problem is in the class NP
(Non-deterministic Polynomial Time)
if the checking requires polynomial time
i.e. O(nc) where C is a fixed number)
Note: problems not algorithms
34
Example
Given n objects of different weights,
split them into two equally heavy piles
{1,2,3,5,7,8}
Algorithm
1+2+3+7 = 8+5
Checkingok ✔
O(2n)
O(n)
35
In Previous Example ...
We know there are algorithms takes O(2n)
But ......
How do we know there isn’t a better algorithm which only takes polynomial time anyway?
36
Some Classes of Problems
P
NP
problems which canbe solved in polynomialtime using a magic box
problems which can be solved in polynomial time
37
Problems in NP but not in P
NPP
NPP?=
is a subset of
but
Most problems forwhich the best knownalgorithm is believed to be O(2n) are in NP
P
NP
38
The Hardest NP Problems
If a polynomial time algorithm is found for any problem in NP-Complete then every problem in NP can be solved in polynomial time.
i,e, P = NP
P
NP
NP-Complete
39
Examples of NP-Complete Problems
The travelling salesman problem Finding the shortest common
superstring Checking whether two finite automata
accept the same language Given three positive integers a, b, and
c, do there exist positive integers such that a*x2 + b*y2 = c
40
How to be Famous
Find a polynomial time algorithm for this problem
Given n objects of different weights,
split them into two equally heavy piles
(or say if this isn’t possible)
41
Memo for In-class test 19 [ /5]
questions my answers
correct answers
comments
1
2
3
4
5