asymptotic growth rate. asymptotic running time the running time of an algorithm as input size...
Post on 19-Dec-2015
213 Views
Preview:
TRANSCRIPT
Asymptotic Growth Rate
Asymptotic Running Time
• The running time of an algorithm as input size approaches infinity is called the asymptotic running time
• We study different notations for asymptotic efficiency.
• In particular, we study tight bounds, upper bounds and lower bounds.
Outline
• Why do we need the different sets?• Definition of the sets O (Oh),
(Omega) and (Theta), o (oh), (omega)
• Classifying examples:– Using the original definition– Using limits
The functions
• Let f(n) and g(n) be asymptotically nonnegative functions whose domains are the set of natural numbers N={0,1,2,…}.
• A function g(n) is asymptotically nonnegative, if g(n)0 for all nn0 where n0N
Big Oh
• Big “oh” - asymptotic upper bound on the growth of an algorithm
• When do we use Big Oh?1. To provide information on the maximum
number of operations that an algorithm performs– Insertion sort is O(n2) in the worst case
• This means that in the worst case it performs at most cn2 operations
2. Theory of NP-completeness1. An algorithm is polynomial if it is O(nk) for some constant k2. P = NP if there is any polynomial time algorithm for any NP-
complete problem Note: Don’t worry about NP now; it is to be discussed much later
in the semester
Definition of Big Oh
• O(f(n)) is the set of functions g(n) such that:there exist positive constants c, and N for which,
0 g(n) cf(n) for all n N• f(n) is called an asymptotically upper
bound for g(n); that is, g(n) is O(f(n)).
g(n) O(f(n))
N n
cf(n)
g(n)
I grow at
most as
fast as f
n2 + 10 n O(n2) Why?
0200400600800
100012001400
0 10 20 30
n2 + 10n
2 n2
take c = 2N = 10n2+10n <=2n2 for all n>=10
Does 5n+2 O(n)?
Proof: From the definition of Big Oh, there must exist c > 0 and integer N > 0 such that 0 5n+2 cn for all n N.
Dividing both sides of the inequality by n > 0 we get: 0 5+2/n c.– 2/n ( > 0) becomes smaller as n increases– For instance, let N = 2 and c = 6
There are many choices here for c and N.
Does 5n+2 O(n)?
If we choose N = 1 then 5+2/n 5+2/1= 7. So any c 7 works. Let’s choose c = 7.
If we choose c = 6, then 0 5+2/n 6. So any N 2 works. Choose N = 2.
In either case (we only need one!) , c > 0 and N > 0 such that 0 5n+2 cn for all n N. So the definition is satisfied and
5n+2 O(n)
Does n2 O(n)? No. We will prove by contradiction that the
definition cannot be satisfied. Assume that n2 O(n).From the definition of Big Oh, there must
exist c > 0 and integer N > 0 such that 0 n2 cn for all n N.
Divide the inequality by n > 0 to get 0 n c for all n N.
n c cannot be true for any n > max{c,N }. This contradicts the assumption. Thus, n2 O(n).
Are they true? • 1,000,000 n2 O(n2) why/why not? • True
• (n - 1)n / 2 O(n2) why /why not? • True
• n / 2 O(n2) why /why not? • True
• lg (n2) O( lg n ) why /why not? • True
• n2 O(n) why /why not? • False
Omega
Asymptotic lower bound on the growth of an algorithm or a problem
When do we use Omega?1. To provide information on the minimum
number of operations that an algorithm performs– Insertion sort is (n) in the best case
• This means that in the best case its instruction count is at least cn,
– It is (n2) in the worst case• This means that in the worst case its instruction
count is at least cn2
Omega (cont.)
2. To provide information on a class of algorithms that solve a problem – Sorting algorithms based on comparison of
keys are (nlgn) in the worst case• This means that all sort algorithms based only on
comparison of keys have to do at least cnlgn operations
– Any algorithm based only on comparison of keys to find the maximum of n elements is (n) in every case
• This means that all algorithms based only on comparison of keys to find maximum have to do at least cn operations
Supplementary topic: Why (nlgn) for sorting?
• n numbers to sort and no special assumptions unlike pigeonhole sorting (we discussed in a previous class), there are n! permutations
• One comparison has only two outcomes• So, lg(n!) comparisons are required in
the worst case• n! is approximately equal to (n/e)n
– Striling’s approximation
n2
Definition of the set Omega
• (f(n)) is the set of functions g(n) such that:there exist positive constants c, and N for which, 0 cf(n) g(n) for all n N
• f(n) is called an asymptotically lower bound for g(n); that is, g(n) =(f(n))
g(n) (f(n))
N n
cf(n)
g(n)
I grow at
least as
fast as f
Is 5n-20 (n)?Proof: From the definition of Omega, there must exist
c>0 and integer N>0 such that 0 cn 5n-20 for all nN
Dividing the inequality by n > 0 we get: 0 c 5-20/n for all n N.
20/n 20, and 20/n becomes smaller as n grows.
There are many choices here for c and N. Since c > 0, 5 – 20/n > 0 and N >4. If we choose c=1, then 5 – 20/n 1 and N 5 Choose N = 5.If we choose c=4, then 5 – 20/n 4 and N 20. Choose N
= 20.
In either case (we only need one!) we have c>o and N>0 such that 0 cn 5n-20 for all n N. So 5n-20 (n).
Are they true?• 1,000,000 n2 (n2) why /why not?• (true)• (n - 1)n / 2(n2) why /why not? • (true)
• n / 2(n2) why /why not? • (false)• lg (n2) ( lg n ) why /why not? • (true)• n2 (n) why /why not? • (true)
Theta
• Asymptotic tight bound on the growth rate of an algorithm – Insertion sort is (n2) in the worst and
average cases• This means that in the worst case and average
cases insertion sort performs cn2 operations
– Binary search is (lg n) in the worst and average cases
• The means that in the worst case and average cases binary search performs clgn operations
Definition of the set Theta
• (f(n)) is the set of functions g(n) such that:there exist positive constants c, d, and N, for which,
0 cf(n) g(n) df(n) for all nN
• f(n) is called an asymptotically tight bound for g(n).
g(n) (f(n))
N n
cf(n)
g(n)
df(n)
We grow
at same
rate
Another Definition of Theta
))(())(())(( nfnfOnf
log n n2 n5
5n + 3 1/2 n2 2n
n1.5 5n2+ 3nnlog n
(n2)
O(n2) (n2)
Small o(n2)
Small (n2)
• We use the last definition and show:
1.
2.
?)(32
1 Does 22 nnn
)(32
1 22 nOnn
)(32
1 22 nnn
?)(32
1 Does 22 nOnn
6 Choose 6. all for
. all for
:get weby inequality the Dividing
. all for
that such and, exist must there definition the From
Choose
chosen be can 1/2 any learly
NNn
Nncn
n
Nncnnn
Nc
c
cC
2
13210
3210
02
232210
00
.2/1
?)(32
1 Does 22 nnn
. and So
. all for
1/4.c Choose finite for 0 Since
6. Since
.
that such and exist must There
1241
12321
41
.2/1 ,3
and 32100
3210
get we02by Dividing
allfor 322120
00
N/c
nn
cn/n
N, c
nc
n
Nnnncn
Nc
N
More • 1,000,000 n2 (n2) why /why not? • (true)
• (n - 1)n / 2(n2) why /why not? • (true)
• n / 2(n2) why /why not? • (false)
• lg (n2) ( lg n ) why /why not? • (true) • n2 (n) why /why not? • (false)
small o
(f(n)) is the set of functions g(n) which satisfy the following condition:For every positive real constant c, there exists a positive integer N, for which,
g(n) cf(n) for all nN
small o
• Little “oh” - used to denote an upper bound that is not asymptotically tight. – n is in o(n3). – n is not in o(n)
small omega
(f(n)) is the set of functions g(n) which satisfy the following condition:For every positive real constant c, there exists a positive integer N, for which,
g(n) cf(n) for all nN
small omega and small o
g(n)(f(n)) if and only if f(n) o(g(n))
Limits can be used to determine Order
c then f (n) = ( g (n)) if c > 0
if lim f (n)/g (n) = 0 then f (n) = o ( g(n))
then f (n) = ( g (n))
• The limit must exist
{n
Example using limits
22
3
2
3
23
3lim
5lim
35lim
,since)(35
n
n
n
n
n
nn
nnn
nnn
L’Hopital’s Rule
If f(x) and g(x) are both differentiable with derivatives f’(x) and g’(x), respectively, and if
exists right the on limit the whenever
then
)('
)('lim
)(
)(lim
)(lim)(lim
xg
xf
xg
xf
xfxg
xx
xx
Example using limits
ax
kk
n
e
n
e
n
e
n
e
nnn
k
k
a
y
xy
n
n
nn
n
n
nn
nonn
n
n
n
n
n
nn
nnn
ln
)!1()1()(
2
2
33
3
3
3
33
1
log
01
/1lim
)'(
)'(loglim
:Rule sHopital'L' Use?log
limlog
lim
,since)(log
103
lim10
lim310
lim
,since )(310
(kth order differentiation of y)
Example using limit
02ln
1lim
'
)'(lglim
lglim
2ln
1'
2ln
ln'lg and
2ln
lnlg
)(lg
nn
n
n
n
n
nn
nn
non
nnn
Example using limits
02ln2
!lim
...2ln2
)1(lim
2ln2lim
2lim
)(')'(:
)2(2ln2ln''2
2
integer positive a is where)2(
2
21
2ln2ln
2ln
knn
n
k
nn
k
nn
k
n
xx
nnnn
nn
nk
k
nkkknn
exeNote
ee
e
kon
Further study of the growth functions
• Properties of growth functions
O <= >= = o < >
Asymptotic Growth RatePart II
Outline
• More examples:• General Properties• Little Oh• Additional properties
Order of AlgorithmOrder of Algorithm
• Property - Complexity Categories:
θ(lg n) θ(n) θ(n lgn) θ(n2) θ(nj) θ(nk) θ(an) θ(bn) θ(n!)
Where k>j>2 and b>a>1. If a complexity
function g(n) is in a category that is to the left of the category containing f(n), then g(n) o(f(n))
Comparing ln n with na (a > 0)
• Using limits we get:
• So ln n = o(na) for any a > 0• When the exponent a is very small,
we need to look at very large values of n to see that na > ln n
01
limln
lim anan ann
n
Values for log10n and n 0.01
0.01
n log n n.01
1 0 11.00E+10 10 1.258925
1.00E+100 100 101.00E+200 200 1001.00E+230 230 199.52621.00E+240 240 251.1886
Values for log10n and n 0.001
n log n n.001
1 0 11.00E+10 10 1.023293
1.00E+100 100 1.2589251E+1000 1000 101E+2000 2000 1001E+3000 3000 10001E+4000 4000 10000
Lower-order terms and constants
• Lower order terms of a function do not matter since lower-order terms are dominated by the higher order term.
• Constants (multiplied by highest order term) do not matter, since they do not affect the asymptotic growth rate
• All logarithms with base b >1 belong to (lg n) since constant a is wherelg
lg
lglog cnc
b
nnb
General Rules• We say a function f (n ) is polynomially bounded
if f (n ) = O ( nk ) for some positive constant k
• We say a function f (n ) is polylogarithmic bounded if f (n ) = O ( lgk n) for some positive constant k
• Exponential functions – grow faster than positive polynomial
functions• Polynomial functions
– grow faster than polylogarithmic functions
More properties
• The following slides show– Two examples of pairs of functions that
are not comparable in terms of asymptotic notation
– How the asymptotic notation can be used in equations
– That Theta, Big Oh, and Omega define a transitive and reflexive order. Theta also satisfies symmetry, while Big Oh and Omega satisfy transpose symmetry
Are n and nsinn comparable with respect to growth rate? yes
Increases from 0 to 1
Increases from 1 to n
Decreases from 1 to 0
Decreases from n to 1
Decreases from 0 to –1
Decreases from 1 to 1/n
Increases from –1 to 0
Increases from 1/n to 1
nsin nnsin
Clearly nsin n = O(n), but nsin n (n)
0
50
100
150
200
250
300
350
400
450
500
550
600
650
700
750
800
850
900
950
1000
1050
1100
0 50 100
150
200
250
300
350
400
450
500
550
600
650
700
750
800
850
900
950
1000
1050
1100
1150
1200
1250
1300
1350
1400
1450
1500
1550
1600
sin(n)
n̂ sin(n)
n
5n
Are n and nsinn+1 comparable with respect to growth rate? no
Increases from 0 to 1
Increases from n to n2
Decreases from 1 to 0
Decreases from n2 to n
Decreases from 0 to –1
Decreases from n to 1
Increases from –1 to 0
Increases from 1 to n
nsin 1sin nn
Clearly nsin n+1 O(n), but nsin n+1 (n)
1
10
100
1000
10000
100000
0 20 40 60 80 100
120
140
160
180
200
220
240
260
280
300
320
340
n
n (̂sin(n)+1)
Are n and nsinn+1 comparable? No
Another example
The following functions are not asymptotically comparable:
nn
nng
n
nnnf
oddfor
evenfor 1)(
oddfor 1
evenfor )(
,))(()( and ,))(()( ngnfngOnf
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
f(n)
g(n)
Transitivity:
If f (n) = (g(n )) and g (n) = (h(n )) then f (n) = (h(n )) .
If f (n) = (g(n )) and g (n) = (h(n )) then f (n) = (h(n )).
If f (n) = (g(n )) and g (n) = (h(n )) then f (n) = (h(n )) .
If f (n) = (g(n )) and g (n) = (h(n )) then f (n) = (h(n )) .
If f (n) = (g(n )) and g (n) = (h(n )) then f (n) = (h(n ))
Reflexivity:
f (n) = (f (n )).
f (n) = (f (n )).
f (n) = (f (n )).
“o” is not reflexive
“” is not reflexive
Symmetry and Transpose symmetry
• Symmetry:f (n) = (g(n)) if and only if g (n) = (f (n )) .
• Transpose symmetry: f (n) = (g(n )) if and only if g (n) = (f (n )). f (n) = (g(n )) if and only if g (n) = (f (n )).
Analogy between asymptotic comparison of functions and comparison of real numbers.
f (n) = ( g(n)) a b
f (n) = ( g(n)) a b
f (n) = ( g(n)) a b
f (n) = ( g(n)) a b
f (n) = ( g(n)) a bf(n) is asymptotically smaller than g(n) if f (n) = ( g(n))
f(n) is asymptotically larger than g(n) if f (n) = ( g(n))
Is O(g(n)) = (g(n)) o(g(n))?
We show a counter example:
The functions are:
g(n) = nand
f(n)O(n) but f(n) (n) and f(n) o(n) Conclusion:
O(g(n)) (g(n)) o(g(n))
even is if 1
odd is if )(
n
nnnf
top related