asymptotic notation, recurrences · asymptotic growth rates let g : n ! r . o -notation: o (g ) is...
TRANSCRIPT
![Page 1: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/1.jpg)
Asymptotic Notation, Recurrences
Mary Cryan
24th September, 2013
ADS (2013/14) – Lecture 2 – slide 1
![Page 2: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/2.jpg)
Asymptotic growth rates
Let g : N→ R.
O-notation: O(g) is the set of all functions f : N→ R for which there areconstants c > 0 and n0 ≥ 0 such that
0 ≤ f (n) ≤ c · g(n), for all n ≥ n0.
“Rate of change of f (n) is at most that of g(n)”
Ω-notation: Ω(g) is the set of all functions f : N→ R for which there areconstants c > 0 and n0 ≥ 0 such that
0 ≤ c · g(n) ≤ f (n), for all n ≥ n0.
“Rate of change of f (n) is at least that of g(n)”
Θ-notation: Θ(g) is the set of all functions f : N→ R for which there areconstants c1, c2 > 0 and n0 ≥ 0 such that
0 ≤ c1 · g(n) ≤ f (n) ≤ c2 · g(n), for all n ≥ n0.
“Rate of change of f (n) and g(n) are about the same”
ADS (2013/14) – Lecture 2 – slide 2
![Page 3: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/3.jpg)
Examples
I Let f (n) = 0.01 · n2 and g(n) = n. Then g = O(f ).
I Let f (n) = ln(n) and g(n) = n. Then g = Ω(f ).
I Let f (n) = 10n + ln(n) and g(n) = n. Then g = Θ(n).
Sometimes O(. . .) appears within a formula, rather than simplyforming the right hand side of an equation. We make sense ofthis by thinking of O(. . .) as standing for some anonymous (butfixed) function from the set of the same name.For example, h(n) = 2O(n) means ∃c > 0, n0 ∈ N such that
h(n) ≤ 2cn for all n > n0.
ADS (2013/14) – Lecture 2 – slide 3
![Page 4: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/4.jpg)
Consequences
Suppose f (n) = O(g(n)) AND g(n) = O(f (n)). What can we say?
What if f (n) = O(g(n)) AND f (n) = Ω(g(n))?
Various consequences of the above conventions:
Θ(n)×Θ(n2) = Θ(n3),
Θ(n) +Θ(n2) = Θ(n2),
Θ(n) +Θ(n) = Θ(n).
ADS (2013/14) – Lecture 2 – slide 4
![Page 5: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/5.jpg)
Reminder of InsertionSort
Algorithm Insertion-Sort(A)
1. for j ← 2 to length[A]
2. do key ← A[j ]3. Insert A[j ] into the sorted sequence A[1 . . . j − 1]
4. i ← j − 15. while i > 0 and A[i ] > key6. do A[i + 1]← A[i ]7. i ← i − 18. A[i + 1]← key
Array A is indexed from j = 1 to n = length[A] (different from Java).
ADS (2013/14) – Lecture 2 – slide 5
![Page 6: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/6.jpg)
running-time of InsertionSort
I The for-loop on line 1 is iterated n − 1 times
I For each execution of the for, the while does ≤ j iterations;
I Each of the comparisons/assignments requires only O(1) basic steps;
I Therefore the total number of steps (=time) is at most
O(1)
n∑j=1
j = O(1)n(n + 1)
2= O(n2).
I This is essentially tight - sorting the list n, n − 1, n − 1, . . . , 3, 2, 1takes Ω(n2) time.
ADS (2013/14) – Lecture 2 – slide 6
![Page 7: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/7.jpg)
reminder of MergeSort
Input: A list A of natural numbers, p, r : 1 ≤ p ≤ r ≤ n.Output: A sorted (increasing order) permutation of A[p . . . r ].
Algorithm Merge-Sort(A, p, r)
1. if p < r then2. q ← bp+r
2 c3. Merge-Sort(A, p, q)
4. Merge-Sort(A, q + 1, r)5. Merge(A, p, q, r)
ADS (2013/14) – Lecture 2 – slide 7
![Page 8: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/8.jpg)
reminder of Merge
Input: A list A of whole numbers, 3 indices p, q, r : 1 ≤ p ≤ q < r , whereA[p . . . q] and A[q + 1 . . . r ] are individually sorted.Output: A sorted (increasing order) permutation of A[p . . . q].
(i) Define an output array B of length r − p + 1.(ii) Perform the merge, consisting of
I between minq − p + 1, r − q and r − p + 1 key comparisons, andbetween 2 minr − q, q − p + 1, r − p + 1 and 2(r − p + 1) indexcomparisons.
I exactly r − p + 1 copy operations (each key copied to B),I 2(r − p + 1) updates of the indices i , j , k as we run through A[p . . . q],
A[q + 1 . . . r ] and B[p . . . r ].I Another r − p + 1 copying operations copying B back into A.
The running-time of Merge is linear in the length of the array, specifically:
4(r−p+1)+3 minq−p+1, r−q, r−p+1 ≤ TMerge(p, q, r) ≤ 7(r−p+1)
Taking n = r − p + 1, we have
4n ≤ TMerge(n) ≤ 7n
ADS (2013/14) – Lecture 2 – slide 8
![Page 9: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/9.jpg)
Running-time of MergeSort
Putn = r − p + 1.
Running time TMS(n) satisfies:
TMS(n) =
Θ(1) if n = 1,
TMS(dn/2e) + TMS(bn/2c) +Θ(n) if n > 1.
The Θ(n) is from analysis of Merge on the previous slide. Analysis ofMergeSort gives bn+1
2 c and dn−12 e as the subarray sizes - these are
same as bn2c and dn2e.
ADS (2013/14) – Lecture 2 – slide 9
![Page 10: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/10.jpg)
Solving recurrences
Methods for deriving/verifying solutions to recurrences:
Induction Guess the solution and verify by induction on n.
Lovely if your recurrence is “NICE” enough that you canguess-and-verify. Rare.
Unfold and sum “Unfold” the recurrence by iterated substitution on the“neat” values of n (often power of 2 case). At some pointa pattern emerges. The “solution” is obtained byevaluating a sum that arises from the pattern.Since the pattern is just a guess for special n, the methodis rigorous only if we verify the solution (e.g., by a directinduction proof, or by the Master Theorem).
Often the only way to do the PROOF neatly is to RELATEto “neat” values of n . . . sometimes powers-of-2
“Master Theorem” Match the recurrence against a template. Read offthe solution from the Master Theorem.
ADS (2013/14) – Lecture 2 – slide 10
![Page 11: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/11.jpg)
Upper bound for MergeSort
Proof by “first principles”When working from first principles, need to replace “extra work” terms(Θ(n) for MergeSort) by terms with explicit constants.So check slide 8.
TMS(n) ≤
1 if n = 1,
TMS(dn/2e) + TMS(bn/2c) + 7n if n > 1.(1)
Unfold-and-sum will give a “guess” for the upper bound:
TMS(n) ≤ 7n lg(n) + n.
ADS (2013/14) – Lecture 2 – slide 11
![Page 12: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/12.jpg)
Upper bound for MergeSort
Proof by “first principles”When working from first principles, need to replace “extra work” terms(Θ(n) for MergeSort) by terms with explicit constants.So check slide 8.
TMS(n) ≤
1 if n = 1,
TMS(dn/2e) + TMS(bn/2c) + 7n if n > 1.(1)
Unfold-and-sum will give a “guess” for the upper bound:
TMS(n) ≤ 7n lg(n) + n.
ADS (2013/14) – Lecture 2 – slide 11
![Page 13: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/13.jpg)
Upper bound for MergeSort
Proof by “first principles”When working from first principles, need to replace “extra work” terms(Θ(n) for MergeSort) by terms with explicit constants.So check slide 8.
TMS(n) ≤
1 if n = 1,
TMS(dn/2e) + TMS(bn/2c) + 7n if n > 1.(1)
Unfold-and-sum will give a “guess” for the upper bound:
TMS(n) ≤ 7n lg(n) + n.
ADS (2013/14) – Lecture 2 – slide 11
![Page 14: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/14.jpg)
Upper bound for MergeSort (n a power-of-2)
T ′MS(n) =
1 if n = 1,
T ′MS(dn/2e) + T ′
MS(bn/2c) + 7n if n > 1.(2)
An equality version of (1). Clearly TMS(n) ≤ T ′MS(n) for n ∈ N.
claim (powers of 2): T ′MS(n) ≤ 7n lg(n) + n if n = 2k , for some k ∈ N
Proof (for powers of 2):Base case k = 0: direct from algorithm (1 comparison).Induction Hypothesis (IH): Upper bound holds for n = 2k−1.Induction Step: Now consider n = 2k and apply the recurrence:
T ′MS(n) = T ′
MS(d2k−1e) + T ′MS(b2k−1c) + 7n
= 2 · T ′MS(2k−1) + 7n
= 2 · 2k−1(7 lg(2k−1) + 1) + 7n ((IH))
= n · 7 lg(n/2) + n + 7n
= 7n(lg(n/2) + 1) + n = 7n lg(n) + n (by lg rules),
AS REQUIRED.
ADS (2013/14) – Lecture 2 – slide 12
![Page 15: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/15.jpg)
Upper bound for MergeSort (n a power-of-2)
T ′MS(n) =
1 if n = 1,
T ′MS(dn/2e) + T ′
MS(bn/2c) + 7n if n > 1.(2)
An equality version of (1). Clearly TMS(n) ≤ T ′MS(n) for n ∈ N.
claim (powers of 2): T ′MS(n) ≤ 7n lg(n) + n if n = 2k , for some k ∈ N
Proof (for powers of 2):Base case k = 0: direct from algorithm (1 comparison).Induction Hypothesis (IH): Upper bound holds for n = 2k−1.Induction Step: Now consider n = 2k and apply the recurrence:
T ′MS(n) = T ′
MS(d2k−1e) + T ′MS(b2k−1c) + 7n
= 2 · T ′MS(2k−1) + 7n
= 2 · 2k−1(7 lg(2k−1) + 1) + 7n ((IH))
= n · 7 lg(n/2) + n + 7n
= 7n(lg(n/2) + 1) + n = 7n lg(n) + n (by lg rules),
AS REQUIRED.
ADS (2013/14) – Lecture 2 – slide 12
![Page 16: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/16.jpg)
Upper bound for MergeSort (n a power-of-2)
T ′MS(n) =
1 if n = 1,
T ′MS(dn/2e) + T ′
MS(bn/2c) + 7n if n > 1.(2)
An equality version of (1). Clearly TMS(n) ≤ T ′MS(n) for n ∈ N.
claim (powers of 2): T ′MS(n) ≤ 7n lg(n) + n if n = 2k , for some k ∈ N
Proof (for powers of 2):Base case k = 0: direct from algorithm (1 comparison).Induction Hypothesis (IH): Upper bound holds for n = 2k−1.Induction Step: Now consider n = 2k and apply the recurrence:
T ′MS(n) = T ′
MS(d2k−1e) + T ′MS(b2k−1c) + 7n
= 2 · T ′MS(2k−1) + 7n
= 2 · 2k−1(7 lg(2k−1) + 1) + 7n ((IH))
= n · 7 lg(n/2) + n + 7n
= 7n(lg(n/2) + 1) + n = 7n lg(n) + n (by lg rules),
AS REQUIRED.
ADS (2013/14) – Lecture 2 – slide 12
![Page 17: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/17.jpg)
Upper bound for MergeSort (n a power-of-2)
T ′MS(n) =
1 if n = 1,
T ′MS(dn/2e) + T ′
MS(bn/2c) + 7n if n > 1.(2)
An equality version of (1). Clearly TMS(n) ≤ T ′MS(n) for n ∈ N.
claim (powers of 2): T ′MS(n) ≤ 7n lg(n) + n if n = 2k , for some k ∈ N
Proof (for powers of 2):
Base case k = 0: direct from algorithm (1 comparison).Induction Hypothesis (IH): Upper bound holds for n = 2k−1.Induction Step: Now consider n = 2k and apply the recurrence:
T ′MS(n) = T ′
MS(d2k−1e) + T ′MS(b2k−1c) + 7n
= 2 · T ′MS(2k−1) + 7n
= 2 · 2k−1(7 lg(2k−1) + 1) + 7n ((IH))
= n · 7 lg(n/2) + n + 7n
= 7n(lg(n/2) + 1) + n = 7n lg(n) + n (by lg rules),
AS REQUIRED.
ADS (2013/14) – Lecture 2 – slide 12
![Page 18: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/18.jpg)
Upper bound for MergeSort (n a power-of-2)
T ′MS(n) =
1 if n = 1,
T ′MS(dn/2e) + T ′
MS(bn/2c) + 7n if n > 1.(2)
An equality version of (1). Clearly TMS(n) ≤ T ′MS(n) for n ∈ N.
claim (powers of 2): T ′MS(n) ≤ 7n lg(n) + n if n = 2k , for some k ∈ N
Proof (for powers of 2):Base case k = 0: direct from algorithm (1 comparison).
Induction Hypothesis (IH): Upper bound holds for n = 2k−1.Induction Step: Now consider n = 2k and apply the recurrence:
T ′MS(n) = T ′
MS(d2k−1e) + T ′MS(b2k−1c) + 7n
= 2 · T ′MS(2k−1) + 7n
= 2 · 2k−1(7 lg(2k−1) + 1) + 7n ((IH))
= n · 7 lg(n/2) + n + 7n
= 7n(lg(n/2) + 1) + n = 7n lg(n) + n (by lg rules),
AS REQUIRED.
ADS (2013/14) – Lecture 2 – slide 12
![Page 19: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/19.jpg)
Upper bound for MergeSort (n a power-of-2)
T ′MS(n) =
1 if n = 1,
T ′MS(dn/2e) + T ′
MS(bn/2c) + 7n if n > 1.(2)
An equality version of (1). Clearly TMS(n) ≤ T ′MS(n) for n ∈ N.
claim (powers of 2): T ′MS(n) ≤ 7n lg(n) + n if n = 2k , for some k ∈ N
Proof (for powers of 2):Base case k = 0: direct from algorithm (1 comparison).Induction Hypothesis (IH): Upper bound holds for n = 2k−1.
Induction Step: Now consider n = 2k and apply the recurrence:
T ′MS(n) = T ′
MS(d2k−1e) + T ′MS(b2k−1c) + 7n
= 2 · T ′MS(2k−1) + 7n
= 2 · 2k−1(7 lg(2k−1) + 1) + 7n ((IH))
= n · 7 lg(n/2) + n + 7n
= 7n(lg(n/2) + 1) + n = 7n lg(n) + n (by lg rules),
AS REQUIRED.
ADS (2013/14) – Lecture 2 – slide 12
![Page 20: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/20.jpg)
Upper bound for MergeSort (n a power-of-2)
T ′MS(n) =
1 if n = 1,
T ′MS(dn/2e) + T ′
MS(bn/2c) + 7n if n > 1.(2)
An equality version of (1). Clearly TMS(n) ≤ T ′MS(n) for n ∈ N.
claim (powers of 2): T ′MS(n) ≤ 7n lg(n) + n if n = 2k , for some k ∈ N
Proof (for powers of 2):Base case k = 0: direct from algorithm (1 comparison).Induction Hypothesis (IH): Upper bound holds for n = 2k−1.Induction Step: Now consider n = 2k and apply the recurrence:
T ′MS(n) = T ′
MS(d2k−1e) + T ′MS(b2k−1c) + 7n
= 2 · T ′MS(2k−1) + 7n
= 2 · 2k−1(7 lg(2k−1) + 1) + 7n ((IH))
= n · 7 lg(n/2) + n + 7n
= 7n(lg(n/2) + 1) + n = 7n lg(n) + n (by lg rules),
AS REQUIRED.
ADS (2013/14) – Lecture 2 – slide 12
![Page 21: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/21.jpg)
Upper bound for MergeSort (n a power-of-2)
T ′MS(n) =
1 if n = 1,
T ′MS(dn/2e) + T ′
MS(bn/2c) + 7n if n > 1.(2)
An equality version of (1). Clearly TMS(n) ≤ T ′MS(n) for n ∈ N.
claim (powers of 2): T ′MS(n) ≤ 7n lg(n) + n if n = 2k , for some k ∈ N
Proof (for powers of 2):Base case k = 0: direct from algorithm (1 comparison).Induction Hypothesis (IH): Upper bound holds for n = 2k−1.Induction Step: Now consider n = 2k and apply the recurrence:
T ′MS(n) = T ′
MS(d2k−1e) + T ′MS(b2k−1c) + 7n
= 2 · T ′MS(2k−1) + 7n
= 2 · 2k−1(7 lg(2k−1) + 1) + 7n ((IH))
= n · 7 lg(n/2) + n + 7n
= 7n(lg(n/2) + 1) + n = 7n lg(n) + n (by lg rules),
AS REQUIRED.
ADS (2013/14) – Lecture 2 – slide 12
![Page 22: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/22.jpg)
Upper bound for MergeSort (n a power-of-2)
T ′MS(n) =
1 if n = 1,
T ′MS(dn/2e) + T ′
MS(bn/2c) + 7n if n > 1.(2)
An equality version of (1). Clearly TMS(n) ≤ T ′MS(n) for n ∈ N.
claim (powers of 2): T ′MS(n) ≤ 7n lg(n) + n if n = 2k , for some k ∈ N
Proof (for powers of 2):Base case k = 0: direct from algorithm (1 comparison).Induction Hypothesis (IH): Upper bound holds for n = 2k−1.Induction Step: Now consider n = 2k and apply the recurrence:
T ′MS(n) = T ′
MS(d2k−1e) + T ′MS(b2k−1c) + 7n
= 2 · T ′MS(2k−1) + 7n
= 2 · 2k−1(7 lg(2k−1) + 1) + 7n ((IH))
= n · 7 lg(n/2) + n + 7n
= 7n(lg(n/2) + 1) + n = 7n lg(n) + n (by lg rules),
AS REQUIRED.
ADS (2013/14) – Lecture 2 – slide 12
![Page 23: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/23.jpg)
Upper bound for MergeSort (n a power-of-2)
T ′MS(n) =
1 if n = 1,
T ′MS(dn/2e) + T ′
MS(bn/2c) + 7n if n > 1.(2)
An equality version of (1). Clearly TMS(n) ≤ T ′MS(n) for n ∈ N.
claim (powers of 2): T ′MS(n) ≤ 7n lg(n) + n if n = 2k , for some k ∈ N
Proof (for powers of 2):Base case k = 0: direct from algorithm (1 comparison).Induction Hypothesis (IH): Upper bound holds for n = 2k−1.Induction Step: Now consider n = 2k and apply the recurrence:
T ′MS(n) = T ′
MS(d2k−1e) + T ′MS(b2k−1c) + 7n
= 2 · T ′MS(2k−1) + 7n
= 2 · 2k−1(7 lg(2k−1) + 1) + 7n ((IH))
= n · 7 lg(n/2) + n + 7n
= 7n(lg(n/2) + 1) + n = 7n lg(n) + n (by lg rules),
AS REQUIRED.
ADS (2013/14) – Lecture 2 – slide 12
![Page 24: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/24.jpg)
Upper bound for MergeSort (n a power-of-2)
T ′MS(n) =
1 if n = 1,
T ′MS(dn/2e) + T ′
MS(bn/2c) + 7n if n > 1.(2)
An equality version of (1). Clearly TMS(n) ≤ T ′MS(n) for n ∈ N.
claim (powers of 2): T ′MS(n) ≤ 7n lg(n) + n if n = 2k , for some k ∈ N
Proof (for powers of 2):Base case k = 0: direct from algorithm (1 comparison).Induction Hypothesis (IH): Upper bound holds for n = 2k−1.Induction Step: Now consider n = 2k and apply the recurrence:
T ′MS(n) = T ′
MS(d2k−1e) + T ′MS(b2k−1c) + 7n
= 2 · T ′MS(2k−1) + 7n
= 2 · 2k−1(7 lg(2k−1) + 1) + 7n ((IH))
= n · 7 lg(n/2) + n + 7n
= 7n(lg(n/2) + 1) + n = 7n lg(n) + n (by lg rules),
AS REQUIRED.
ADS (2013/14) – Lecture 2 – slide 12
![Page 25: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/25.jpg)
Upper bound for MergeSort (general n)
Note: Argument above WON’T imply TMS(n) ′ ≤ 7n lg(n) + n for general n.
Non power-of-2 bound: We have n ∈ N, n not an power of 2.
STEP 1: Prove that TMS(n) ′ is monotone increasing.
Induction Hypothesis (IH):If n ∈ N then T ′MS(n) < T ′
MS(m) for all n < m.
Base Case (n = 1): T ′MS(1) = 1. For m > 2, T ′
MS(m) ≥ m ≥ 2 > T ′MS(1).
Induction Step (n): Suppose true for all k ∈ N, k < n. Consider n. By n ≥ 2,the recurrence holds for n, so
T ′MS(n) = T ′
MS(dn/2e) + T ′MS(bn/2c) + 7n. (3)
For m > n, apply the recurrence to get
TMS(m) = T ′MS(dm/2e) + T ′
MS(bm/2c) + 7m.
By b n2c, d n
2e < n (need strict <), we can apply the (IH) with k = b n2c or k = d n
2e.By b n
2c ≤ bm2 c and d n
2e ≤ dm2 e our (IH) tells us that 1st two terms for T ′
MS(m)are ≥ than the partner terms for T ′
MS(n).But also 7n < 7m . . . ⇒ T ′
MS(n) < T ′MS(m).
So by Induction, STEP 1 is proved.
ADS (2013/14) – Lecture 2 – slide 13
![Page 26: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/26.jpg)
Upper bound for MergeSort (general n)
Note: Argument above WON’T imply TMS(n) ′ ≤ 7n lg(n) + n for general n.
Non power-of-2 bound: We have n ∈ N, n not an power of 2.
STEP 1: Prove that TMS(n) ′ is monotone increasing.
Induction Hypothesis (IH):If n ∈ N then T ′MS(n) < T ′
MS(m) for all n < m.
Base Case (n = 1): T ′MS(1) = 1. For m > 2, T ′
MS(m) ≥ m ≥ 2 > T ′MS(1).
Induction Step (n): Suppose true for all k ∈ N, k < n. Consider n. By n ≥ 2,the recurrence holds for n, so
T ′MS(n) = T ′
MS(dn/2e) + T ′MS(bn/2c) + 7n. (3)
For m > n, apply the recurrence to get
TMS(m) = T ′MS(dm/2e) + T ′
MS(bm/2c) + 7m.
By b n2c, d n
2e < n (need strict <), we can apply the (IH) with k = b n2c or k = d n
2e.By b n
2c ≤ bm2 c and d n
2e ≤ dm2 e our (IH) tells us that 1st two terms for T ′
MS(m)are ≥ than the partner terms for T ′
MS(n).But also 7n < 7m . . . ⇒ T ′
MS(n) < T ′MS(m).
So by Induction, STEP 1 is proved.
ADS (2013/14) – Lecture 2 – slide 13
![Page 27: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/27.jpg)
Upper bound for MergeSort (general n)
Note: Argument above WON’T imply TMS(n) ′ ≤ 7n lg(n) + n for general n.
Non power-of-2 bound: We have n ∈ N, n not an power of 2.
STEP 1: Prove that TMS(n) ′ is monotone increasing.
Induction Hypothesis (IH):If n ∈ N then T ′MS(n) < T ′
MS(m) for all n < m.
Base Case (n = 1): T ′MS(1) = 1. For m > 2, T ′
MS(m) ≥ m ≥ 2 > T ′MS(1).
Induction Step (n): Suppose true for all k ∈ N, k < n. Consider n. By n ≥ 2,the recurrence holds for n, so
T ′MS(n) = T ′
MS(dn/2e) + T ′MS(bn/2c) + 7n. (3)
For m > n, apply the recurrence to get
TMS(m) = T ′MS(dm/2e) + T ′
MS(bm/2c) + 7m.
By b n2c, d n
2e < n (need strict <), we can apply the (IH) with k = b n2c or k = d n
2e.By b n
2c ≤ bm2 c and d n
2e ≤ dm2 e our (IH) tells us that 1st two terms for T ′
MS(m)are ≥ than the partner terms for T ′
MS(n).But also 7n < 7m . . . ⇒ T ′
MS(n) < T ′MS(m).
So by Induction, STEP 1 is proved.
ADS (2013/14) – Lecture 2 – slide 13
![Page 28: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/28.jpg)
Upper bound for MergeSort (general n)
Note: Argument above WON’T imply TMS(n) ′ ≤ 7n lg(n) + n for general n.
Non power-of-2 bound: We have n ∈ N, n not an power of 2.
STEP 1: Prove that TMS(n) ′ is monotone increasing.
Induction Hypothesis (IH):If n ∈ N then T ′MS(n) < T ′
MS(m) for all n < m.
Base Case (n = 1): T ′MS(1) = 1. For m > 2, T ′
MS(m) ≥ m ≥ 2 > T ′MS(1).
Induction Step (n): Suppose true for all k ∈ N, k < n. Consider n. By n ≥ 2,the recurrence holds for n, so
T ′MS(n) = T ′
MS(dn/2e) + T ′MS(bn/2c) + 7n. (3)
For m > n, apply the recurrence to get
TMS(m) = T ′MS(dm/2e) + T ′
MS(bm/2c) + 7m.
By b n2c, d n
2e < n (need strict <), we can apply the (IH) with k = b n2c or k = d n
2e.By b n
2c ≤ bm2 c and d n
2e ≤ dm2 e our (IH) tells us that 1st two terms for T ′
MS(m)are ≥ than the partner terms for T ′
MS(n).But also 7n < 7m . . . ⇒ T ′
MS(n) < T ′MS(m).
So by Induction, STEP 1 is proved.
ADS (2013/14) – Lecture 2 – slide 13
![Page 29: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/29.jpg)
Upper bound for MergeSort (general n)
Note: Argument above WON’T imply TMS(n) ′ ≤ 7n lg(n) + n for general n.
Non power-of-2 bound: We have n ∈ N, n not an power of 2.
STEP 1: Prove that TMS(n) ′ is monotone increasing.
Induction Hypothesis (IH):If n ∈ N then T ′MS(n) < T ′
MS(m) for all n < m.
Base Case (n = 1): T ′MS(1) = 1. For m > 2, T ′
MS(m) ≥ m ≥ 2 > T ′MS(1).
Induction Step (n): Suppose true for all k ∈ N, k < n. Consider n. By n ≥ 2,the recurrence holds for n, so
T ′MS(n) = T ′
MS(dn/2e) + T ′MS(bn/2c) + 7n. (3)
For m > n, apply the recurrence to get
TMS(m) = T ′MS(dm/2e) + T ′
MS(bm/2c) + 7m.
By b n2c, d n
2e < n (need strict <), we can apply the (IH) with k = b n2c or k = d n
2e.By b n
2c ≤ bm2 c and d n
2e ≤ dm2 e our (IH) tells us that 1st two terms for T ′
MS(m)are ≥ than the partner terms for T ′
MS(n).But also 7n < 7m . . . ⇒ T ′
MS(n) < T ′MS(m).
So by Induction, STEP 1 is proved.
ADS (2013/14) – Lecture 2 – slide 13
![Page 30: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/30.jpg)
Upper bound for MergeSort (general n)
Note: Argument above WON’T imply TMS(n) ′ ≤ 7n lg(n) + n for general n.
Non power-of-2 bound: We have n ∈ N, n not an power of 2.
STEP 1: Prove that TMS(n) ′ is monotone increasing.
Induction Hypothesis (IH):If n ∈ N then T ′MS(n) < T ′
MS(m) for all n < m.
Base Case (n = 1): T ′MS(1) = 1. For m > 2, T ′
MS(m) ≥ m ≥ 2 > T ′MS(1).
Induction Step (n): Suppose true for all k ∈ N, k < n. Consider n. By n ≥ 2,the recurrence holds for n, so
T ′MS(n) = T ′
MS(dn/2e) + T ′MS(bn/2c) + 7n. (3)
For m > n, apply the recurrence to get
TMS(m) = T ′MS(dm/2e) + T ′
MS(bm/2c) + 7m.
By b n2c, d n
2e < n (need strict <), we can apply the (IH) with k = b n2c or k = d n
2e.By b n
2c ≤ bm2 c and d n
2e ≤ dm2 e our (IH) tells us that 1st two terms for T ′
MS(m)are ≥ than the partner terms for T ′
MS(n).But also 7n < 7m . . . ⇒ T ′
MS(n) < T ′MS(m).
So by Induction, STEP 1 is proved.
ADS (2013/14) – Lecture 2 – slide 13
![Page 31: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/31.jpg)
Upper bound for MergeSort (general n)
Note: Argument above WON’T imply TMS(n) ′ ≤ 7n lg(n) + n for general n.
Non power-of-2 bound: We have n ∈ N, n not an power of 2.
STEP 1: Prove that TMS(n) ′ is monotone increasing.
Induction Hypothesis (IH):If n ∈ N then T ′MS(n) < T ′
MS(m) for all n < m.
Base Case (n = 1): T ′MS(1) = 1. For m > 2, T ′
MS(m) ≥ m ≥ 2 > T ′MS(1).
Induction Step (n): Suppose true for all k ∈ N, k < n. Consider n. By n ≥ 2,the recurrence holds for n, so
T ′MS(n) = T ′
MS(dn/2e) + T ′MS(bn/2c) + 7n. (3)
For m > n, apply the recurrence to get
TMS(m) = T ′MS(dm/2e) + T ′
MS(bm/2c) + 7m.
By b n2c, d n
2e < n (need strict <), we can apply the (IH) with k = b n2c or k = d n
2e.By b n
2c ≤ bm2 c and d n
2e ≤ dm2 e our (IH) tells us that 1st two terms for T ′
MS(m)are ≥ than the partner terms for T ′
MS(n).But also 7n < 7m . . . ⇒ T ′
MS(n) < T ′MS(m).
So by Induction, STEP 1 is proved.
ADS (2013/14) – Lecture 2 – slide 13
![Page 32: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/32.jpg)
Upper bound for MergeSort (general n)
Note: Argument above WON’T imply TMS(n) ′ ≤ 7n lg(n) + n for general n.
Non power-of-2 bound: We have n ∈ N, n not an power of 2.
STEP 1: Prove that TMS(n) ′ is monotone increasing.
Induction Hypothesis (IH):If n ∈ N then T ′MS(n) < T ′
MS(m) for all n < m.
Base Case (n = 1): T ′MS(1) = 1. For m > 2, T ′
MS(m) ≥ m ≥ 2 > T ′MS(1).
Induction Step (n): Suppose true for all k ∈ N, k < n. Consider n. By n ≥ 2,the recurrence holds for n, so
T ′MS(n) = T ′
MS(dn/2e) + T ′MS(bn/2c) + 7n. (3)
For m > n, apply the recurrence to get
TMS(m) = T ′MS(dm/2e) + T ′
MS(bm/2c) + 7m.
By b n2c, d n
2e < n (need strict <), we can apply the (IH) with k = b n2c or k = d n
2e.
By b n2c ≤ bm
2 c and d n2e ≤ dm
2 e our (IH) tells us that 1st two terms for T ′MS(m)
are ≥ than the partner terms for T ′MS(n).
But also 7n < 7m . . . ⇒ T ′MS(n) < T ′
MS(m).So by Induction, STEP 1 is proved.
ADS (2013/14) – Lecture 2 – slide 13
![Page 33: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/33.jpg)
Upper bound for MergeSort (general n)
Note: Argument above WON’T imply TMS(n) ′ ≤ 7n lg(n) + n for general n.
Non power-of-2 bound: We have n ∈ N, n not an power of 2.
STEP 1: Prove that TMS(n) ′ is monotone increasing.
Induction Hypothesis (IH):If n ∈ N then T ′MS(n) < T ′
MS(m) for all n < m.
Base Case (n = 1): T ′MS(1) = 1. For m > 2, T ′
MS(m) ≥ m ≥ 2 > T ′MS(1).
Induction Step (n): Suppose true for all k ∈ N, k < n. Consider n. By n ≥ 2,the recurrence holds for n, so
T ′MS(n) = T ′
MS(dn/2e) + T ′MS(bn/2c) + 7n. (3)
For m > n, apply the recurrence to get
TMS(m) = T ′MS(dm/2e) + T ′
MS(bm/2c) + 7m.
By b n2c, d n
2e < n (need strict <), we can apply the (IH) with k = b n2c or k = d n
2e.By b n
2c ≤ bm2 c and d n
2e ≤ dm2 e our (IH) tells us that 1st two terms for T ′
MS(m)are ≥ than the partner terms for T ′
MS(n).
But also 7n < 7m . . . ⇒ T ′MS(n) < T ′
MS(m).So by Induction, STEP 1 is proved.
ADS (2013/14) – Lecture 2 – slide 13
![Page 34: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/34.jpg)
Upper bound for MergeSort (general n)
Note: Argument above WON’T imply TMS(n) ′ ≤ 7n lg(n) + n for general n.
Non power-of-2 bound: We have n ∈ N, n not an power of 2.
STEP 1: Prove that TMS(n) ′ is monotone increasing.
Induction Hypothesis (IH):If n ∈ N then T ′MS(n) < T ′
MS(m) for all n < m.
Base Case (n = 1): T ′MS(1) = 1. For m > 2, T ′
MS(m) ≥ m ≥ 2 > T ′MS(1).
Induction Step (n): Suppose true for all k ∈ N, k < n. Consider n. By n ≥ 2,the recurrence holds for n, so
T ′MS(n) = T ′
MS(dn/2e) + T ′MS(bn/2c) + 7n. (3)
For m > n, apply the recurrence to get
TMS(m) = T ′MS(dm/2e) + T ′
MS(bm/2c) + 7m.
By b n2c, d n
2e < n (need strict <), we can apply the (IH) with k = b n2c or k = d n
2e.By b n
2c ≤ bm2 c and d n
2e ≤ dm2 e our (IH) tells us that 1st two terms for T ′
MS(m)are ≥ than the partner terms for T ′
MS(n).But also 7n < 7m . . . ⇒ T ′
MS(n) < T ′MS(m).
So by Induction, STEP 1 is proved.
ADS (2013/14) – Lecture 2 – slide 13
![Page 35: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/35.jpg)
Upper bound for MergeSort (general n)
Note: Argument above WON’T imply TMS(n) ′ ≤ 7n lg(n) + n for general n.
Non power-of-2 bound: We have n ∈ N, n not an power of 2.
STEP 1: Prove that TMS(n) ′ is monotone increasing.
Induction Hypothesis (IH):If n ∈ N then T ′MS(n) < T ′
MS(m) for all n < m.
Base Case (n = 1): T ′MS(1) = 1. For m > 2, T ′
MS(m) ≥ m ≥ 2 > T ′MS(1).
Induction Step (n): Suppose true for all k ∈ N, k < n. Consider n. By n ≥ 2,the recurrence holds for n, so
T ′MS(n) = T ′
MS(dn/2e) + T ′MS(bn/2c) + 7n. (3)
For m > n, apply the recurrence to get
TMS(m) = T ′MS(dm/2e) + T ′
MS(bm/2c) + 7m.
By b n2c, d n
2e < n (need strict <), we can apply the (IH) with k = b n2c or k = d n
2e.By b n
2c ≤ bm2 c and d n
2e ≤ dm2 e our (IH) tells us that 1st two terms for T ′
MS(m)are ≥ than the partner terms for T ′
MS(n).But also 7n < 7m . . . ⇒ T ′
MS(n) < T ′MS(m).
So by Induction, STEP 1 is proved.
ADS (2013/14) – Lecture 2 – slide 13
![Page 36: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/36.jpg)
Upper bound for MergeSort (general n) cont’d.
STEP 2: Choose a “power of 2” to relate to n.
I Want an upper bound, so need a power of 2 greater than n.
I So define n = 2dlg(n)e (this will be “m”).
I We know n ≤ n but n < 2n.
I Monotonicity property from STEP 1 tells us T ′MS(n) ≤ T ′
MS(n)
I Proof of Upper bound for powers of 2 tells us T ′MS(n) ≤ 7n lg(n) + n.
I By n < 2n, we get
T ′MS(n) ≤ T ′
MS(n) = 7n(lg(n)) + n < 7(2n) lg(2n) + 2n = 14n lg(n) + 16n.
So for any n ∈ N we have T ′MS(n) ≤ 14n lg(n) + 16n.
The “truth” is quite a bit better, but not well-suited to an inductive proof.
ADS (2013/14) – Lecture 2 – slide 14
![Page 37: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/37.jpg)
Upper bound for MergeSort (general n) cont’d.
STEP 2: Choose a “power of 2” to relate to n.
I Want an upper bound, so need a power of 2 greater than n.
I So define n = 2dlg(n)e (this will be “m”).
I We know n ≤ n but n < 2n.
I Monotonicity property from STEP 1 tells us T ′MS(n) ≤ T ′
MS(n)
I Proof of Upper bound for powers of 2 tells us T ′MS(n) ≤ 7n lg(n) + n.
I By n < 2n, we get
T ′MS(n) ≤ T ′
MS(n) = 7n(lg(n)) + n < 7(2n) lg(2n) + 2n = 14n lg(n) + 16n.
So for any n ∈ N we have T ′MS(n) ≤ 14n lg(n) + 16n.
The “truth” is quite a bit better, but not well-suited to an inductive proof.
ADS (2013/14) – Lecture 2 – slide 14
![Page 38: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/38.jpg)
Upper bound for MergeSort (general n) cont’d.
STEP 2: Choose a “power of 2” to relate to n.
I Want an upper bound, so need a power of 2 greater than n.
I So define n = 2dlg(n)e (this will be “m”).
I We know n ≤ n but n < 2n.
I Monotonicity property from STEP 1 tells us T ′MS(n) ≤ T ′
MS(n)
I Proof of Upper bound for powers of 2 tells us T ′MS(n) ≤ 7n lg(n) + n.
I By n < 2n, we get
T ′MS(n) ≤ T ′
MS(n) = 7n(lg(n)) + n < 7(2n) lg(2n) + 2n = 14n lg(n) + 16n.
So for any n ∈ N we have T ′MS(n) ≤ 14n lg(n) + 16n.
The “truth” is quite a bit better, but not well-suited to an inductive proof.
ADS (2013/14) – Lecture 2 – slide 14
![Page 39: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/39.jpg)
Upper bound for MergeSort (general n) cont’d.
STEP 2: Choose a “power of 2” to relate to n.
I Want an upper bound, so need a power of 2 greater than n.
I So define n = 2dlg(n)e (this will be “m”).
I We know n ≤ n but n < 2n.
I Monotonicity property from STEP 1 tells us T ′MS(n) ≤ T ′
MS(n)
I Proof of Upper bound for powers of 2 tells us T ′MS(n) ≤ 7n lg(n) + n.
I By n < 2n, we get
T ′MS(n) ≤ T ′
MS(n) = 7n(lg(n)) + n < 7(2n) lg(2n) + 2n = 14n lg(n) + 16n.
So for any n ∈ N we have T ′MS(n) ≤ 14n lg(n) + 16n.
The “truth” is quite a bit better, but not well-suited to an inductive proof.
ADS (2013/14) – Lecture 2 – slide 14
![Page 40: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/40.jpg)
Upper bound for MergeSort (general n) cont’d.
STEP 2: Choose a “power of 2” to relate to n.
I Want an upper bound, so need a power of 2 greater than n.
I So define n = 2dlg(n)e (this will be “m”).
I We know n ≤ n but n < 2n.
I Monotonicity property from STEP 1 tells us T ′MS(n) ≤ T ′
MS(n)
I Proof of Upper bound for powers of 2 tells us T ′MS(n) ≤ 7n lg(n) + n.
I By n < 2n, we get
T ′MS(n) ≤ T ′
MS(n) = 7n(lg(n)) + n < 7(2n) lg(2n) + 2n = 14n lg(n) + 16n.
So for any n ∈ N we have T ′MS(n) ≤ 14n lg(n) + 16n.
The “truth” is quite a bit better, but not well-suited to an inductive proof.
ADS (2013/14) – Lecture 2 – slide 14
![Page 41: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/41.jpg)
Upper bound for MergeSort (general n) cont’d.
STEP 2: Choose a “power of 2” to relate to n.
I Want an upper bound, so need a power of 2 greater than n.
I So define n = 2dlg(n)e (this will be “m”).
I We know n ≤ n but n < 2n.
I Monotonicity property from STEP 1 tells us T ′MS(n) ≤ T ′
MS(n)
I Proof of Upper bound for powers of 2 tells us T ′MS(n) ≤ 7n lg(n) + n.
I By n < 2n, we get
T ′MS(n) ≤ T ′
MS(n) = 7n(lg(n)) + n < 7(2n) lg(2n) + 2n = 14n lg(n) + 16n.
So for any n ∈ N we have T ′MS(n) ≤ 14n lg(n) + 16n.
The “truth” is quite a bit better, but not well-suited to an inductive proof.
ADS (2013/14) – Lecture 2 – slide 14
![Page 42: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/42.jpg)
Upper bound for MergeSort (general n) cont’d.
STEP 2: Choose a “power of 2” to relate to n.
I Want an upper bound, so need a power of 2 greater than n.
I So define n = 2dlg(n)e (this will be “m”).
I We know n ≤ n but n < 2n.
I Monotonicity property from STEP 1 tells us T ′MS(n) ≤ T ′
MS(n)
I Proof of Upper bound for powers of 2 tells us T ′MS(n) ≤ 7n lg(n) + n.
I By n < 2n, we get
T ′MS(n) ≤ T ′
MS(n) = 7n(lg(n)) + n < 7(2n) lg(2n) + 2n = 14n lg(n) + 16n.
So for any n ∈ N we have T ′MS(n) ≤ 14n lg(n) + 16n.
The “truth” is quite a bit better, but not well-suited to an inductive proof.
ADS (2013/14) – Lecture 2 – slide 14
![Page 43: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/43.jpg)
Upper bound for MergeSort (general n) cont’d.
STEP 2: Choose a “power of 2” to relate to n.
I Want an upper bound, so need a power of 2 greater than n.
I So define n = 2dlg(n)e (this will be “m”).
I We know n ≤ n but n < 2n.
I Monotonicity property from STEP 1 tells us T ′MS(n) ≤ T ′
MS(n)
I Proof of Upper bound for powers of 2 tells us T ′MS(n) ≤ 7n lg(n) + n.
I By n < 2n, we get
T ′MS(n) ≤ T ′
MS(n) = 7n(lg(n)) + n < 7(2n) lg(2n) + 2n = 14n lg(n) + 16n.
So for any n ∈ N we have T ′MS(n) ≤ 14n lg(n) + 16n.
The “truth” is quite a bit better, but not well-suited to an inductive proof.
ADS (2013/14) – Lecture 2 – slide 14
![Page 44: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/44.jpg)
Upper bound for MergeSort (general n) cont’d.
STEP 2: Choose a “power of 2” to relate to n.
I Want an upper bound, so need a power of 2 greater than n.
I So define n = 2dlg(n)e (this will be “m”).
I We know n ≤ n but n < 2n.
I Monotonicity property from STEP 1 tells us T ′MS(n) ≤ T ′
MS(n)
I Proof of Upper bound for powers of 2 tells us T ′MS(n) ≤ 7n lg(n) + n.
I By n < 2n, we get
T ′MS(n) ≤ T ′
MS(n) = 7n(lg(n)) + n < 7(2n) lg(2n) + 2n = 14n lg(n) + 16n.
So for any n ∈ N we have T ′MS(n) ≤ 14n lg(n) + 16n.
The “truth” is quite a bit better, but not well-suited to an inductive proof.
ADS (2013/14) – Lecture 2 – slide 14
![Page 45: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/45.jpg)
Notes on the proof
Our “first principles” proof is essentially a direct proof of a sub-case ofthe Master Theorem.
For “first principles” O(·) proofs where b·c, d·e is involved, there are a fewcommon steps:
I Consider an equality version T ′(·) of the recurrence T (·) such thatT (n) ≤ T ′(n) holds for all n ∈ N.
I prove the result for T ′ for the “NEAT” case (power-of-2 here, butwould be power-of-d if bn/dc, dn/de was involved)
I STEP 1: Prove T ′(n) is monotonically increasing with n forgeneral n.
I STEP 2: Consider the closest power-of-d greater than n, say n, for anon-neat n ∈ N. Then exploit T (n) ≤ T ′(n) (by definition),T ′(n) ≤ T ′(n) (by STEP 1), and then substitute in the provenupper bound for T ′(n) (because n is “NEAT”).
ADS (2013/14) – Lecture 2 – slide 15
![Page 46: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/46.jpg)
Notes on the proof
Our “first principles” proof is essentially a direct proof of a sub-case ofthe Master Theorem.
For “first principles” O(·) proofs where b·c, d·e is involved, there are a fewcommon steps:
I Consider an equality version T ′(·) of the recurrence T (·) such thatT (n) ≤ T ′(n) holds for all n ∈ N.
I prove the result for T ′ for the “NEAT” case (power-of-2 here, butwould be power-of-d if bn/dc, dn/de was involved)
I STEP 1: Prove T ′(n) is monotonically increasing with n forgeneral n.
I STEP 2: Consider the closest power-of-d greater than n, say n, for anon-neat n ∈ N. Then exploit T (n) ≤ T ′(n) (by definition),T ′(n) ≤ T ′(n) (by STEP 1), and then substitute in the provenupper bound for T ′(n) (because n is “NEAT”).
ADS (2013/14) – Lecture 2 – slide 15
![Page 47: Asymptotic Notation, Recurrences · Asymptotic growth rates Let g : N ! R . O -notation: O (g ) is the set of all functions f : N ! R for which there are constants c > 0 and n 0 0](https://reader035.vdocument.in/reader035/viewer/2022081406/5f128c7343bd736dce7e8406/html5/thumbnails/47.jpg)
Reading and Working
Reading Assignment
Inf2B ADS Lecture Notes 2 and 8.
[CLRS] Sections 2.1, 2.2 and 2.3 (of 3rd or 2nd edition). Also Section 3.1(omitting the bits on the little-o and little-ω notation at the end).
(all this material should be familiar from Inf2B and your math classes)
Tutorials (will start week 3, on Tuesday)
Fill the Doodle poll TODAY linked-to from the ADS course webpage (ifyou haven’t yet)
ADS (2013/14) – Lecture 2 – slide 16