2il50 data structures spring 2015 lecture 3: heaps
TRANSCRIPT
![Page 1: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/1.jpg)
2IL50 Data Structures
Spring 2015
Lecture 3: Heaps
![Page 2: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/2.jpg)
Solving recurrences
one more time …
![Page 3: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/3.jpg)
Solving recurrences
Easiest: Master theoremcaveat: not always applicable
Alternatively: Guess the solution and use the substitution method to prove that your guess it is correct.
How to guess:
1. expand the recursion
2. draw a recursion tree
![Page 4: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/4.jpg)
Example(A)
► A is an array of length n
1. n = A.length
2. if n==1
3. then return A[1]
4. else begin
5. Copy A[1… n/2 ] to auxiliary array B[1... n/2 ]
6. Copy A[1… n/2 ] to auxiliary array C[1… n/2 ]
7. b = Example(B); c = Example(C)
8. for i = 1 to n
9. do for j = 1 to i
10. do A[i] = A[j]
11. return 43
12. end
Example
![Page 5: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/5.jpg)
Example(A)
► A is an array of length n
1. n = A.length
2. if n==1
3. then return A[1]
4. else begin
5. Copy A[1… n/2 ] to auxiliary array B[1... n/2 ]
6. Copy A[1… n/2 ] to auxiliary array C[1… n/2 ]
7. b = Example(B); c = Example(C)
8. for i = 1 to n
9. do for j = 1 to i
10. do A[i] = A[j]
11. return 43
12. end
Example
Let T(n) be the worst case running time of Example on an array of length n.
Lines 1,2,3,4,11, and 12 take Θ(1) time.Lines 5 and 6 take Θ(n) time.Line 7 takes Θ(1) + 2 T( n/2 ) time.Lines 8 until 10 take
time.
If n=1 lines 1,2,3 are executed,else lines 1,2, and 4 until 12 are executed.
➨ T(n):
➨ use master theorem …
)Θ(nΘ(i)Θ(1) 2n
1i
n
1i
i
1j
Θ(1) if n=1
2T(n/2) + Θ(n2) if n>1
![Page 6: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/6.jpg)
Let a and b be constants, let f(n) be a function, and let T(n)be defined on the nonnegative integers by the recurrence
T(n) = aT(n/b) + f(n)
Then we have:
1. If f(n) = O(nlog a – ε) for some constant ε > 0, then T(n) = Θ(nlog a).
2. If f(n) = Θ(nlog a), then T(n) = Θ(nlog a log n)
3. If f(n) = Ω(nlog a + ε) for some constant ε > 0, and if af(n/b) ≤ cf(n) for some constant c < 1 and all sufficiently large n, then T(n) = Θ(f(n))
The master theorem
b
b
b
b
b
![Page 7: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/7.jpg)
Quiz
Recurrence
1. T(n) = 4 T(n/2) + Θ(n3)
2. T(n) = 4 T(n/2) + Θ(n)
3. T(n) = T(n/2) + 1
4. T(n) = T(n/3) + T(2n/3) + n
5. T(n) = 9 T(n/3) + Θ(n2)
6. T(n) = √n T(√n) + n
Master theorem?
yes
yes
yes
no
yes
no
T(n) = Θ(n3)
T(n) = Θ(n2)
T(n) = Θ(log n)
T(n) = Θ(n log n)
T(n) = Θ(n2 log n)
T(n) = Θ(n log log n)
![Page 8: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/8.jpg)
Tips
Analysis of recursive algorithms:find the recursion and solve with master theorem if possible
Analysis of loops: summations
Some standard recurrences and sums:
T(n) = 2T(n/2) + Θ(n) ➨
½ n(n+1) = Θ(n2)
Θ(n3)
T(n) = Θ(n log n)
n
1i
i
n
1i
2i
![Page 9: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/9.jpg)
Heaps
![Page 10: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/10.jpg)
Event-driven simulation
Stores a set of events, processes first event (highest priority)
Supporting data structure: insert event find (and extract) event with highest priority change the priority of an event
![Page 11: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/11.jpg)
Priority queue
Max-priority queueabstract data type (ADT) that stores a set S of elements, each with an associated key (integer value).
Operations
Insert(S, x): inserts element x into S, that is, S ← S ⋃ {x}
Maximum(S): returns the element of S with the largest key
Extract-Max(S): removes and returns the element of S with the largest key
Increase-Key(S, x, k): give key[x] the value k
condition: k is larger than the current value of key[x]
Min-priority queue …
![Page 12: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/12.jpg)
Implementing a priority queue
Today
Insert Maximum Extract-Max Increase-Key
sorted list
sorted array
Θ(1) Θ(1)
Θ(1)
Θ(n)
Θ(n)
Θ(n)
Θ(n)Θ(n)
Insert Maximum Extract-Max Increase-Key
heap Θ(1)Θ(log n) Θ(log n)Θ(log n)
![Page 13: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/13.jpg)
Max-heap
Heapnearly complete binary tree, filled on all levels except possibly the lowest.(lowest level is filled from left to right)
Max-heap property: for every node i other than the root
key[Parent(i)] ≥ key[i]
35
1930
1112830
517 2
35
1930
21830
35
1930
12830
![Page 14: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/14.jpg)
Properties of a max-heap
LemmaThe largest element in a max-heap is stored at the root.
Proof:
![Page 15: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/15.jpg)
Properties of a max-heap
LemmaThe largest element in a max-heap is stored at the root.
Proof: x root
y arbitrary node
z1, z2, …, zk nodes on path between x and y
max-heap property ➨ key[x] ≥ key[z1] ≥ … ≥ key[zk] ≥ key[y]
➨ the largest element is stored at x ■
x
y
z1
z2
![Page 16: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/16.jpg)
Implementing a heap with an array
A.length = length of array A
heap-size[A] =number of elements in the heap
35
1930
1112830
217 5
35 30 19 30 8 12 11 17 2 5
1 2 3heap-size[A]
4
array A[1 … A.length]
![Page 17: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/17.jpg)
Implementing a heap with an array
kth node on level j is stored at position
left child of node at position i = Left(i) =
right child of node at position i = Right(i) =
parent of node at position i = Parent(i) =
1 2 3heap-size[A]
4
array A[1 … A.length]
level 0
level 1
level 2, position 3
A[2j+k-1]
2i
2i + 1
i / 2
![Page 18: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/18.jpg)
Priority queue
Max-priority queueabstract data type (ADT) that stores a set S of elements, each with an associated key (integer value).
Operations
Insert(S, x): inserts element x into S, that is, S ← S ⋃ {x}
Maximum(S): returns the element of S with the largest key
Extract-Max(S): removes and returns the element of S with the largest key
Increase-Key(S, x, k): give key[x] the value k
condition: k is larger than the current value of key[x]
![Page 19: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/19.jpg)
Implementing a max-priority queue
Set S is stored as a heap in an array A.
Operations: Maximum, Extract-Max, Insert, Increase-Key.
35 30 19 30 8 12 11 17 2 5
1 2 3heap-size[A]
4
![Page 20: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/20.jpg)
Implementing a max-priority queue
Set S is stored as a heap in an array A.
Operations: Maximum, Extract-Max, Insert, Increase-Key.
Maximum(A)
1. if heap-size[A] < 12. then error3. else return A[1]
running time: O(1)
35 30 19 30 8 12 11 17 2 5
1 2 3heap-size[A]
4
![Page 21: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/21.jpg)
Implementing a max-priority queue
Set S is stored as a heap in an array A.
Operations: Maximum, Extract-Max, Insert, Increase-Key.
35 30 19 30 8 12 11 17 2 5
1 2 3heap-size[A]
4
![Page 22: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/22.jpg)
17
3530
Implementing a max-priority queue
Set S is stored as a heap in an array A.
Operations: Maximum, Extract-Max, Insert, Increase-Key.
1930
1112830
217 5
![Page 23: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/23.jpg)
Set S is stored as a heap in an array A.
Operations: Maximum, Extract-Max, Insert, Increase-Key.
Heap-Extract-Max(A)
1. if heap-size[A] < 1 2. then error3. max = A[1]4. A[1] = A [heap-size[A]]5. heap-size[A] = heap-size[A]–16. Max-Heapify(A,1)7. return max
355
30
Implementing a max-priority queue
1930
11128
217 5
restore heap property
![Page 24: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/24.jpg)
Max-Heapify
Max-Heapify(A, i)
► ensures that the heap whose root is stored at position i has the max-heap property
► assumes that the binary trees rooted at Left(i) and Right(i) are max-heaps
![Page 25: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/25.jpg)
10
1035
40
Max-Heapify
Max-Heapify(A, i)
► ensures that the heap whose root is stored at position i has the max-heap property
► assumes that the binary trees rooted at Left(i) and Right(i) are max-heaps
Max-Heapify(A,1)
exchange A[1] with largest child
Max-Heapify(A,2)
exchange A[2] with largest child
Max-Heapify(A,5)
A[5] larger than its children ➨ done.
30
10
1940
111235
217 5
![Page 26: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/26.jpg)
Max-Heapify
Max-Heapify(A, i)
► ensures that the heap whose root is stored at position i has themax-heap property
► assumes that the binary trees rooted at Left(i) and Right(i) are max-heaps
1. if Left(i) ≤ heap-size[A] and A[Left(i)] > A[i]2. then largest = Left(i)3. else largest = i4. if Right(i) ≤ heap-size[A] and A[Right(i)] > A[largest]5. then largest = Right(i)6. if largest ≠ i7. then exchange A[i ] and A[largest]8. Max-Heapify(A, largest)
running time? O(height of the subtree rooted at i) = O(log n)
![Page 27: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/27.jpg)
Implementing a max-priority queue
Set S is stored as a heap in an array A.
Operations: Maximum, Extract-Max, Insert, Increase-Key.
Insert (A, key)
1. heap-size[A] = heap-size[A] +12. A[heap-size[A]] = - infinity3. Increase-Key(A, heap-size[A], key)
![Page 28: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/28.jpg)
Implementing a max-priority queue
Set S is stored as a heap in an array A.
Operations: Maximum, Extract-Max, Insert, Increase-Key.
![Page 29: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/29.jpg)
Implementing a max-priority queue
Set S is stored as a heap in an array A.
Operations: Maximum, Extract-Max, Insert, Increase-Key.
Increase-Key(A, i, key)
1. if key < A[i]2. then error3. A[i] = key4. while i>1 and A[parent(i)] < A[i]5. do exchange A[parent(i)] and A[i]6. i = parent(i)
running time: O(log n)
30
35
1930
11128
217 5
42
![Page 30: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/30.jpg)
Building a heap
Build-Max-Heap(A)
► Input: array A[1..n] of numbers► Output: array A[1..n] with the same numbers, but rearranged, such
that the max-heap property holds
1. heap-size = A.length2. for i = A.length downto 13. do Max-Heapify(A,i)
P(k): nodes k, k+1, …, n are each the root of a max-heap
Loop InvariantP(k+1) holds before line 3 is executed with i=k, P(k) holds afterwards
starting at A.length / 2 is sufficient
![Page 31: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/31.jpg)
Building a heap
Build-Max-Heap(A)
1. heap-size = A.length2. for i = A.length downto 13. do Max-Heapify(A,i)
∑i O(1 + height of i)
= ∑0 ≤ h ≤ log n (# nodes of height h) ∙ O(1+h)
≤ ∑0 ≤ h ≤ log n (n / 2h+1) ∙ O(1+h)
= O(n) ∙ ∑0 ≤ h ≤ log n h / 2h+1
= O(n)
h=0
h=1
h=2
O(height of node i)
height of node i # edges longest simple downward path from i to a leaf.
![Page 32: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/32.jpg)
The sorting problem
Input: a sequence of n numbers ‹a1, a2, …, an›
Output: a permutation of the input such that ‹ai1 ≤ … ≤ ain›
Important properties of sorting algorithms:
running time: how fast is the algorithm in the worst case
in place: only a constant number of input elements are ever stored outside the input array
![Page 33: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/33.jpg)
Sorting with a heap: Heapsort
HeapSort(A)
1. Build-Max-Heap(A)2. for i = A.length downto 23. do exchange A[1] and A[i ]4. heap-size[A] = heap-size[A] – 15. Max-Heapify(A,1)
P(k): A[k+1 .. n] is sorted and contains the n-k largest elements, A[1..k] is a max-heap on the remaining elements
Loop InvariantP(k) holds before lines 3-5 are executed with i=k, P(k-1) holds afterwards
![Page 34: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/34.jpg)
Sorting algorithms
worst case running time in place
InsertionSort
MergeSort
HeapSort
Θ(n2) yes
Θ(n log n) noyesΘ(n log n)
![Page 35: 2IL50 Data Structures Spring 2015 Lecture 3: Heaps](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e705503460f94b6e76d/html5/thumbnails/35.jpg)
Announcements
Wednesday 3+4
Reading and writing mathematical proofs