Analysis of Algorithms Analysis of Algorithms

Heaps

2

In today’s lectureIn today’s lecture

HeapsHeaps Heap AttributesHeap Attributes Heap FunctionsHeap Functions Heap PropertyHeap Property Algorithm Max-Heapify Algorithm Max-Heapify Complexity of Max-HeapifyComplexity of Max-Heapify Algorithm Build-Max-HeapAlgorithm Build-Max-Heap ComplexityComplexity Algorithm Heap-SortAlgorithm Heap-Sort ComplexityComplexity

3

HeapsHeaps A heap is a data structure defined only on A heap is a data structure defined only on “nearly “nearly

full”full” or complete binary tress. or complete binary tress. A A complete binary treecomplete binary tree is one where all the is one where all the

internal nodesinternal nodes have have exactlyexactly 2 2 children, and all the children, and all the leaves are on the same level. An internal node leaves are on the same level. An internal node comprises all the nodes except the leaf nodes.comprises all the nodes except the leaf nodes.

Leaf nodesLeaf nodes are nodes without children. are nodes without children.

Leaf node

Internal nodeEdge

4

Heap AttributesHeap Attributes Height of a nodeHeight of a node

The number of edges starting at that node, and going down to The number of edges starting at that node, and going down to the furthest leaf.the furthest leaf.

Height of the heapHeight of the heap The maximum number of edges from the root to a leaf.The maximum number of edges from the root to a leaf.

Root

Height of highlighted node = 1

Height of root = 3

5

Heap FunctionsHeap Functions CompleteComplete binary tree– if not full, then the binary tree– if not full, then the onlyonly

unfilled level is unfilled level is filled in from left to rightfilled in from left to right..

2 3

4 5 6 7

8 9 10 11 12 13

1

Parent(i)

return i/2Left(i)

return 2i

Right(i)

return 2i + 1

11 22 33 44 55 66 77 88 99 1100

1111

1122

1133

Physical Implementation of a heap

can be a simple array.

6

Heap PropertyHeap Property The heap property of a tree is a condition The heap property of a tree is a condition

that must be true for the tree to be that must be true for the tree to be considered a heap.considered a heap.

Min-heap propertyMin-heap property: : for min-heaps, requiresfor min-heaps, requiresA[parent(i)] A[parent(i)] A[i] A[i]So, the root of any sub-tree holds the So, the root of any sub-tree holds the leastleast value in value in

that sub-tree.that sub-tree. Max-heap propertyMax-heap property::

for max-heaps, requiresfor max-heaps, requiresA[parent(i)] A[parent(i)] A[i] A[i]The root of any sub-tree holds the The root of any sub-tree holds the greatestgreatest value value

in the sub-tree.in the sub-tree.

7

HeapsHeaps Looks like a max heap, but max-heap property is violated at indices 11 and Looks like a max heap, but max-heap property is violated at indices 11 and

12. Why?12. Why? Because those nodes’ parents have smaller values than their children.Because those nodes’ parents have smaller values than their children.

2

83

84

85

76

87

18

69

710

511

812

9

1

9

121211111010998877665544332211

998855776611887788888899

8

Algorithm: Max-Heapify(A,i)Algorithm: Max-Heapify(A,i) Assume we are given a tree Assume we are given a tree

represented by an represented by an array Aarray A.. ii is an is an index into the arrayindex into the array such that such that

i i length[A] length[A], and the trees rooted , and the trees rooted at Left(i) and Right(i) are heaps.at Left(i) and Right(i) are heaps.

Then, to Then, to create a heapcreate a heap rooted at A[i], rooted at A[i], use use procedure Max-Heapify(A,i)procedure Max-Heapify(A,i)

Let us first see the Let us first see the idea behind Max-idea behind Max-Heapify().Heapify().

9

Max-Heapify(A,1)Max-Heapify(A,1)

20 9

19 18 6 7

17 16 17 16 1

10

10 9

19 18 6 7

17 16 17 16 1

20

10 9

19 18 6 7

17 16 17 16 1

20

19 9

10 18 6 7

17 16 17 16 1

20

19 9

10 18 6 7

17 16 17 16 1

20

19 9

17 18 6 7

10 16 17 16 1

20

10

Algorithm: Max-Heapify(A,i)Algorithm: Max-Heapify(A,i)

Max-Heapify(A,i)Max-Heapify(A,i)1.1. l l Left(i) Left(i)2.2. r r Right(i) Right(i)3.3. if A[l] > A[i] and l if A[l] > A[i] and l heap-size[A] heap-size[A] 4.4. largest largest l l5.5. else largest else largest i i6.6. if A[r] > A[largest] and r if A[r] > A[largest] and r heap-size[A] heap-size[A] 7.7. largest largest r r8.8. if largest if largest I I

swap( A[i], A[largest])swap( A[i], A[largest])Max-Heapify(A, largest)Max-Heapify(A, largest)

10.10. else else returnreturn..

11

If If either child of A[i] is greatereither child of A[i] is greater than A[i], the than A[i], the greatest greatest child is exchangedchild is exchanged with with A[i]A[i]. So, A[i] . So, A[i] moves downmoves down in the heap. in the heap.

The The move of A[i]move of A[i] may have caused a may have caused a violation violation of the max-heap propertyof the max-heap property at it’s new location. at it’s new location.

So, we must So, we must recursively call Max-Heapify(A,i)recursively call Max-Heapify(A,i) at the location i where the node “lands”.at the location i where the node “lands”.

The above is the top-down approach of Max-The above is the top-down approach of Max-Heapify(A,i)Heapify(A,i)

Obviously, Max-Heapify(A,i) can’t be bottom-Obviously, Max-Heapify(A,i) can’t be bottom-up. Why?up. Why?

Algorithm: Max-Heapify(A,i)Algorithm: Max-Heapify(A,i)

12

Complexity of Max-Heapify()Complexity of Max-Heapify() Consider Consider worst caseworst case:: Comparison will be performed at each Comparison will be performed at each

level.level. How many levels? Alternatively, what How many levels? Alternatively, what

is the height of the heap?is the height of the heap? (lg n). How? (lg n). How? Hence, complexity of Max-Heapify() Hence, complexity of Max-Heapify()

for a tree of n nodes is O(lg n).for a tree of n nodes is O(lg n).

13

HeapsHeaps How to build a heap from scratch? How to build a heap from scratch?

We call Max-Heapify(A,i) for every i We call Max-Heapify(A,i) for every i starting at last node and going to the starting at last node and going to the root.root.

Do we need to start at the bottom of the Do we need to start at the bottom of the heap?heap?

Bad-Build-Max-Heap(A)Bad-Build-Max-Heap(A)1.1. heap-sizeheap-size[A] [A] lengthlength[A][A]2.2. for for i i lengthlength[A] downto 1[A] downto 13.3. Max-Heapify(A,i)Max-Heapify(A,i)

14

How Bad-Build-Max-Heap(A) How Bad-Build-Max-Heap(A) works ?works ?

1010998877665544332211

77881414101099161622331144

2 3

4 5 6 7

8 9 10

1

1 32 16 6 10

14 8 7

42 3

4 5 6 7

8 9 10

1

1 32 16 6 10

14 8 7

4

2 3

4 5 6 7

8 9 10

1

1 3

2 16 6 10

14 8 7

4 2 3

4 5 6 7

8 9 10

1

1 3

2 16 6 10

14 8 7

4 2 3

4 5 6 7

8 9 10

1

1 3

2 16 6 10

14 8 7

4

15

Improved Build-Max-HeapImproved Build-Max-Heap An important observation before we An important observation before we

proceed furtherproceed further There is There is no needno need to call Max-Heapify() on to call Max-Heapify() on leaf leaf

nodes.nodes. This call always return without any change to This call always return without any change to

orignal Heap. orignal Heap. So, what will be better array index to start So, what will be better array index to start

with?with? Start at internal node having second-last Start at internal node having second-last

and last indices of array as children and last indices of array as children indices. indices.

2 3

4 5 6 7

8 9 10

1

1 3

2 16 6 10

14 8 7

4Start Here

16

The internal node with the largest index is at The internal node with the largest index is at most equal to length[A]/2. most equal to length[A]/2.

Build-Max-Heap(A)Build-Max-Heap(A)1.1. heap-sizeheap-size[A] [A] lengthlength[A][A]2.2. for for i i lengthlength[A]/2[A]/2 downto 1 downto 13.3. Max-Heapify(A,i)Max-Heapify(A,i)

Improved Build-Max-HeapImproved Build-Max-Heap

2 3

4 5 6 7

8 9 10

1

1 3

2 16 6 10

14 8 7

4

17

Build-Max-Heap() Running TimeBuild-Max-Heap() Running Time By quick examination, By quick examination, Build-Max-Heap() is Build-Max-Heap() is

O(n lgn).O(n lgn). Max-Heapify is (lg n), while it is being Max-Heapify is (lg n), while it is being

performed for n nodes.performed for n nodes.

18

Number of Nodes at Height hNumber of Nodes at Height h Binary Tree Property:Binary Tree Property:

Number of nodes at any non-root level is Number of nodes at any non-root level is halfhalf of the of the total total number of nodesnumber of nodes up to that level. up to that level.

The The number of leavesnumber of leaves in a binary heap is equal to in a binary heap is equal to nn/2/2 where where nn is the total number of nodes in the tree is the total number of nodes in the tree

Ceiling(Ceiling(nn/2) when /2) when nn is odd is odd.. If these leaves are removed, the number of new If these leaves are removed, the number of new

leaves will be Ceiling(leaves will be Ceiling(nn/2/2) or /2/2) or Ceiling(n/2Ceiling(n/222).). If  this process is continued for If  this process is continued for h levelsh levels the number the number

of leaves at that level will be of leaves at that level will be ceiling(ceiling(nn/2/2hh+1+1).).

19

HeapsortHeapsort How can we get the keys sorted in an array A?How can we get the keys sorted in an array A? A heap may be fulfilling Max/Min-Heap property is not A heap may be fulfilling Max/Min-Heap property is not

necessarily sorted. necessarily sorted. We will use a simple algorithm to sort the contents of a heap. We will use a simple algorithm to sort the contents of a heap. Heapsort(A)Heapsort(A)1. Build-Max-Heap(A)1. Build-Max-Heap(A)2. for i = 2. for i = lengthlength[A] [A] downtodownto 2 23.3. exchange A[1] exchange A[1] A[i] A[i]4.4. heap-sizeheap-size[A] = [A] = heap-sizeheap-size[A]-1[A]-15.5. Max-Heapify(A,1)Max-Heapify(A,1) Build-Max-Heap() is O(nlgn) Build-Max-Heap() is O(nlgn) Max-Heapify() is O(lg n), but is executed n times, so runtime Max-Heapify() is O(lg n), but is executed n times, so runtime

for Heapsort(A) is O(nlgn).for Heapsort(A) is O(nlgn).

20

HeapsortHeapsortHeapsort(A)Heapsort(A)1. Build-Max-Heap(A)1. Build-Max-Heap(A)2. for i 2. for i lengthlength[A] [A] downtodownto 2 23.3. dodo exchange A[1] exchange A[1] A[i] A[i]4.4. heap-sizeheap-size[A] [A] heap-sizeheap-size[A]-1[A]-15.5. Max-Heapify(A,1)Max-Heapify(A,1)

On line 1, we create the heap. Keep in mind that the largest On line 1, we create the heap. Keep in mind that the largest element is at the root.element is at the root.

Why not exchange it with the element that is currently in the Why not exchange it with the element that is currently in the root at the last index.root at the last index.

Must call Max-Heapify() afterwards as the move may violate Must call Max-Heapify() afterwards as the move may violate the Heap Property. the Heap Property.

Now we can ignore the last element in A for the rest of the Now we can ignore the last element in A for the rest of the working and repeat the procedure for second-last element.working and repeat the procedure for second-last element.

Have to decrement Heap-Size[A] in order to ignore the Have to decrement Heap-Size[A] in order to ignore the element currently at the last position. element currently at the last position.

21

HeapSort() AnimationHeapSort() Animation

See See http://www.cs.brockport.edu/cs/java/apps/http://www.cs.brockport.edu/cs/java/apps/sorters/heapsort.htmlsorters/heapsort.html

For a wonderful HeapSort() animation.For a wonderful HeapSort() animation.