lecture 8 and 9: trees and heap sort - coms10007 -...
TRANSCRIPT
Lecture 8 and 9: Trees and Heap SortCOMS10007 - Algorithms
Dr. Christian Konrad
19.02.2019 and 25.02.2019
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 1 / 23
Sorting Algorithms seen so far
Sorting Algorithms seen so far
Insertion-Sort: O(n2) in worst, in place, stable
Merge-Sort: O(n log n) in worst case, NOT in place, stable
Heap Sort (best of the two)
O(n log n) in worst case, in place, NOT stable
Uses a heap data structure (a heap is special tree)
Data Structures
Data storage format that allows for efficient access andmodification
Building block of many efficient algorithms
For example, an array is a data structure
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 2 / 23
Trees
Definition: A tree T = (V ,E ) of size n is a tuple consisting of
V = {v1, v2, . . . , vn} and E = {e1, e2, . . . , en−1}
with |V | = n and |E | = n − 1 with ei = {vj , vk} for some j 6= ksuch that for every node vi there is at least one edge ej such thatvi ∈ ej . V are the nodes/vertices and E are the edges of T .
X X 7
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 3 / 23
Rooted Trees
Definition: (rooted tree) A rooted tree is a triple T = (v ,V ,E )such that T = (V ,E ) is a tree and v ∈ V is a designed nodethat we call the root of T .
Definition: (leaf, internal node) A leaf in a tree is a node withexactly one incident edge. A node that is not a leaf is called aninternal node.
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 4 / 23
Children, Parent, and Degree
Further Definitions:
The parent of a node v is the closestnode on a path from v to the root.The root does not have a parent.
The children of a node v are v ’sneighbors except its parent.
The height of a tree is the length of a longest root-to-leafpath.
The degree deg(v) of a node v is the number of incident edgesto v . Since every edge is incident to two vertices we have∑
v∈Vdeg(v) = 2 · |E | = 2(n − 1) .
The level of a vertex v is the length of the unique path fromthe root to v plus 1.
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 5 / 23
Properties of Trees
Property: Every tree has at least 2 leaves
Proof Let L ⊆ V be the subset of leaves. Suppose that there is atmost 1 leaf, i.e., |L| ≤ 1. Then:∑v∈V
deg(v) =∑v∈L
deg(v) +∑
v∈V \L
deg(v)
≥ |L| · 1 + (|V | − |L|) · 2 = 2|V | − |L| ≥ 2n − 1 ,
a contradiction to the fact that∑
v∈V deg(v) = 2(n − 1) in everytree.
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 6 / 23
Binary Trees
Definition: (k-ary tree) A (rooted) tree is k-ary if every nodehas at most k children. If k = 2 then the tree is called binary.A k ary tree is
full if every internal node has exactly k children,
complete if all levels except possibily the last is entirelyfilled (and last level is filled from left to right),
perfect if all levels are entirely filled.
complete 3-ary tree full 3-ary tree perfect binary tree
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 7 / 23
Height of Perfect and Complete k-ary Trees
Height of k-ary Trees
The number of nodes in a perfect k-ary tree of height i − 1 is
i−1∑j=0
k j =k i − 1
k − 1.
In other words, a perfect k-ary tree on n nodes has height:
logk(n(k − 1) + 1) = O(logk n) .
Similarly, a complete k-ary tree has height O(logk n).
Remark: The runtime of many algorithms that use tree datastructures depends on the height of these trees. We are thereforeinterested in using complete/perfect trees.
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 8 / 23
Priority Queues
Priority Queue:Data structure that allows the following operations:
Build(.): Create data structure given a set of data items
Extract-Max(.): Remove the maximum element from the datastructure
others...
Sorting using a Priority Queue
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 9 / 23
From Array to Tree
Interpretation of an Array as a Complete Binary Tree
Easy Navigation:
Parent of i : bi/2cLeft/Right Child of i : 2i and 2i + 1
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 10 / 23
Heap Property
The Heap Property
Key of nodes larger than keys of their children
Heap Property → Maximum at rootImportant for Extract-Max(.)
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 11 / 23
Heap Property
The Heap Property
Key of nodes larger than keys of their children
Heap Property → Maximum at rootImportant for Extract-Max(.)
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 11 / 23
Heap Property
The Heap Property
Key of nodes larger than keys of their children
Heap Property → Maximum at rootImportant for Extract-Max(.)
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 11 / 23
Heap Property
The Heap Property
Key of nodes larger than keys of their children
Heap Property → Maximum at rootImportant for Extract-Max(.)
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 11 / 23
The Heapify Operation
Constructing a Heap: Build(.)Given a binary tree, transform it into one that fulfills the HeapProperty
1 Traverse tree with regards to right-to-left array ordering
2 If node does not fulfill Heap Property: Heapify()
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 12 / 23
The Heapify Operation
Constructing a Heap: Build(.)Given a binary tree, transform it into one that fulfills the HeapProperty
1 Traverse tree with regards to right-to-left array ordering
2 If node does not fulfill Heap Property: Heapify()
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 12 / 23
The Heapify Operation
Constructing a Heap: Build(.)Given a binary tree, transform it into one that fulfills the HeapProperty
1 Traverse tree with regards to right-to-left array ordering
2 If node does not fulfill Heap Property: Heapify()
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 12 / 23
The Heapify Operation
Constructing a Heap: Build(.)Given a binary tree, transform it into one that fulfills the HeapProperty
1 Traverse tree with regards to right-to-left array ordering
2 If node does not fulfill Heap Property: Heapify()
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 12 / 23
The Heapify Operation
Constructing a Heap: Build(.)Given a binary tree, transform it into one that fulfills the HeapProperty
1 Traverse tree with regards to right-to-left array ordering
2 If node does not fulfill Heap Property: Heapify()
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 12 / 23
The Heapify Operation
Constructing a Heap: Build(.)Given a binary tree, transform it into one that fulfills the HeapProperty
1 Traverse tree with regards to right-to-left array ordering
2 If node does not fulfill Heap Property: Heapify()
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 12 / 23
The Heapify Operation
Constructing a Heap: Build(.)Given a binary tree, transform it into one that fulfills the HeapProperty
1 Traverse tree with regards to right-to-left array ordering
2 If node does not fulfill Heap Property: Heapify()
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 12 / 23
The Heapify Operation
Constructing a Heap: Build(.)Given a binary tree, transform it into one that fulfills the HeapProperty
1 Traverse tree with regards to right-to-left array ordering
2 If node does not fulfill Heap Property: Heapify()
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 12 / 23
The Heapify Operation
Constructing a Heap: Build(.)Given a binary tree, transform it into one that fulfills the HeapProperty
1 Traverse tree with regards to right-to-left array ordering
2 If node does not fulfill Heap Property: Heapify()
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 12 / 23
The Heapify Operation
Constructing a Heap: Build(.)Given a binary tree, transform it into one that fulfills the HeapProperty
1 Traverse tree with regards to right-to-left array ordering
2 If node does not fulfill Heap Property: Heapify()
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 12 / 23
The Heapify Operation
Constructing a Heap: Build(.)Given a binary tree, transform it into one that fulfills the HeapProperty
1 Traverse tree with regards to right-to-left array ordering
2 If node does not fulfill Heap Property: Heapify()
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 12 / 23
The Heapify Operation
Constructing a Heap: Build(.)Given a binary tree, transform it into one that fulfills the HeapProperty
1 Traverse tree with regards to right-to-left array ordering
2 If node does not fulfill Heap Property: Heapify()
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 12 / 23
The Heapify Operation
Constructing a Heap: Build(.)Given a binary tree, transform it into one that fulfills the HeapProperty
1 Traverse tree with regards to right-to-left array ordering
2 If node does not fulfill Heap Property: Heapify()
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 12 / 23
The Heapify Operation
Constructing a Heap: Build(.)Given a binary tree, transform it into one that fulfills the HeapProperty
1 Traverse tree with regards to right-to-left array ordering
2 If node does not fulfill Heap Property: Heapify()
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 12 / 23
The Heapify Operation
Constructing a Heap: Build(.)Given a binary tree, transform it into one that fulfills the HeapProperty
1 Traverse tree with regards to right-to-left array ordering
2 If node does not fulfill Heap Property: Heapify()
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 12 / 23
Runtime of Heapify()
Heapify()Let p be the key of a node and let c1, c2 be the keys of its children
Let c = max{c1, c2}If c > p then exchange nodes with keys p and c
call Heapify() at node with key c
Runtime:
Exchanging nodes requires time O(1)
The number of recursive calls is bounded by the height of thetree, i.e., O(log n)
Runtime of Heapify: O(log n).
Constructing a Heap: Build(.) Runtime O(n log n)
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 13 / 23
Improved Analysis of Heap Construction
More Precise Analysis of the Heap Construction Step
Heapify(x): O(depth of subtree rooted at x) = O(log n)
Observe: Most nodes close to the “bottom”
Analysis:
Let i be the largest integersuch that n′ := 2i − 1 and n′ < n
There are at most n′ internalnodes (candidates for Heapify())
These nodes are contained in aperfect binary tree
This tree has height i − 1
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 14 / 23
Improved Analysis of Heap Construction
More Precise Analysis of the Heap Construction Step
Heapify(x): O(depth of subtree rooted at x) = O(log n)
Observe: Most nodes close to the “bottom”
Analysis:
Let i be the largest integersuch that n′ := 2i − 1 and n′ < n
There are at most n′ internalnodes (candidates for Heapify())
These nodes are contained in aperfect binary tree
This tree has height i − 1
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 14 / 23
Improved Analysis of Heap Construction
More Precise Analysis of the Heap Construction Step
Heapify(x): O(depth of subtree rooted at x) = O(log n)
Observe: Most nodes close to the “bottom”
Analysis:
Let i be the largest integersuch that n′ := 2i − 1 and n′ < n
There are at most n′ internalnodes (candidates for Heapify())
These nodes are contained in aperfect binary tree
This tree has height i − 1
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 14 / 23
Improved Analysis of Heap Construction
More Precise Analysis of the Heap Construction Step
Heapify(x): O(depth of subtree rooted at x) = O(log n)
Observe: Most nodes close to the “bottom”
Analysis:
Let i be the largest integersuch that n′ := 2i − 1 and n′ < n
There are at most n′ internalnodes (candidates for Heapify())
These nodes are contained in aperfect binary tree
This tree has height i − 1
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 14 / 23
Improved Analysis of Heap Construction
More Precise Analysis of the Heap Construction Step
Heapify(x): O(depth of subtree rooted at x) = O(log n)
Observe: Most nodes close to the “bottom”
Analysis:
Let i be the largest integersuch that n′ := 2i − 1 and n′ < n
There are at most n′ internalnodes (candidates for Heapify())
These nodes are contained in aperfect binary tree
This tree has height i − 1
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 14 / 23
Improved Analysis of Heap Construction
More Precise Analysis of the Heap Construction Step
Heapify(x): O(depth of subtree rooted at x) = O(log n)
Observe: Most nodes close to the “bottom”
Analysis:
Let i be the largest integersuch that n′ := 2i − 1 and n′ < n
There are at most n′ internalnodes (candidates for Heapify())
These nodes are contained in aperfect binary tree
This tree has height i − 1
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 14 / 23
Improved Analysis of Heap Construction
More Precise Analysis of the Heap Construction Step
Heapify(x): O(depth of subtree rooted at x) = O(log n)
Observe: Most nodes close to the “bottom”
Analysis:
Let i be the largest integersuch that n′ := 2i − 1 and n′ < n
There are at most n′ internalnodes (candidates for Heapify())
These nodes are contained in aperfect binary tree
This tree has height i − 1
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 14 / 23
Improved Analysis of Heap Construction
AnalysisWe sum over all relevant levels, count thenumber of nodes per level, and multiplywith the depth of their subtrees:
i∑j=1
2i−j︸︷︷︸nodes in level i − j
· j︸︷︷︸depth of subtree
i∑j=1
2i−j · j = 2i ·i∑
j=1
j
2j= O(2i ) = O(n′) = O(n) .
We’ll prove∑i
j=1j2j
= O(1) very soon...!
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 15 / 23
The Complete Algorithm
Putting Everything Together
1 Build-heap()2 Repeat n times:
1 Swap root with last element2 Decrease size of heap by 13 Heapify(root)
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 16 / 23
The Complete Algorithm
Putting Everything Together
1 Build-heap()2 Repeat n times:
1 Swap root with last element2 Decrease size of heap by 13 Heapify(root)
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 16 / 23
The Complete Algorithm
Putting Everything Together
1 Build-heap()2 Repeat n times:
1 Swap root with last element2 Decrease size of heap by 13 Heapify(root)
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 16 / 23
The Complete Algorithm
Putting Everything Together
1 Build-heap()2 Repeat n times:
1 Swap root with last element2 Decrease size of heap by 13 Heapify(root)
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 16 / 23
The Complete Algorithm
Putting Everything Together
1 Build-heap()2 Repeat n times:
1 Swap root with last element2 Decrease size of heap by 13 Heapify(root)
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 16 / 23
The Complete Algorithm
Putting Everything Together
1 Build-heap()2 Repeat n times:
1 Swap root with last element2 Decrease size of heap by 13 Heapify(root)
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 16 / 23
The Complete Algorithm
Putting Everything Together
1 Build-heap()2 Repeat n times:
1 Swap root with last element2 Decrease size of heap by 13 Heapify(root)
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 16 / 23
The Complete Algorithm
Putting Everything Together
1 Build-heap()2 Repeat n times:
1 Swap root with last element2 Decrease size of heap by 13 Heapify(root)
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 16 / 23
The Complete Algorithm
Putting Everything Together
1 Build-heap()2 Repeat n times:
1 Swap root with last element2 Decrease size of heap by 13 Heapify(root)
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 16 / 23
The Complete Algorithm
Putting Everything Together
1 Build-heap()2 Repeat n times:
1 Swap root with last element2 Decrease size of heap by 13 Heapify(root)
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 16 / 23
The Complete Algorithm
Putting Everything Together
1 Build-heap()2 Repeat n times:
1 Swap root with last element2 Decrease size of heap by 13 Heapify(root)
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 16 / 23
The Complete Algorithm
Putting Everything Together
1 Build-heap()2 Repeat n times:
1 Swap root with last element2 Decrease size of heap by 13 Heapify(root)
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 16 / 23
The Complete Algorithm
Putting Everything Together
1 Build-heap() O(n)2 Repeat n times:
1 Swap root with last element O(1)2 Decrease size of heap by 1 O(1)3 Heapify(root) O(log n)
Runtime: O(n log n)
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 16 / 23
Heapsort is Not Stable
Example:
1 Build-heap()2 Repeat n times:
1 Swap root with last element2 Decrease size of heap by 13 Heapify(root)
1 is moved from left to the right past 1 and 1
Heap-sort not stable
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 17 / 23
Heapsort is Not Stable
Example:
1 Build-heap()2 Repeat n times:
1 Swap root with last element2 Decrease size of heap by 13 Heapify(root)
1 is moved from left to the right past 1 and 1
Heap-sort not stable
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 17 / 23
Heapsort is Not Stable
Example:
1 Build-heap()2 Repeat n times:
1 Swap root with last element2 Decrease size of heap by 13 Heapify(root)
1 is moved from left to the right past 1 and 1
Heap-sort not stable
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 17 / 23
Heapsort is Not Stable
Example:
1 Build-heap()2 Repeat n times:
1 Swap root with last element2 Decrease size of heap by 13 Heapify(root)
1 is moved from left to the right past 1 and 1
Heap-sort not stable
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 17 / 23
(Trick for Bounding Sums)
How to bound∑n−1
i=0i2i
:
Sn−1 :=n−1∑i=0
i
2i.
Trick: Consider 12Sn−1
Sn−1 =1
2+
2
4+
3
8+
4
16+ · · ·+ n − 1
2n−1
1
2Sn−1 =
1
4+
2
8+
3
16+ · · ·+ n − 1
2n
Sn−1 −1
2Sn−1 =
1
2+
1
4+
1
8+ · · ·+ 1
2n−1+
n − 1
2n
=n−1∑i=0
1
2i+
n − 1
2n=
12n −
12
12 − 1
+n − 1
2n= O(1) .
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 18 / 23
Where we are
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 19 / 23
Where we are
Lecture Material1 Peak finding2 O-notation3 Theta, Omega, RAM Model4 Linear/binary search, Induction5 Loop invariants, insertion-sort6 Merge sort 1 (divide-and-conquer)7 Merge sort 2, maximum subarray problem8 Trees, Heap-sort (1)9 Heap-sort (2), Exercises10- Quick-sort, sorting LB, radix-sort
Recurrences, Divide-and-conquer, dynamic programmingBasic data structures
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 20 / 23
FromPiazza/Drop-in/Office
Hours...
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 21 / 23
Are all Functions Asymptotically Comparable?
Let f , g be positive functions. Is the following statement true?
Claim. f (n) /∈ O(g(n))⇒ g(n) ∈ O(f (n)) . false!
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 22 / 23
Are all Functions Asymptotically Comparable? (2)
f (n) = n and g(n) = n1+0.1 sin(n)
Not all Functions are asymptotically comparable!
Observe that n1+0.1 sin(n) is infinitely often equal to n1.1 andinfinintely often equal to n0.9
Therefore, neither f (n) ∈ O(g(n)) nor g(n) ∈ O(f (n))
Another Example:
f (n) = n
g(n) = n2 if n even and g(n) =√
n if n odd
Dr. Christian Konrad Lecture 8 and 9: Trees and Heap Sort 23 / 23