2il50 data structures spring 2015 lecture 1: introduction
TRANSCRIPT
![Page 1: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/1.jpg)
2IL50 Data Structures
Spring 2015
Lecture 1: Introduction
![Page 2: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/2.jpg)
Algorithms
Algorithma well-defined computational procedure that takes some value, or a set of values, as input and produces some value, or a set of values, as output.
Algorithmsequence of computational steps that transform the input into the output.
Algorithms researchdesign and analysis of algorithms and data structures for computational problems.
![Page 3: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/3.jpg)
Data structures
Data Structurea way to store and organize data to facilitate access and modifications.
Abstract data typedescribes functionality (which operations are supported).
Implementationa way to realize the desired functionality how is the data stored (array, linked list, …) which algorithms implement the operations
![Page 4: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/4.jpg)
The course
Design and analysis of efficient algorithms for some basic computational problems.
Basic algorithm design techniques and paradigms
Algorithms analysis: O-notation, recursions, …
Basic data structures
Basic graph algorithms
![Page 5: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/5.jpg)
Some administration first
before we really get started …
![Page 6: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/6.jpg)
Organization
Lecturer: Prof. Dr. Bettina Speckmann, MF 4.105,[email protected]
I’m here every day but Friday …
Web page:http://www.win.tue.nl/~speckman/2IL50.html
Book: T.H. Cormen, C.E. Leiserson, R.L. Rivest and C. Stein.
Introduction to Algorithms (3rd edition)
mandatory
![Page 7: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/7.jpg)
Prerequisites
Being able to work with basic programming constructs such as linked lists, arrays, loops …
Being able to apply standard proving techniques such as proof by induction, proof by contradiction ...
Being familiar with sums and logarithms such as discussed in Chapter 3 and Appendix A of the textbook.
If you think you might lack any of this knowledge, please come and talk to me immediately!
![Page 8: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/8.jpg)
Grading scheme 2IL50
1. 6 homework assignments, the best 5 of which each count for 10% of the final grade.
2. A written exam (closed book) which counts for the remaining 50% of the final grade.
If you reach less than 50% of the possible points on the homework assignments, then you are not allowed to participate in the final exam nor in the second chance exam. You will fail the course and your next chance will be next year. Your grade will be the minimum of 5 and the grade you achieved. If you reach less than 50% of the points on the final exam, then you will fail the course, regardless of the points you collected with the homework assignments. However, you are allowed to participate in the second chance exam. The grade of the second chance exam replaces the grade for the first exam, that is, your homework assignments always count for 50% of your grade.
Do the homework assignments!
![Page 9: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/9.jpg)
Homework Assignments
Posted on web-page on Monday before lecture.
Due Sundays at 23:59 as .pdf in the electronic mailbox of your instructor.
Late assignments will not be accepted. Only 5 out of 6 assignments count, hence there are no exceptions.
Must be typeset – use Latex! See example file on web-page.
Name scheme: Ai-LastName.pdf. If your name is Anton van Gelderland and you submit the 1st assignment, then your file must be named A1-vanGelderland.pdf.
Use the tag [2IL50] in the subject line of your email.
Any questions: Stop by my office whenever you want (except Fridays!)
(send email if you want to make sure that I have time)
![Page 10: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/10.jpg)
Academic Dishonesty
Academic Dishonesty
All class work has to be done independently. You are of course allowed to discuss the material presented in class, homework assignments, or general solution strategies with me or your classmates, but you have to formulate and write up your solutions by yourself. You must not copy from the internet, your friends, or other textbooks. Problem solving is an important component of this course so it is really in your best interest to try and solve all problems by yourself. If you represent other people's work as your own then that constitutes fraud and will be dealt with accordingly.
![Page 11: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/11.jpg)
Organization
Components:
1. Lectures Monday 5+6 AUD 6 Wednesday 3+4 AUD 3
2. Tutorials Wednesday 1+2 see web-page for roomsand instructors!
The instructors will explain the solutions to the homework assign-ments of the previous week and answer any questions that arise.
3. Office hours Thursday 17:00-18:00 around MF 4.104b
4. Reading and writing mathematical proofs
![Page 12: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/12.jpg)
How to read and write mathematical proofs
Lecturer: Dr. Kevin Verbeek, MF 4.106,[email protected]
Web page:Part of Data Structures:
http://www.win.tue.nl/~speckman/2IL50.html
Book: Daniel SolowHow to Read and Do Proofs (5th edition)
not mandatory
![Page 13: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/13.jpg)
![Page 14: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/14.jpg)
Signing up for groups
Group registration is open from today 02-02-2015 at 18:00 until tomorrow 03-02-2015 at 18:00.
![Page 15: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/15.jpg)
Some statistics …
![Page 16: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/16.jpg)
Sorting
let’s get started …
![Page 17: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/17.jpg)
The sorting problem
Input: a sequence of n numbers ‹a1, a2, …, an›
Output: a permutation of the input such that ‹ai1 ≤ … ≤ ain›
The input is typically stored in arrays
Numbers ≈ Keys
Additional information (satellite data) may be stored with keys
We will study several solutions ≈ algorithms for this problem
![Page 18: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/18.jpg)
Describing algorithms
A complete description of an algorithm consists of three parts:
1. the algorithm
(expressed in whatever way is clearest and most concise, can be English and / or pseudocode)
2. a proof of the algorithm’s correctness
3. a derivation of the algorithm’s running time
![Page 19: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/19.jpg)
InsertionSort
Like sorting a hand of playing cards: start with empty left hand, cards on table remove cards one by one, insert into
correct position to find position, compare to cards in hand from right to left cards in hand are always sorted
InsertionSort is a good algorithm to sort a small number of elements an incremental algorithm
Incremental algorithmsprocess the input elements one-by-one and maintain the solution for the elements processed so far.
![Page 20: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/20.jpg)
Incremental algorithms
Incremental algorithmsprocess the input elements one-by-one and maintain the solution for the elements processed so far.
In pseudocode:
IncAlg(A)
// incremental algorithm which computes the solution of a problem with input A = {x1,…,xn}
1. initialize: compute the solution for {x1}
2. for j = 2 to n
3. do compute the solution for {x1,…,xj} using the (already computed) solution for {x1,…,xj-1}
no “begin - end”, just indentation
Check book for more pseudocode conventions
![Page 21: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/21.jpg)
InsertionSort
InsertionSort(A) // incremental algorithm that sorts array A[1..n] in non-decreasing
order1. initialize: sort A[1]2. for j = 2 to A.length3. do sort A[1..j] using the fact that A[1.. j-1] is already sorted
![Page 22: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/22.jpg)
InsertionSort
InsertionSort(A) // incremental algorithm that sorts array A[1..n] in non-decreasing
order1. initialize: sort A[1]2. for j = 2 to A.length3. do key = A[j] 4. i = j -15. while i > 0 and A[i] > key6. do A[i+1] = A[i] 7. i = i -18. A[i +1] = key
1 3 14 17 28 6 …
1 j n
InsertionSort is an in place algorithm: the numbers are rearranged within the array with only constant extra space.
![Page 23: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/23.jpg)
Correctness proof
Use a loop invariant to understand why an algorithm gives the correct answer.
Loop invariant (for InsertionSort)At the start of each iteration of the “outer” for loop (indexed by j) the subarray A[1..j-1] consists of the elements originally in A[1..j-1] but in sorted order.
![Page 24: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/24.jpg)
Correctness proof
To proof correctness with a loop invariant we need to show three things:
InitializationInvariant is true prior to the first iteration of the loop.
MaintenanceIf the invariant is true before an iteration of the loop, it remains true before the next iteration.
TerminationWhen the loop terminates, the invariant (usually along with the reason that the loop terminated) gives us a useful property that helps show that the algorithm is correct.
![Page 25: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/25.jpg)
Correctness proof
InsertionSort(A)1. initialize: sort A[1]2. for j = 2 to A.length3. do key = A[j] 4. i = j -15. while i > 0 and A[i] > key6. do A[i+1] = A[i] 7. i = i -18. A[i +1] = key
InitializationJust before the first iteration, j = 2 ➨ A[1..j-1] = A[1], which is the element originally in A[1], and it is trivially sorted.
Loop invariant At the start of each iteration of the “outer” for loop (indexed by j) the subarray A[1..j-1] consists of the elements originally in A[1..j-1] but in sorted order.
![Page 26: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/26.jpg)
Correctness proof
InsertionSort(A)1. initialize: sort A[1]2. for j = 2 to A.length3. do key = A[j] 4. i = j -15. while i > 0 and A[i] > key6. do A[i+1] = A[i] 7. i = i -18. A[i +1] = key
MaintenanceStrictly speaking need to prove loop invariant for “inner” while loop. Instead, note that body of while loop moves A[j-1], A[j-2], A[j-3], and so on, by one position to the right until proper position of key is found (which has value of A[j]) ➨ invariant maintained.
Loop invariant At the start of each iteration of the “outer” for loop (indexed by j) the subarray A[1..j-1] consists of the elements originally in A[1..j-1] but in sorted order.
![Page 27: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/27.jpg)
Correctness proof
InsertionSort(A)1. initialize: sort A[1]2. for j = 2 to A.length3. do key = A[j] 4. i = j -15. while i > 0 and A[i] > key6. do A[i+1] = A[i] 7. i = i -18. A[i +1] = key
TerminationThe outer for loop ends when j > n; this is when j = n+1 ➨ j-1 = n. Plug n for j-1 in the loop invariant ➨ the subarray A[1..n] consists of the elements originally in A[1..n] in sorted order.
Loop invariant At the start of each iteration of the “outer” for loop (indexed by j) the subarray A[1..j-1] consists of the elements originally in A[1..j-1] but in sorted order.
![Page 28: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/28.jpg)
Another sorting algorithm
using a different paradigm …
![Page 29: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/29.jpg)
MergeSort
A divide-and-conquer sorting algorithm.
Divide-and-conquerbreak the problem into two or more subproblems, solve the subproblems recursively, and then combine these solutions to create a solution to the original problem.
![Page 30: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/30.jpg)
D&CAlg(A)
// divide-and-conquer algorithm that computes the solution of a problem with input A = {x1,…,xn}
1. if # elements of A is small enough (for example 1)
2. then compute Sol (the solution for A) brute-force
3. else
4. split A in, for example, 2 non-empty subsets A1 and A2
5. Sol1 = D&CAlg(A1)
6. Sol2 = D&CAlg(A2)
7. compute Sol (the solution for A) from Sol1 and Sol28. return Sol
Divide-and-conquer
![Page 31: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/31.jpg)
MergeSort(A)
// divide-and-conquer algorithm that sorts array A[1..n]
1. if A.length == 1
2. then compute Sol (the solution for A) brute-force
3. else
4. split A in 2 non-empty subsets A1 and A2
5. Sol1 = MergeSort(A1)
6. Sol2 = MergeSort(A2)
7. compute Sol (the solution for A) from Sol1 en Sol2
MergeSort
![Page 32: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/32.jpg)
MergeSort(A)
// divide-and-conquer algorithm that sorts array A[1..n]
1. if A.length == 1
2. then skip
3. else
4. n = A.length ; n1 = n/2 ; n2 = n/2 ;
copy A[1.. n1] to auxiliary array A1[1.. n1]
copy A[n1+1..n] to auxiliary array A2[1.. n2]
5. MergeSort(A1)
6. MergeSort(A2)
7. Merge(A, A1, A2)
MergeSort
![Page 33: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/33.jpg)
MergeSort
3 14 1 28 17 8 21 7 4 35
1 3 4 7 8 14 17 21 28 35
8 21 7 4 353 14 1 28 17
1 3 14 17 28 4 7 8 21 35
3 14
3 14
1 28 17
1 17 28
3 14
![Page 34: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/34.jpg)
MergeSort
Merging
1 3 4 7 8 14 17 21 28 35A
1 3 14 17 28A14 7 8 21 35A2
![Page 35: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/35.jpg)
MergeSort: correctness proof
Induction on n (# of input elements)
proof that the base case (n small) is solved correctly
proof that if all subproblems are solved correctly, then the complete problem is solved correctly
![Page 36: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/36.jpg)
MergeSort: correctness proof
Proof (by induction on n)
Base case: n = 1, trivial ✔
Inductive step: assume n > 1. Note that n1 < n and n2 < n.
Inductive hypothesis ➨ arrays A1 and A2 are sorted correctly
Remains to show: Merge(A, A1, A2) correctly constructs a sortedarray A out of the sorted arrays A1 and A2 … etc.
■
MergeSort(A)
1. if length[A] = 1
2. then skip
3. else
4. n = A.length ; n1 = n/2 ; n2 = n/2 ;
copy A[1.. n1] to auxiliary array A1[1.. n1]
copy A[n1+1..n] to auxiliary array A2[1.. n2]
5. MergeSort(A1)
6. MergeSort(A2)
7. Merge(A, A1, A2)
Lemma MergeSort sorts the array A[1..n] correctly.
![Page 37: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/37.jpg)
QuickSort
another divide-and-conquer sorting algorithm…
![Page 38: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/38.jpg)
QuickSort
QuickSort(A)
// divide-and-conquer algorithm that sorts array A[1..n]
1. if length[A] ≤ 1
2. then skip
3. else
4. pivot = A[1]
5. move all A[i] with A[i] < pivot into auxiliary array A1
6. move all A[i] with A[i] > pivot into auxiliary array A2
7. move all A[i] with A[i] = pivot into auxiliary array A3
8. QuickSort(A1)
9. QuickSort(A2)
10. A = “A1 followed by A3 followed by A2”
![Page 39: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/39.jpg)
Analysis of algorithms
some informal thoughts – for now …
![Page 40: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/40.jpg)
Analysis of algorithms
Can we say something about the running time of an algorithm without implementing and testing it?
InsertionSort(A)1. initialize: sort A[1]2. for j = 2 to A.length3. do key = A[j] 4. i = j -15. while i > 0 and A[i] > key6. do A[i+1] = A[i] 7. i = i -18. A[i +1] = key
![Page 41: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/41.jpg)
Analysis of algorithms
Analyze the running time as a function of n (# of input elements)
best case average case worst case
elementary operationsadd, subtract, multiply, divide, load, store, copy, conditional and unconditional branch, return …
An algorithm has worst case running time T(n) if for any input of size n the maximal number of elementary operations executed is T(n).
![Page 42: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/42.jpg)
Analysis of algorithms: example
InsertionSort: 15 n2 + 7n – 2
MergeSort: 300 n lg n + 50 n
n=10 n=100 n=1000
1568 150698 1.5 x 107
10466 204316 3.0 x 106
InsertionSort6 x faster
InsertionSort1.35 x faster
MergeSort5 x faster
n = 1,000,000 InsertionSort 1.5 x 1013
MergeSort 6 x 109 2500 x faster !
![Page 43: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/43.jpg)
Analysis of algorithms
It is extremely important to have efficient algorithms for large inputs
The rate of growth (or order of growth) of the running time is far more important than constants
InsertionSort: Θ(n2)
MergeSort: Θ(n log n)
![Page 44: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/44.jpg)
Θ-notation
Intuition: concentrate on the leading term, ignore constants
19 n3 + 17 n2 - 3n becomes Θ(n3)
2 n lg n + 5 n1.1 - 5 becomes
n - ¾ n √n becomes
(precise definition next lecture…)
Θ(n1.1)
---
![Page 45: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/45.jpg)
Some rules and notation
log n denotes log2 n
We have for a, b, c > 0 :
1. logc (ab) =
logc a + logc b
2. logc (ab) =
b logc a
3. loga b =
logc b / logc a
![Page 46: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/46.jpg)
Find the leading term
lg35n vs. √n ?
logarithmic functions grow slower than polynomial functions lga n grows slower than nb for all constants a > 0 and b > 0
n100 vs. 2 n ?
polynomial functions grow slower than exponential functions na grows slower than bn for all constants a > 0 and b > 1
![Page 47: 2IL50 Data Structures Spring 2015 Lecture 1: Introduction](https://reader035.vdocument.in/reader035/viewer/2022081511/56649d6f5503460f94a50646/html5/thumbnails/47.jpg)
Announcements
Today 7+8 Reading and writing mathematical proofs
Today 7+8+9 Emergency Latex office hour in MF 11
Today 18:00 Group registration opens on OASE
Wednesday 1+2 tutorials, discuss solutions to homework A1 from 2014
(see web-page)