complexity analysis of algorithmsjordicf/teaching/fme... · •complexity analysis is a technique...
TRANSCRIPT
![Page 1: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/1.jpg)
Complexity Analysisof Algorithms
Jordi Cortadella
Department of Computer Science
![Page 2: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/2.jpg)
Estimating runtime
• What is the runtime of g(n)?
Introduction to Programming © Dept. CS, UPC 2
void g(int n) {for (int i = 0; i < n; ++i) f();
}
void g(int n) {for (int i = 0; i < n; ++i)
for (int j = 0; j < n; ++j) f();}
![Page 3: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/3.jpg)
Estimating runtime
• What is the runtime of g(n)?
Introduction to Programming © Dept. CS, UPC 3
void g(int n) {for (int i = 0; i < n; ++i)
for (int j = 0; j <= i; ++j) f();}
![Page 4: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/4.jpg)
Complexity analysis
• A technique to characterize the execution time of an algorithm independently from the machine, the language and the compiler.
• Useful for:– evaluating the variations of execution time with regard
to the input data
– comparing algorithms
• We are typically interested in the execution time of large instances of a problem, e.g., when n , (asymptotic complexity).
Introduction to Programming © Dept. CS, UPC 4
![Page 5: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/5.jpg)
Big O
• A method to characterize the execution time of an algorithm:
– Adding two square matrices is O(n2)
– Searching in a dictionary is O(log n)
– Sorting a vector is O(n log n)
– Solving Towers of Hanoi is O(2n)
– Multiplying two square matrices is O(n3)
– …
• The O notation only uses the dominating terms of the execution time. Constants are disregarded.
Introduction to Programming © Dept. CS, UPC 5
![Page 6: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/6.jpg)
Big O: formal definition
• Let T(n) be the execution time of an algorithm when the size of input data is n.
• T(n) is O(f(n)) if there are positive constants c and n0
such that T(n) cf(n) when n n0.
Introduction to Programming © Dept. CS, UPC 6
n0 n
cf(n)
T(n)
![Page 7: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/7.jpg)
Big O: example
• Let T(n) = 3n2 + 100n + 5, then T(n) = O(n2)
• Proof:
– Let c = 4 and n0 = 100.05
– For n 100.05, we have that 4n2 3n2 + 100n + 5
• T(n) is also O(n3), O(n4), etc.Typically, the smallest complexity is used.
Introduction to Programming © Dept. CS, UPC 7
![Page 8: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/8.jpg)
Big O: examples
Introduction to Programming © Dept. CS, UPC 8
![Page 9: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/9.jpg)
Complexity ranking
Introduction to Programming © Dept. CS, UPC 9
![Page 10: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/10.jpg)
Complexity analysis: examples
Let us assume that f() has complexity O(1)
Introduction to Programming © Dept. CS, UPC 10
for (int i = 0; i < n; ++i) f();
for (int i = 0; i < n; ++i)for (int j = 0; j < n; ++j) f();
for (int i = 0; i < n; ++i)for (int j = 0; j <= i; ++j) f();
for (int i = 0; i < n; ++i)for (int j = 0; j < n; ++j)for (int k = 0; k < n; ++k) f();
for (int i = 0; i < m; ++i)for (int j = 0; j < n; ++j)for (int k = 0; k < p; ++k) f();
![Page 11: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/11.jpg)
Complexity analysis: recursionvoid f(int n) {
if (n > 0) { DoSomething(n); // O(n)f(n/2);
}}
Introduction to Programming © Dept. CS, UPC 11
![Page 12: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/12.jpg)
Complexity analysis: recursionvoid f(int n) {
if (n > 0) { DoSomething(n); // O(n)f(n/2); f(n/2);
}}
Introduction to Programming © Dept. CS, UPC 12
n
n/2
n/4
…
1 1
…
1 1
n/4
…
1 1
…
1 1
n/2
n/4
…
1 1
…
1 1
n/4
…
1 1
…
1 1
![Page 13: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/13.jpg)
Complexity analysis: recursionvoid f(int n) {
if (n > 0) { DoSomething(n); // O(n)f(n-1);
}}
Introduction to Programming © Dept. CS, UPC 13
![Page 14: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/14.jpg)
Complexity analysis: recursionvoid f(int n) {
if (n > 0) { DoSomething(); // O(1)f(n-1); f(n-1);
}}
Introduction to Programming © Dept. CS, UPC 14
![Page 15: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/15.jpg)
Asymptotic complexity (small values)
Introduction to Programming © Dept. CS, UPC 15
![Page 16: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/16.jpg)
Asymptotic complexity (larger values)
Introduction to Programming © Dept. CS, UPC 16
![Page 17: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/17.jpg)
Execution time: example
Let us consider that every operation can be executed in 1 ns (10-9 s).
Introduction to Programming © Dept. CS, UPC 17
![Page 18: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/18.jpg)
How about “big data”?
Algorithm Analysis © Dept. CS, UPC 18
Source: Jon Kleinberg and Éva Tardos, Algorithm Design, Addison Wesley 2006.
This is often the practical limit for big data
![Page 19: Complexity Analysis of Algorithmsjordicf/Teaching/FME... · •Complexity analysis is a technique to analyze and compare algorithms (not programs). •It helps to have preliminary](https://reader034.vdocument.in/reader034/viewer/2022052612/5f0af04d7e708231d42e14e6/html5/thumbnails/19.jpg)
Summary• Complexity analysis is a technique to analyze and compare
algorithms (not programs).
• It helps to have preliminary back-of-the-envelope estimations of runtime (milliseconds, seconds, minutes, days, years?).
• Worst-case analysis is sometimes overly pessimistic. Average case is also interesting (not covered in this course).
• In many application domains (e.g., big data) quadratic complexity, 𝑂 𝑛2 , is not acceptable.
• Recommendation: avoid last-minute surprises by doing complexity analysis before writing code.
Introduction to Programming © Dept. CS, UPC 19