tecniche di programmazione a.a. 2020/2021
TRANSCRIPT
![Page 1: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/1.jpg)
Computational complexity
Tecniche di Programmazione – A.A. 2020/2021
![Page 2: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/2.jpg)
How to Measure Efficiency?
A.A. 2020/21Tecniche di programmazione
Critical resources
programmer’s effort
time, space (disk, RAM)
Analysis
empirical (run programs)
analytical (asymptotic algorithm analysis)
Worst case vs. Average case
2
![Page 3: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/3.jpg)
Moore’s “Law”?
A.A. 2020/21Tecniche di programmazione3
https://en.wikipedia.org/wiki/Moore%27s_law
![Page 4: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/4.jpg)
Moore’s “Law”?
A.A. 2020/21Tecniche di programmazione4
![Page 5: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/5.jpg)
Problems and Algorithms
A.A. 2020/21Tecniche di programmazione5
We know the efficiency of the solution
… but what about the difficulty of the problem?
Different concepts
Algorithm complexity
Problem complexity
![Page 6: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/6.jpg)
Analytical Approach
A.A. 2020/21Tecniche di programmazione
For most algorithms, running time depends on “size” of
the input
Running time is expressed as T(n)
some function T
input size n
6
![Page 7: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/7.jpg)
Bubble sort
A.A. 2020/21Tecniche di programmazione7
![Page 8: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/8.jpg)
Analysis
A.A. 2020/21Tecniche di programmazione
The bubble sort takes (n2-n)/2 “steps”
Different implementations/assembly languages
Program A on an Intel Pentium IV: T(n) = 58*(n2-n)/2
Program B on a Motorola: T(n) = 84*(n2-2n)/2
Program C on an Intel Pentium V: T(n) = 44*(n2-n)/2
Note that each has an n2 term
as n increases, the other terms
will drop out
8
![Page 9: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/9.jpg)
Analysis
A.A. 2020/21Tecniche di programmazione
As a result:
Program A on Intel Pentium IV: T(n) ≈ 29n2
Program B on Motorola: T(n) ≈ 42n2
Program C on Intel Pentium V: T(n) ≈ 22n2
9
![Page 10: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/10.jpg)
Analysis
A.A. 2020/21Tecniche di programmazione
As processors change, the constants will always change
The exponent on n will not
We should not care about the constants
As a result:
Program A: T(n) ≈ n2
Program B: T(n) ≈ n2
Program C: T(n) ≈ n2
Bubble sort: T(n) ≈ n2
10
![Page 11: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/11.jpg)
Complexity Analysis
A.A. 2020/21Tecniche di programmazione
O( ∙ )
big o (big oh)
( ∙ )
big omega
( ∙ )
big theta
12
![Page 12: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/12.jpg)
O( ∙ ) = Upper Bounding Running Time
A.A. 2020/21Tecniche di programmazione13
Upper Bounding Running Time
f(n) is O(g(n)) if f grows “at most as fast as” g
![Page 13: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/13.jpg)
Example
A.A. 2020/21Tecniche di programmazione14
(log n)2 = O(n)
![Page 14: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/14.jpg)
Common Misunderstanding
A.A. 2020/21Tecniche di programmazione
3x3 + 5x2 – 9 = O(x3)
However, also true are:
3x3 + 5x2 – 9 = O(x4)
x3 = O(3x3 + 5x2 – 9)
sin(x) = O(x4)
Note:
Usage of big-O typically involves mentioning
only the most dominant term
“The running time is O(x2.5)”
15
![Page 15: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/15.jpg)
( ∙ ) = Lower Bounding Running Time
A.A. 2020/21Tecniche di programmazione16
f(n) is (g(n)) if f grows “at least as fast as” g
cg(n) is an approximation to f(n) bounding from below
![Page 16: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/16.jpg)
( ∙ ) = Tightly Bounding Running Time
A.A. 2020/21Tecniche di programmazione17
f(n) is (g(n)) if f is essentially the same as g, to
within a constant multiple
![Page 17: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/17.jpg)
Big-, Big-O, and Big-
A.A. 2020/21Tecniche di programmazione18
![Page 18: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/18.jpg)
Practical approach
A.A. 2020/21Tecniche di programmazione19
![Page 19: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/19.jpg)
A.A. 2020/21Tecniche di programmazione20
![Page 20: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/20.jpg)
Would it be possible?
A.A. 2020/21Tecniche di programmazione21
Algorithm Foo Bar
Complexity O(n2) O(2n)
n = 100 10s 4s
n = 1000 12s 4.5s
![Page 21: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/21.jpg)
Determination of Time Complexity
A.A. 2020/21Tecniche di programmazione22
Because of the approximations available through Big-O ,
the actual T(n) of an algorithm is not calculated
T(n) may be determined empirically
Big-O is usually determined by application of some simple
5 rules
![Page 22: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/22.jpg)
Rule #1
A.A. 2020/21Tecniche di programmazione23
Simple program statements are assumed to take a
constant amount of time which is
O(1)
![Page 23: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/23.jpg)
Rule #2
A.A. 2020/21Tecniche di programmazione24
Differences in execution time of simple statements is
ignored
![Page 24: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/24.jpg)
Rule #3
A.A. 2020/21Tecniche di programmazione25
In conditional statements the worst case is always used
![Page 25: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/25.jpg)
Rule #4 – the “sum” rule
A.A. 2020/21Tecniche di programmazione26
The running time of a sequence of steps has the order
of the running time of the largest
E.g.,
f(n) = O(n2)
g(n) = O(n3)
f(n) + g(n) = O(n3)Worst case
(valid for big-O, not
for big-θ)
![Page 26: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/26.jpg)
Rule #5 – the “product” rule
A.A. 2020/21Tecniche di programmazione27
If two processes are constructed such that the second
process is repeated a number of times for each
execution of the first process, then O is equal to the
product of the orders of magnitude of the two
processes
E.g.,
For example, a two-dimensional array has one for loop inside
another and each internal loop is executed n times for each
value of the external loop.
The function is O(n2)
![Page 27: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/27.jpg)
Nested Loops
A.A. 2020/21Tecniche di programmazione28
for(int t=0; t<n; ++t) {
for(int u=0; u<n; ++u) {
++zap;
}
}
O(n)O(1)
![Page 28: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/28.jpg)
Nested Loops
A.A. 2020/21Tecniche di programmazione29
for(int t=0; t<n; ++t) {
for(int u=0; u<n; ++u) {
++zap;
}
}
O(n*1)
![Page 29: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/29.jpg)
Nested Loops
A.A. 2020/21Tecniche di programmazione30
for(int t=0; t<n; ++t) {
for(int u=0; u<n; ++u) {
++zap;
}
}
O(n)
O(n)
![Page 30: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/30.jpg)
Nested Loops
A.A. 2020/21Tecniche di programmazione31
for(int t=0; t<n; ++t) {
for(int u=0; u<n; ++u) {
++zap;
}
}
O(n2)
![Page 31: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/31.jpg)
Nested Loops
A.A. 2020/21Tecniche di programmazione32
Note: Running time grows with nesting rather than the
length of the code
for(int t=0; t<n; ++t) {
for(int u=0; u<n; ++u) {
++zap;
}
}
O(n2)
![Page 32: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/32.jpg)
More Nested Loops
A.A. 2020/21Tecniche di programmazione33
for(int t=0; t<n; ++t) {
for(int u=t; u<n; ++u) {
++zap;
}
}
( )( ) ( )2
21
0 22
1nO
nnnnin
n
i=
−=
−=−
−
=
tn −
![Page 33: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/33.jpg)
Sequential statements
A.A. 2020/21Tecniche di programmazione34
Running time: max(O(n), O(n2)) = O(n2)
for(int z=0; z<n; ++z)
zap[z] = 0;
for(int t=0; t<n; ++t) {
for(int u=t; u<n; ++u) {
++zap;
}
}
O(n)
O(n2)
![Page 34: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/34.jpg)
Conditionals
A.A. 2020/21Tecniche di programmazione35
for(int t=0; t<n; ++t) {
if(t%2) {
for(int u=t; u<n; ++u) {
++zap;
}
} else {
zap = 0;
}
}
O(n)
O(1)
![Page 35: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/35.jpg)
Conditionals
A.A. 2020/21Tecniche di programmazione36
for(int t=0; t<n; ++t) {
if(t%2) {
for(int u=t; u<n; ++u) {
++zap;
}
} else {
zap = 0;
}
}
O(n2)
![Page 36: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/36.jpg)
Tips
A.A. 2020/21Tecniche di programmazione37
Focus only on the dominant (high cost) operations and
avoid a line-by-line exact analysis
Break algorithm down into “known” pieces
Identify relationships between pieces
Sequential is additive
Nested (loop / recursion) is multiplicative
Drop constants
Keep only dominant factor for each variable
![Page 37: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/37.jpg)
Basic Asymptotic Efficiency Classes
A.A. 2020/21Tecniche di programmazione38
![Page 38: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/38.jpg)
Basic Asymptotic Efficiency Classes
A.A. 2020/21Tecniche di programmazione39
![Page 39: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/39.jpg)
Basic Asymptotic Efficiency Classes
A.A. 2020/21Tecniche di programmazione40
![Page 40: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/40.jpg)
Basic Asymptotic Efficiency Classes
A.A. 2020/21Tecniche di programmazione41
![Page 41: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/41.jpg)
Basic Asymptotic Efficiency Classes
A.A. 2020/21Tecniche di programmazione42
![Page 42: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/42.jpg)
Basic Asymptotic Efficiency Classes
A.A. 2020/21Tecniche di programmazione43
![Page 43: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/43.jpg)
Basic Asymptotic Efficiency Classes
A.A. 2020/21Tecniche di programmazione44
![Page 44: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/44.jpg)
Basic Asymptotic Efficiency Classes
A.A. 2020/21Tecniche di programmazione45
![Page 45: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/45.jpg)
ArrayList vs. LinkedList
A.A. 2020/21Tecniche di programmazione46
ArrayList LinkedList
add(element) O(1) O(1)
remove(object) O(n) + O(n) O(n) + O(1)
get(index) O(1) O(n)
set(index, element) O(1) O(n) + O(1)
add(index, element) O(1) + O(n) O(n) + O(1)
remove(index) O(n) O(n) + O(1)
contains(object) O(n) O(n)
indexOf(object) O(n) O(n)
![Page 46: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/46.jpg)
Recursion Complexity
Recursion
![Page 47: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/47.jpg)
Divide et Impera – Divide and Conquer
Solve ( Problem ) {
if( problem is trivial )
Solution = Solve_trivial ( Problem ) ;
else {
List<SubProblem> subProblems = Divide ( Problem ) ;
For ( each subP[i] in subProblems ) {
SubSolution[i] = Solve ( subP[i] ) ;
}
Solution = Combine ( SubSolution[ 1..N ] ) ;
}
return Solution ;
}
do recursion
A.A. 2020/21Tecniche di programmazione48
![Page 48: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/48.jpg)
What about complexity?
A.A. 2020/21Tecniche di programmazione49
a = number of sub-problems for a problem
b = how smaller sub-problems are than the original one
n = size of the original problem
T(n) = complexity of Solve
…our unknown complexity function
Θ(1) = complexity of Solve_trivial
…otherwise it wouldn’t be trivial
D(n) = complexity of Divide
C(n) = complexity of Combine
![Page 49: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/49.jpg)
Divide et Impera – Divide and Conquer
Solve ( Problem ) {
if( problem is trivial )
Solution = Solve_trivial ( Problem ) ;
else {
List<SubProblem> subProblems = Divide ( Problem ) ;
For ( each subP[i] in subProblems ) {
SubSolution[i] = Solve ( subP[i] ) ;
}
Solution = Combine ( SubSolution[ 1..a ] ) ;
}
return Solution ;
}
T(n)
Θ(1)
D(n)
C(n)
T(n/b)
a times
A.A. 2020/21Tecniche di programmazione50
![Page 50: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/50.jpg)
Complexity computation
T(n) =
(1) for n c
D(n) + a T(n/b) + C(n) for n > c
Recurrence Equation not easy to solve in the general case
Special case:
If D(n)+C(n)=(n)
We obtain T(n) = (n log n).
A.A. 2020/21Tecniche di programmazione51
![Page 51: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/51.jpg)
Examples
A.A. 2020/21Tecniche di programmazione52
Algorithm Solve_
trivial
Divide a b Combine Complexity
Dicotomic
search
1 1 1 2 0 Log(n)
Merge Sort 1 1 2 2 n n Log(n)
Permutation 0 1 n n-1 0 Γ(n)
Combinations 0 1 k n 0 k^n
Factorial 1 0 1 n-1 0 n
Fibonacci 1 0 2 n-1 0 2^n
![Page 52: Tecniche di Programmazione A.A. 2020/2021](https://reader031.vdocument.in/reader031/viewer/2022012023/6169cf4711a7b741a34b9c43/html5/thumbnails/52.jpg)
Licenza d’uso
A.A. 2020/21Tecniche di programmazione53
Queste diapositive sono distribuite con licenza Creative Commons“Attribuzione - Non commerciale - Condividi allo stesso modo (CC BY-NC-SA)”
Sei libero: di riprodurre, distribuire, comunicare al pubblico, esporre in pubblico,
rappresentare, eseguire e recitare quest'opera
di modificare quest'opera
Alle seguenti condizioni: Attribuzione — Devi attribuire la paternità dell'opera agli autori
originali e in modo tale da non suggerire che essi avallino te o il modo in cui tu usi l'opera.
Non commerciale — Non puoi usare quest'opera per fini commerciali.
Condividi allo stesso modo — Se alteri o trasformi quest'opera, o se la usi per crearne un'altra, puoi distribuire l'opera risultante solo con una licenza identica o equivalente a questa.
http://creativecommons.org/licenses/by-nc-sa/3.0/