l-03 complexity analysis 2.ppt
TRANSCRIPT
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 1/24
Data Structures (FALL-2011)
Program Efficiency
&
Complexity Analysis
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 2/24
Data Structures (FALL-2011)
Efficient Algorithm
A program that looks up a specific entry in a sorted list of size
n. Suppose this program were implemented on Computer A, a
state-of-the-art machine, using a linear search algorithm, and
on Computer B, a much slower machine, using a binary search
algorithm. Benchmark testing on the two computers running
their respective programs might look something like the
following:
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 3/24
Data Structures (FALL-2011)
n (list size)Computer A run-time
(in nanoseconds)
Computer B run-time
(in nanoseconds)
15 7 ns 100,000 ns
65 32 ns 150,000 ns
250 125 ns 200,000 ns
1,000 500 ns 250,000 ns
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 4/24
Data Structures (FALL-2011)
Based on these metrics, it would be easy to
jump to the conclusion that Computer A is
running an algorithm that is far superior in
efficiency to that of Computer B. However, if
the size of the input-list is increased to a
sufficient number, that conclusion is
dramatically demonstrated to be in error:
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 5/24
Data Structures (FALL-2011)
n
(list size)
Computer A run-time
(in nanoseconds)
Computer B run-time
(in nanoseconds)
15 7 ns 100,000 ns
65 32 ns 150,000 ns
250 125 ns 200,000 ns
1,000 500 ns 250,000 ns
... ... ...
1,000,000 500,000 ns 500,000 ns
4,000,000 2,000,000 ns 550,000 ns
16,000,000 8,000,000 ns 600,000 ns
... ... ...
63,072 × 1012 31,536 × 1012 ns,
or 1 year
1,375,000 ns,
or 1.375 milliseconds
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 6/24
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 7/24
Data Structures (FALL-2011)
Big Oh Notation [1]
If f(N) and g(N) are two complexity functions, we say
f(N) = O(g(N))
(read "f(N) is order g(N)", or "f(N) is big-O of g(N)")
if there are constants c and N0 such that
f(N) ≤ c * g(N)
for all sufficiently large N.
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 8/24
Data Structures (FALL-2011)
Example (1)
• Consider
f(n)=2n2+3
and g(n)=n2
Is f(n)=O(g(n))? i.e. Is 2n2+3 = O(n2)?
Proof:
2n2+3 ≤ c * n2
Assume N0 =1 and c=1?
Assume N0
=1 and c=2?
Assume N0 =1 and c=3?
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 9/24
Data Structures (FALL-2011)
Comparing Functions
• As inputs get larger, any algorithm of a smaller
order will be more efficient than an algorithm
of a larger order
T
i m e ( s t e p s )
Input (size)
3N = O(N)
0.05 N2 = O(N2)
N = 60
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 10/24
Data Structures (FALL-2011)
Big-Oh Notation
• Even though it is correct to say “7n - 3 is O(n3)”,
a better statement is “7n - 3 is O(n)”, that is, one
should make the approximation as tight as possible
• Simple Rule:
Drop lower order terms and constant factors
7n-3 is O(n)
8n2log n + 5n2 + n is O(n2log n)
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 11/24
Data Structures (FALL-2011)
Asymptotic notation
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 12/24
Data Structures (FALL-2011)
Size does matter[1]
What happens if we double the input size N?
N log2N 5N N log2N N2 2N
8 3 40 24 64 256
16 4 80 64 256 65536
32 5 160 160 1024 ~109
64 6 320 384 4096 ~1019
128 7 640 896 16384 ~1038
256 8 1280 2048 65536 ~1076
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 13/24
Data Structures (FALL-2011)
Size does matter[2]
• Suppose a program has run time O(n!) and therun time for
n = 10 is 1 second
For n = 12, the run time is 2 minutes
For n = 14, the run time is 6 hours
For n = 16, the run time is 2 monthsFor n = 18, the run time is 50 years
For n = 20, the run time is 200 centuries
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 14/24
Data Structures (FALL-2011)
Standard Analysis Techniques
• Constant time statements
• Analyzing Loops
• Analyzing Nested Loops
• Analyzing Sequence of Statements
• Analyzing Conditional Statements
D S (FA 2011)
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 15/24
Data Structures (FALL-2011)
Constant time statements
• Simplest case: O(1) time statements
• Assignment statements of simple data typesint x = y;
• Arithmetic operations:x = 5 * y + 4 - z;
• Array referencing:A[j] = 5;
• Array assignment:
j, A[j] = 5;• Most conditional tests:
if (x < 12) ...
D t St t (FALL 2011)
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 16/24
Data Structures (FALL-2011)
Analyzing Loops[1]
• Any loop has two parts: – How many iterations are performed?
– How many steps per iteration?
int sum = 0,j;for (j=0; j < N; j++)
sum = sum +j;
– Loop executes N times (0..N-1) – 4 = O(1) steps per iteration
• Total time is N * O(1) = O(N*1) = O(N)
D t St t (FALL 2011)
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 17/24
Data Structures (FALL-2011)
Analyzing Loops[2]
• What about this for loop?
int sum =0, j;
for (j=0; j < 100; j++)
sum = sum +j;
• Loop executes 100 times
• 4 = O(1) steps per iteration
• Total time is 100 * O(1) = O(100 * 1) = O(100)= O(1)
D t St t (FALL 2011)
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 18/24
Data Structures (FALL-2011)
Analyzing Nested Loops[1]
• Treat just like a single loop and evaluate each level of nesting as needed:
int j,k;
for (j=0; j<N; j++)
for (k=N; k>0; k--)sum += k+j;
• Start with outer loop: – How many iterations? N
– How much time per iteration? Need to evaluate inner loop• Inner loop uses O(N) time
• Total time is N * O(N) = O(N*N) = O(N2)
D t St t (FALL 2011)
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 19/24
Data Structures (FALL-2011)
Analyzing Nested Loops[2]
• What if the number of iterations of one loopdepends on the counter of the other?
int j,k;
for (j=0; j < N; j++)for (k=0; k < j; k++)
sum += k+j;
• Analyze inner and outer loop together:• Number of iterations of the inner loop is:
• 0 + 1 + 2 + ... + (N-1) = O(N2)
Data Str ct res (FALL 2011)
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 20/24
Data Structures (FALL-2011)
Analyzing Sequence of Statements
• For a sequence of statements, compute their complexity functions individually and addthem up
for (j=0; j < N; j++)for (k =0; k < j; k++)
sum = sum + j*k;
for (l=0; l < N; l++)
sum = sum -l;cout<<“Sum=”<<sum;
Total cost is O(N2) + O(N) +O(1) = O(N2)SUM RULE
O(N2
)
O(N)
O(1)
Data Structures (FALL 2011)
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 21/24
Data Structures (FALL-2011)
Analyzing Conditional Statements
What about conditional statements such as
if (condition)
statement1;
elsestatement2;
where statement1 runs in O(N) time and statement2 runs inO(N2) time?
We use "worst case" complexity: among all inputs of size N, thatis the maximum running time?
The analysis for the example above is O(N2)
Data Structures (FALL 2011)
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 22/24
Data Structures (FALL-2011)
int CompareSmallestNumber (int array[ ])
{
int x, curMin;
// set smallest value to first item in array
curMin = array[0];
// iterate through array to find smallest valuefor (x = 1; x < 10; x++)
{
if( array[x] < curMin)
{ curMin = array[x]; }}
// return smallest value in the array return curMin;
}
Data Structures (FALL 2011)
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 23/24
Data Structures (FALL-2011)
int CompareToAllNumbers (int array[ ])
{
bool is Min;
int x, y;
// iterate through each
for (int x = 0; x < 10; x++)
{ isMin = true;
for (int y = 0; y < 10; y++)
{/* compare the value in array[x] to the other values if we find that array[x]
is greater than any of the values in array[y] then we know that the value in array[x] is notthe minimum
remember that the 2 arrays are exactly the same, we are just taking out one
value with index 'x' and comparing to the other values in the array with index 'y' */
if( array[x] > array[y])
isMin = false;
}
if(isMin)
break;
}
return array[x];
}
Data Structures (FALL 2011)
7/29/2019 L-03 Complexity Analysis 2.ppt
http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 24/24
Data Structures (FALL-2011)
Exercise
• f(n) = 3x2 + 2x + 2
• f(n) = 10n + 5n2 + n3
• f(n) = 10n + 5 and g(n) = n
SHOW f(n) is O(g(n))• f(n)=10n3+6n2+5