9 big o-notation

Big O Notation

outline

Performance vs Complexity

Algorithm Efficiency

Big O Notation

Performance vs Complexity

• Performance:

How much time/memory/disk/… is used by the algorithm

Depends on the machine, compiler, code,…

• Complexity:

How do the resource requirements of a program/algorithm scale

Complexity affects performancebut not the other way aroundIf I have an algorithm with high complexity, it will take longer time to run.However, if I have a very slow program, that does not mean that my algorithm is very complex; that could be all other kind of issues that made the program run slow.

Complexity of an Algorithm

We are going to count the number of the steps that are taken during the execution

Assumption (simplification):

Each step costs the same amount

+, -, return a value, access an array element, ….

Complexity of an Algorithm (cont.)• Example 1:

• private int getLastElement (int [] numbers)

• {

return numbers[numbers.length -1];

}

Number of steps: 4

Constant time function (independent of array size)

Notes: Let’s look at each step that need to be executed. It is always 4 steps to access the last element, no matter the number of elements.

Complexity of an Algorithm (cont.)

• Example 2:• private static int sum(int [] numbers)

• {

int sum = 0;

for (int i = 0; i < number.length; i++){

sum = sum + number[i];

}

return sum;

}

Outside loop: 3 + 2 Number of steps:6n + 5Linear time functionLoop body: 6

INSTRUCTOR-NOTES:This time we use a loop.We will start by the steps that will be executed only once and the are outside the loop.Then we count the steps that are going to be repeated over and over again.

Algorithm Efficiency

Let’s look at the following algorithm for initializing the values in an array:

final int N = 500;

int [] counts = new int[N];

for (int i=0; i<counts.length; i++)

counts[i] = 0;

The length of time the algorithm takes to execute depends on the value of N

Algorithm Efficiency (cont.)

In that algorithm, we have one loop that processes all of the elements in the array

Intuitively:

– If N was half of its value, we would expect the algorithm to take half the time

– If N was twice its value, we would expect the algorithm to take twice the time

That is true and we say that the algorithm efficiency relative to N is linear


Let’s look at another algorithm for initializing the values in a different array:

final int N = 500;

int [] [] counts = new int[N][N];

for (int i=0; i<counts.length; i++)

for (int j=0; j<counts[i].length; j++)

counts[i][j] = 0;

The length of time the algorithm takes to execute still depends on the value of N


However, in the second algorithm, we have two nested loops to process the elements in the two dimensional array

Intuitively:

– If N is half its value, we would expect the algorithm to take one quarter the time

– If N is twice its value, we would expect the algorithm to take quadruple the time

That is true and we say that the algorithm efficiency relative to N is quadratic

Big O notation

Why do we need Big-O?

In computer science, we use Big-O to classify algorithms based on their scalability.

Basically, it tells you how fast a function grows or declines.

The letter O is used because the rate of growth of a function is also called its order.

It also allows us to estimate the worst case running time of an algorithm as a function of the input size.

Big O notation

Examples:

– We can say that the first algorithm is O(n)

– We can say that the second algorithm is O(n2)

For any algorithm that has a function g(n) of the parameter n that describes its length of time to execute, we can say the algorithm is O(g(n))

We only include the fastest growing term and ignore any multiplying by or adding of constants (which makes sense because those depend on the particular hardware the program is run on)

Big O notation (cont.)

Growth Functions:Eight functions O(n) that occur frequently in the analysis of algorithms (in order of increasing rate of growth relative to n):

– Constant 1

– Logarithmic log n

– Linear n

– Log Linear n log n

– Quadratic n2

– Cubic n3

– Exponential 2n

– Exhaustive Search n!


Growth Rates Compared:


The growth list is useful because of the following fact:

if a function f(n) is a sum of functions, one of which grows faster than the others, then the faster growing one determines the order of f(n).

Example:

If f(n) = 10 log(n) + 5 (log(n))3 + 7 n + 3 n 2 + 6 n 3,

then f(n) = O(n 3).


Big-O for a Problem:

O(g(n)) for a problem means there is some O(g(n)) algorithm that solves the problem

Don’t assume that the specific algorithm that you are currently using is the best solution for the problem

There may be other correct algorithms that grow at a smaller rate with increasing n

Many times, the goal is to find an algorithm with the smallest possible growth rate


Role of Data Structures:

That brings up the topic of the structure of the data on which the algorithm operates

If we are using an algorithm manually on some amount of data, we intuitively try to organize the data in a way that minimizes the number of steps that we need to take

Publishers offer dictionaries with the words listed in alphabetical order to minimize the length of time it takes us to look up a word



We can do the same thing for algorithms in our computer programs

Example: Finding a numeric value in a list

If we assume that the list is unordered, we must search from the beginning to the end

On average, we will search half the list

Worst case, we will search the entire list

Algorithm is O(n), where n is size of array


Example:

int [] list = {7, 2, 9, 5, 6, 4};

for (int i=0; i<list.length, i++)

if (value == list[i])

statement; // found it

// didn’t find it


If we assume that the list is ordered, we can still search the entire list from the beginning to the end to determine if we have a match

But, we do not need to search that way

Because the values are in numerical order, we can use a binary search algorithm

Algorithm is O(logn), where n is size of array



The difference in the structure of the data between an unordered list and an ordered list can be used to reduce algorithm Big-O

This is the role of data structures and why we study them

We need to be as clever in organizing our data efficiently as we are in figuring out an algorithm for processing it efficiently


Example:

Find an order for

f(x) = 7 x5 + 5 x3 – x + 4

O(x5)


Sorting Algorithms Big-O:

http://bigocheatsheet.com/

http://bigocheatsheet.com/

9 big o-notation

Education

number of steps

algorithm cont

length i

following algorithm

algorithm efficiencylets

int counts

high complexity

longer time