CSC 212 –Data Structures

Lecture 15:Big-Oh Notation

Question of the Day

Use the numbers two, three, four, and five, one addition operator and one equality operatorWrite mathematical equation (e.g., not in

Java) so equality operator holds

5 + 4 = 32

Analysis Techniques

Running time is important, … … but cannot always compare

Lots of ways to solve a single problemSolutions have lots of implementations

Pseudo-Code

Only for human eyes Ignore unimportant & implementation details Instead use "pseudo-code"

Pseudo-code isn't realUsed for outlining, designing, & analysisNo formal definition or “proper” writingUse a language-like manner

Pseudo-Code

Include important detailsLoops, assignments, method calls, etc.Helps better analyze algorithm

Remember only to understand algorithm Ignore punctuation and formalismsWritten to allow people to understand and

analyze

Pseudo-code Example

int factorial(n, n Z+)returnVariable 1while (n > 0)

returnVariable returnVariable * nn n – 1

return returnVariable

Big-Oh Notation

Computes code complexityWorst-case analysis of performanceRelated to total execution time

Used to compare approachesOnly requires algorithmsNot need to implement everythingAvoids unrelated details

E.g., Compiler, CPU, users’ typing speed

Algorithmic Analysis

1.E+00

1.E+01

1.E+02

1.E+03

1.E+04

1.E+05

1.E+06

1.E+07

1.E+08

1.E+09

2 4 8 16 32 64 128 256 512 1024n

1log nnn^2n^32^n

Algorithm Analysis

Approximate time to run a program with n inputs on 1GHz machine:

n = 10 n = 50 n = 100 n = 1000 n = 106

O(n log n) 35 ns 200 ns 700 ns 10000 ns 20 ms

O(n2) 100 ns 2500 ns 10000 ns 1 ms 17 min

O(n5) 0.1 ms 0.3 s 10.8 s 11.6 days 3x1013 years

O(2n) 1000 ns 13 days 4 x 1014 years Too long! Too long!

O(n!) 4 ms Too long! Too long! Too long! Too long!

Big-Oh Notation

Want results for large data setsWorst case is 2 minutes -- nobody caresOnly consider major details

Ignore multipliersSo, O(5n) = O(2n) = O(n) Multipliers usually implementation-specific

Ignore lesser terms So, O(n5 + n2) = O(n5)Does 17 minutes matter after 3x1013 years?

What is n?

Analysis in respect to size of inputQuestion is what is size of input?

Quick rules of thumb:Analyze values below x n = xAnalyze data in an array n = size of arrayAnalyze linked list n = size of linked listAnalyze 2 arrays n = sum of array sizes

Analyzing an Algorithm

Counts primitive operations executedAssignmentsMethod callsPerforming arithmetic operationComparing two values Indexing into arrayFollowing a referenceReturning a method

Primitive Statements

Run in constant time: O(1)Fastest time possible

Sequences also run in constant timeTrue if sequence not impacted by inputO(5) = O(5 * 1) = O(1)Big-Oh is rough estimate – ignore constant

multipliers

Simple Loops

for (int i = 0; i < n.length; i++) { }

while (i < n) { i++; }

Each loop executed n timesLoops only contain primitive statementsTotal complexity of each loop is: O(n)

More Complicated Loops

for (int i = 0; i < n; i += 2) { }

i assigned 0, 2, 4, 6, ... until larger than n Loop executes n/2 timesLoop only contains primitive statementsTotal complexity of loop is:

O(n/2) = O(½ * n) = O(n)

Even More Complicated Loops

for (int i = 1; i < n; i *= 2) { }

i assigned 1, 2, 4, 8, ... until larger than n Loop executes log2 n timesLoop only contains primitive statementsTotal complexity of loop is:

O(log2 n) = O(log n)

Even More Complicated Loops

for (int i = 1; i < n; i *= 3) { }

i assigned 1, 3, 9, 27, ... until larger than n Loop executes log3 n timesLoop only contains primitive statementsTotal complexity of loop is:

O(log3 n) = O(log 3/log 2 * log n) = O(log n)

Loop Time Complexity

When loop control variable increases:Does not change: takes O(1) or O(∞) timeBy constant value: takes O(n) timeBy constant multiple: takes O(log n) time

Nested Loops

for (int i = 0; i < n; i++) {for (int j = 0; j < n; j++) { }

} Inner loop (j) executes in O(n) timeOuter loop (i) executes n times

But each pass takes O(n) time, not O(1) timeTotal complexity of nested loops:

O(n) * O(n) = O(n2)

Your Turn

Get back into groups and do activity

Before Next Lecture…

Start week #6 assignment Continue lab #5 Programming assignment #2 next week