CS 121Week 12 - Wednesday

Last time

What did we talk about last time? Hunters and prey Class variables Big Oh notation

Questions?

Project 4

Big Oh Notation

Mathematical description

We want to compare the running time of one program to another

We want a mathematical description with the following characteristics: Worst case

We care mostly about how bad things could be

AsymptoticWe focus on the behavior as the input size gets larger and larger

Big Oh notation

Enter Big Oh notation Big Oh simplifies a complicated running

time function into a simple statement about its worst case growth rate

All constant coefficients are ignored All low order terms are ignored 3n + 3 is O(n) Big Oh is a statement that a particular

running time is no worse than some function, with appropriate constants

Big Oh simplification examples 147n3 + 2n2 + 5n + 12083 is

O(n3)

n1000 + 2n isO(2n)

15n2 + 6n + 7log n + 145 isO(n2)

659n + nlog n + 87829 isO(n log n)

Note: In CS, we use log2 unless stated otherwise

Big Oh Examples

Mulitiplication by hand

How long does it take to do multiplication by hand?

123

x 456

738

615

492__

56088

Let’s assume that the length of the numbers is n digits (n multiplications + n carries) x n digits + (n + 1

digits) x n additions Running time: O(n2)

Finding the largest element in an array

How do we find the largest element in an array?

Running time: O(n) if n is the length of the array What if the array is sorted in ascending order?

Running time: O(1)

int largest = array[0];

for( int i = 1; i < array.length; i++ )if( array[i] > largest )

largest = array[i];

System.out.println("Largest: " + largest);

System.out.println("Largest: " + array[array.length-1]);

Bubble sort

Here is some code that sorts an array in ascending order

What is its running time?

Running time: O(n2)

for( int i = 0; i < array.length; i++ )for( int j = 0; j < array.length - 1; j++ )

if( array[j] > array[j + 1] ){

int temp = array[j];array[j] = array[j + 1];

array[j + 1] = temp; }

Comparison of Big Oh Classes

Hierarchy of complexities Here is a table of several different complexity

measures, in ascending order, with their functions evaluated at n = 100

Description Big Oh f(100)

Constant O(1) 1

Logarithmic O(log n) 6.64

Linear O(n) 100

Linearithmic O(n log n) 664.39

Quadratic O(n2) 10000

Cubic O(n3) 1000000

Exponential O(2n) 1.27 x 1030

Factorial O(n!) 9.33 x 10157

Practical implications

Computers get faster, but not in unlimited ways If computers get 10 times faster, here is how much a

problem from each class could grow and still be solvable

Description Big Oh Increase in Size

Constant O(1) Unlimited

Logarithmic O(log n) 1000

Linear O(n) 10

Linearithmic O(n log n) 10

Quadratic O(n2) 3-4

Cubic O(n3) 2-3

Exponential O(2n) Hardly changes

Factorial O(n!) Hardly changes

Rules of thumb

There is nothing better than constant time Logarithmic time means that the problem can

become much larger and only take a little longer Linear time means that time grows with the problem Linearithmic time is just a little worse than linear Quadratic time means that expanding the problem

size significantly could make it impractical Cubic time is about the reasonable maximum if we

expect the problem to grow Exponential and factorial time mean that we cannot

solve anything but the most trivial problem instances

Memory Usage

Memory usage

Memory usage can be a problem If you run out of memory, your

program can crash Memory usage can have serious

performance consequences too

Memory

Remember, there are multiple levels of memory on a computer

Each next level is on the order of 500 times larger and 500 times slower

Cache•Actually on the CPU•Fast and expensive

RAM•Primary memory for a desktop computer•Pretty fast and relatively expensive

Hard Drive•Secondary memory for a desktop computer•Slow and cheap

1000X 1000X

1000X 1000X

Size Speed

Memory speed

If you can do a lot of number crunching without leaving cache, that will be very fast

If you have to fetch data from RAM, that will slow things down

If you have to read and write data to the hard drive (unavoidable with large pieces of data like digital movies), you will slow things down a lot

Measuring memory

Memory can be easier to estimate than running time

Depending on your input, you will allocate a certain number of objects, arrays, and primitive data types

It is possible to count the storage for each item allocated

Remember that a reference to an object or an array costs an additional 4 bytes on top of the size of the object

Memory footprints

Here are the sizes of various types in Java

Note that the N refers to the number of elements in the array or String

Type Bytes

boolean 1

char 2

int 4

double 8

Type Bytes

boolean[] 16 + N

char[] 16 + 2N

int[] 16 + 4N

double[] 16 + 8N

Type Bytes

reference 4

String 40 + 2N

object 8 + size of members

array of objects 16 + (4 + size of members)N

Searching

Searching for a number

Lets say that I give you a list of numbers, and I ask you, “Is 37 on this list?”

As a human, you have no problem answering this question, as long as the list is reasonably short

What if the list is an array, and I want you to write a Java program to find some number?

Search algorithm

Easy! We just look through every element in

the array until we find it or run out

If we find it, we return the index, otherwise we return -1

public static int find( int[] array, int number ) {for( int i = 0; i < array.length; i++ )

if( array[i] == number )return i;

return -1;}

How long does it take?

Unfortunately for you, we know about Big Oh notation

Now we have some way to measure how long this algorithm takes

How long, if n is the length of the array?

O(n) time because we have to look through every element in the array, in the worst case

Can we do better?

Is there any way to go smaller than O(n)?

What complexity classes even exist that are smaller than O(n)? O(1) O(log n)

Well, on average, we only need to check half the numbers, that’s ½ n which is still O(n)

Darn…

Binary Search

We can’t do better unless…

We can do better with more information

For example, if the list is sorted, then we can use that information somehow

How? We can play a High-Low game

Binary search

Repeatedly divide the search space in half

We’re looking for 37, let’s say

5423 31

Check the middle(Too high)

Check the middle(Too low)

Check the middle(Too low)

Check the middle(Found it!)

37

So, is that faster than linear search?

How long can it take? What if you never find what you’re

looking for? Well, then, you’ve narrowed it down

to a single spot in the array that doesn’t have what you want

And what’s the maximum amount of time that could have taken?

Running time for binary search

We cut the search space in half every time

At worst, we keep cutting n in half until we get 1

Let’s say x is the number of times we look:

The running time is O(log n)

Binary Search Examples

Guessing game

We can apply this idea to a guessing game

First we tell the computer that we are going to pick a number between 1 and n

We pick, and it tries to narrow down the number

It should only take log n tries Remember log2(1,000,000) is only

about 20

Interview question

This is a classic interview question asked by Microsoft, Amazon, and similar companies

Imagine that you have 9 red balls One of them is just slightly heavier

than the others, but so slightly that you can’t feel it

You have a very accurate two pan balance you can use to compare balls

Find the heaviest ball in the smallest number of weighings

What’s the smallest possible number?

It’s got to be 8 or fewer We could easily test one ball against

every other ball

There must be some cleverer way to divide them up

Something that is related somehow to binary search

That’s it!

We can divide the balls in half each time

If those all balance, it must be the one we left out to begin with

Nope, we can do better

How? They key is that you can actually cut the number of balls

into three parts each time

We weigh 3 against 3, if they balance, then we know the 3 left out have the heavy ball

When it’s down to 3, weigh 1 against 1, again knowing that it’s the one left out that’s heavy if they balance

Thinking outside the box, er, ball

The cool thing is… Yes, this is “cool” in the CS sense, not in

the real sense Anyway, the cool thing is that we are

trisecting the search space each time

This means that it takes log3 n weighings to find the heaviest ball

We could do 27 balls in 3 weighings, 81 balls in 4 weighings, etc.

Quiz

Upcoming

Next time…

Sorting Lab 12

Reminders

Finish Project 4 Due Friday before midnight