8/3/2019 More Bi Go

1/25

More Big-O Notation,

Sorting Algorithms, Etc.December 1, 2009

8/3/2019 More Bi Go

2/25

Sorting Algorithms

Sorting: putting a list of items in-order

Usually: numerical or lexicographical

Long time studied array ofalgorithms/computer science

Silver Bullet?

Simple

Efficient

8/3/2019 More Bi Go

3/25

From Last Time.

Big-O Notation

Think: How much work does this algorithmneed to do?

A way to describe the efficiency of asearch

Look at the worst case analysis of an

algorithm

8/3/2019 More Bi Go

4/25

From Last Time

Suggestions for determining Big-O Start by looking at the number of nested

loops

Look for recursion Remember:

Length of a list (which could be stored inan Array, ArrayList, etc) is usuallyrepresented by the lowercase letter n What if I know the number of elements?

Still do your analysis in terms of n.

8/3/2019 More Bi Go

5/25

Why Do We Care?

What is Analyzing Algorithms (and Big-O notation) good for?

Comparison of algorithms

Advertising a new algorithm

Making decisions on which algorithm touse

Determining the scalability of an algorithm

8/3/2019 More Bi Go

6/25

Common Types

O(1) Constant

O(log n) Logarithmic

O(n) Linear

O(n2) Quadratic

O(n3) Cubic

8/3/2019 More Bi Go

7/25

Random Thought

It is important when reporting the Big-O to useproper Big-O Notation

If the Big-O of an algorithm is linear,

Then you should report the Big-O as O(n)

You must include the letter O and the ()

Remember: n is not the same as O(n)

Why?

It is the notation that has always been used

There are other symbols that are used to mean slightlydifferent thingsbut we will not get into that in this course

Plus, I am sure you want to get it right on your final exam

8/3/2019 More Bi Go

8/25

What is the Big-O of:

int n = 20;

double[] numbers = new double[n];

numbers = fillArray();

int sum = 0;

for(int x=0; x

8/3/2019 More Bi Go

9/25

And the Big-O is:

First look at the loop:

for(int x=0; x

8/3/2019 More Bi Go

10/25

What is the Big-O of:

int n = 20;

double[] numbers = new double[n];

numbers = fillArray();

int sum = 0;for(int x=0; x

8/3/2019 More Bi Go

11/25

And the Big-O is:

What is the difference between thisnew code and the old code?

The array of numbers is gone through

completely twice.

There are n elements in the array

Therefore, 2 * n amount of work.

What is the Big-O? Big-O = O(2n) = O(n)

Coefficients are dropped

8/3/2019 More Bi Go

12/25

More on Big-O Notation

Coefficients are dropped:

You would not report a Big-O of O(2n).Instead, you would say it was O(n)

Why? We want the order of the

algorithm for comparison purposes

8/3/2019 More Bi Go

13/25

More on Big-O Notation

The largest terms dominate:

If you calculated that an algorithm took n2+ 3n + 1 of work, the proper and correct

Big-O notation is O(n2)

The O(n2) is the dominate term and the 3n+ 1 part is considered negligible

8/3/2019 More Bi Go

14/25

Stable Sort

A stable sort is a sort algorithm thatmaintains the relative order of data items inthe list

In other words, it there are multiple dataitems in the list that have the same value,then the sort completes these data items arein the same order as the original

8, 1, 5, 2, 3, 3, 4, 3, 6, 7

1, 2, 3, 3, 3, 4, 5, 6, 7, 8

8/3/2019 More Bi Go

15/25

Comparison Sorts

Sort algorithms that mainly usecomparison operators (e.g., lessthan/greater than)

Common Comparison Sorts: Insertion Sort

Selection Sort

Merge Sort Bubble Sort

Quick Sort

8/3/2019 More Bi Go

16/25

Bubble Sort

Another simple comparison sorting algorithm

The basics:

Repeatedly walkthrough a list of data that needs

to be sorted compare adjacent data items andswapping (if needed)

Algorithm stops: when a walkthrough of the listdoes not require any swaps

Naming: Elements bubble there way to thetop

(I dont name, I just teach them)

8/3/2019 More Bi Go

17/25

Bubble Sort

Example: Start: 5, 1, 3, 6, 4

1, 5, 3, 6, 4

1, 3, 5, 6, 4

1, 3, 5, 6, 4 (no change) 1, 3, 5, 4, 6

1, 3, 5, 4, 6 (no change)

1, 3, 5, 4, 6 (no change)

1, 3, 4, 5, 6

1, 3, 4, 5, 6 (no change)

It will check the array 1 last time and there will beno swaps, so the algorithm is done

8/3/2019 More Bi Go

18/25

Bubble Sort Pseudocode

Given: array A of data to be sorted

Do{

IsSwap = false;

for(int x=0; x A[x+1]) {

swap(A[x], A[x+1]);

IsSwap = true;

}//end if

}//end for loop

}While(IsSwap == true)

8/3/2019 More Bi Go

19/25

Big-O For Bubble Sort

How many loops? Two nested loops

For Loop: n-1 times

Do While Loop: Until swaps stop - but when is that? Remember we are dealing with worst case!

What is the worst case?

987654321

Exact opposite of sorted

Therefore, n, number of swaps

The Do While controls the number of times the For Loop

happens. Therefore, worst case, amount of work for thealgorithm: n * (n-1)

n * (n-1) = n2 - n

Therefore, O(n2)

8/3/2019 More Bi Go

20/25

Advantages of Bubble Sort

This algorithm can easily identify anylist of data items already in the properorder

Surprisingly, not all sort algorithms arewritten with this feature in mind One of the reasons there is yet another O(n2)

algorithm - each is designed slightly differently

This algorithm would also work well fordata that is close to being in the properorder

8/3/2019 More Bi Go

21/25

Quick Sort

A comparison sorting algorithm using adivide and conquer technique

The basics:

Pick an element to be the pivot

Partition Operation

Create sub-lists for above and below pivot

Place pivot between sub-lists

Recursively sort each sub-lists

8/3/2019 More Bi Go

22/25

Quick Sort Example

Original List: 3, 9, 8, 5, 2, 1, 7, 6, 4

Select 4 as pivot

Sub-list Less: 3, 2, 1

Sub-list More: 9, 8,5, 7, 6

Sort Sub-list Less: 1, 2, 3

Sort Sub-list More: 5, 6, 7, 8, 9

8/3/2019 More Bi Go

23/25

Quick Sort Pseudocode

Given:

ArrayList A of data to be sorted

ArrayList SubLess and ArrayList SubMore

QuickSort(A) {

Select pivot point (I used: pivot = A.Size-1)

for(int x=0; x

8/3/2019 More Bi Go

24/25

Big-O For Quick Sort

Loops? One For Loop: n-1

Recursion? Yes: QuickSort both ArrayListsSubLess and SubMore

Worst case: pick the pivot that only creates sub-lists of size 1 and n-1 This cases the # of recursive calls to be n-1 number of

times

Total number of calls to Quicksort: n-1 + 1 = n

Total work = n * (n-1) = n2

- n O(n2)

Just FYI: average case: n log n Why? Average tree depth: log n

8/3/2019 More Bi Go

25/25

Other Types of Complexity

Average Complexity

Use random inputs

Studying what happens most of the time

Memory Complexity

Studying how much memory is neededbeyond the list of items to complete the

search Most common sorts have O(1) memory

complexity (space for swaps, etc)