4: Sorting andComplexity

Today• Insertion sort (again)• Common complexity• Other sorting algorithms‣Merge sort‣Quick sort

• Project 2 – Comparison of sorting algorithms

2

Last Week• Sorting Problem‣Order data in sequence

• Insertion Sort‣ Insert value one at a time

3

[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

Sort

Insertion Sort – Complexity

• Time complexity‣How long time does it take?‣How much longer if the input data size doubles?‣Can we answer without/before executing?

• Not real time Number of test/swap operations4

def insert_sort_inplace(a):for i in range(len(a)):

for j in range(i-1, -1, -1):if a[j] > a[j+1]:

swap(a, j, j+1)return a

Insertion Sort – Complexity

• 𝒏𝒏 = input size; here len(a)• #𝐭𝐭𝐭𝐭𝐭𝐭𝐭𝐭𝐭𝐭 = 𝟏𝟏 + 𝟐𝟐 + ⋯+ 𝒏𝒏 − 𝟏𝟏

= �𝒊𝒊=𝟏𝟏

𝒏𝒏−𝟏𝟏

𝒊𝒊 =𝒏𝒏(𝒏𝒏 − 𝟏𝟏)

𝟐𝟐=𝒏𝒏𝟐𝟐

𝟐𝟐−𝒏𝒏

𝟐𝟐5

def insert_sort_inplace(a):for i in range(len(a)):

for j in range(i-1, -1, -1):if a[j] > a[j+1]:

swap(a, j, j+1)return a

= 𝑂𝑂(𝑛𝑛2)

Insertion Sort – Optimized

• Better algorithm• Fewer tests• But still same

complexity 𝑂𝑂 𝑛𝑛2

6

def insert_sort_inplace(a):for i in range(len(a)):

for j in range(i-1, -1, -1):if a[j] > a[j+1]:

swap(a, j, j+1)return a

def insert_sort_inplace_optimized(a):for i in range(len(a)):

for j in range(i-1, -1, -1):if a[j] > a[j+1]:

swap(a, j, j+1)else:

breakreturn a

Common Complexity

Classes

7

Common Complexity Classes

8

constant 定数時間

logarithmic 対数

linear 線形関数

linearithmic,loglinear 準線形、線形対数

quadratic 二乗時間

cubic 三乗時間

Common Complexity Classes

9

input size (n)

com

puta

tion

time

More Complexity Classes

10

input size (n)

com

puta

tion

time

More Complexity Classes

11input size (n)

com

puta

tion

time

Merge SortWikipedia

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Merge Sort

13

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

SORT SORT

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

MERGE

Merge Sort

14

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

0 1 2 3 4 5 6 7

0 1 2 3

0 1 2 3

4 5 6 7

4 5 6 7

8 9 10 11 12 13 14 15

8 9 10 11

8 9 10 11

12 13 14 15

12 13 14 15

QuicksortWikipedia

Quicksort

16

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

pivot1 2 3 4 5 67 8 9 10 11 12 13 14 15

1 2 3 4 5 6 7 8 9 10 11 12 13 14 150

pivot

SORT1 2 3 4 5 6

pivot

SORT7 8 9 10 11 12 13 14 15

7

Pivotvalue

1 2 3 4 5 6 8 9 10 11 12 13 14 15 16

Summary• Sort Algorithms‣ Insertion sort‣Merge sort‣Quicksort

• Complexity‣Depends on problem size‣Count number of steps:(assignment / test / …)‣Asymptotic order

• Divide-and-conquer‣Solve smaller problems and then combine the results (link)‣Merge sort algorithm ‣Binary search algo (link)

17

Project 2Comparison of

sorting algorithms

Project 2 – Objectives• Goals:‣Compare some sorting algorithms‣See/understand a more complex program‣Obtain nice curves and write a report

•What to do:‣ Implement merge sort‣Evaluate three sorting algorithms (insertion, merge, quick)‣Write a report (2-3 pages) including graphic

19

Project 2 – Evaluation (25pts)• Implement merge sort (~5pts)• Obtain curves comparing sorting algos (~5pts)‣ Use the provided code test_sort.py.‣ You may change some parts (e.g. REPETITIONS, TEST_SIZES, …)

• Write the report (~10pts)• Add something personal (~5pts)‣ Other algorithm (bubble sort, selection sort, …)‣ Improve/simplify the given code‣ …

• Deadline: End of January (exact date TBD) 20

+ some bonus pointsfor great solutions,

and/or nice reports