1

Efficiency of Algorithms:Efficiency of Algorithms:Analyzing algorithm segments,Analyzing algorithm segments,

Insertion sort algorithmInsertion sort algorithm Logarithmic OrdersLogarithmic Orders

Binary Search AlgorithmBinary Search Algorithm

2

Computing an Order Computing an Order of an Algorithm Segmentof an Algorithm Segment

• Consider the following algorithm segment:max:=a[1]; min:=b[1]for i:=2 to n

if max < a[i] then max:=a[i] ; if min > b[i] then min:=b[i]

next iif min ≥ max then med:=(min+max)/2

• Analysis: Number of loop iterations: n-1Comparisons in each iteration: 2

Thus, elementary operations in the loop: 2(n-1) Operations after the loop: 3

Total number of operations in the segment: 2(n-1)+3 = 2n+1 • Order of the segment: O(n)

3

The Insertion Sort AlgorithmThe Insertion Sort Algorithm• Insertion sort arranges the elements of one-dimensional array

a[1], …, a[n] into increasing order.• for i:=2 to n

(insert a[i] into the sorted sequence a[1],…,a[i-1])key:=a[i]for j:=1 to i-1

if key<a[j] then goto (*)next j(*) (shift the values of a[j],…,a[i-1] to a[j+1],…,a[i])

for k:=i to j+1a[k]:=a[k-1]

next ka[j]:=key

next i

4

The Insertion Sort Algorithm:The Insertion Sort Algorithm:ExampleExample

• Arrange array <8, 2, 7, 9, 1, 4> into increasing order.

8 2 7 9 1 4

2 8 7 9 1 4

2 7 8 9 1 4

2 7 8 9 1 4

1 2 7 8 9 4

1 2 4 7 8 9

5

The Insertion Sort Algorithm:The Insertion Sort Algorithm:AnalysisAnalysis

• For each i (outer loop),

maximum number of elementary operations is i – 1 .• Thus, the total number of elementary operations:

1+2+…+(n-1) = n(n-1)/2 = .5n2 - .5n• The insertion sort algorithm is O(n2) .

6

Logarithmic functionLogarithmic function• Definition: The logarithmic function with base b

(b>0, b1)is the following function from R+ to R:

logb(x) = the exponent to which b must raised to obtain x .

Symbolically, logbx = y by = x .

• Property: If the base b>1, then the logarithmic function is increasing:

if x1<x2 , then logb(x1) < logb(x2) .Note: Logarithmic function grows very slowly,

e.g., log2(1,024)=10, log2(1,048,576)=20 .

A property of A property of logarithmic functionlogarithmic function

• Proposition 1: If k is an integer and x is a real number with 2k x <

2k+1, then log2x = k .• Proof:

2k x < 2k+1

log2(2k) log2(x) < log2(2k+1) (since log2x increasing)

k log2(x) < k+1 (by definition of log2x)

log2x = k (by definition of floor function)

■

8

An application of logarithmsAn application of logarithms• Question: Given a positive integer n,

how many binary digits are needed to represent n?• Solution: Binary representation of n: 1ck-1ck-2…c2c1c0

which corresponds to n = 2k + ck-1∙2k-1 + … + c2∙22 + c1∙2 + c0

Since ci 1,n = 2k + ck-1∙2k-1 + … + c2∙22 + c1∙2 + c0

2k + 2k-1 + … + 22 + 2 + 1 = (2k+1-1) / (2-1) (as a sum of geometric sequence) = 2k+1-1 < 2k+1 (1)On the other hand,

n = 2k + ck-1∙2k-1 + … + c2∙22 + c1∙2 + c0 ≥ 2k

(2)Combining (1) and (2): 2k n < 2k+1 (3)

Based on (3) and Proposition 1, k = log2n and the number of binary digits is log2n + 1.

9

Exponential and Logarithmic Exponential and Logarithmic OrdersOrders

For all real numbers b and r with b>1 and r>0

and for all sufficiently large values of x,

logbx xr;

( which implies that logbx is O(xr) )

xr bx

( which implies that xr is O(bx) ) For all real numbers b with b>1

and for all sufficiently large values of x,

x x logbx x2

(which implies that x is O(x logbx) and x logbx is O(x2) )

Binary Search AlgorithmBinary Search Algorithm• The algorithm searches for an element x

in an ascending array of elements a[1],…,a[n] .• Algorithm body:

index:=0, bot:=1, top:=n

while (top ≥ bot and index=0)

mid := (bot+top) / 2if a[mid] = x then index := mid

if a[mid] > x

then top := mid-1

else bot := mid+1

end while

Output: index

(If index has the value 0 then x is not in the array;

otherwise, index gives the index of the array where x is located.)

11

Binary Search Algorithm: Binary Search Algorithm: ExampleExample

• Suppose a[1]=Amy, a[2]=Bob, a[3]=Dave, a[4]=Erin, a[5]=Jeff, a[6]=John, a[7]=Larry, a[8]=Mary, a[9]=Mike, a[10]=Sam, a[11]=Steve, a[12]=Tom.

(sorted in alphabetical order)• Search for x=Erin.• The table tracing the binary search algorithm:

index 0 4

bot 1 1 4

top 12 5 5

mid 6 3 4

12

The Efficiency of The Efficiency of the Binary Search Algorithmthe Binary Search Algorithm

• At each iteration,

the length of the new subarray to be searched

is approximately half of the previous one.• If n = 2k+m, where 0 m < 2k,

then n can be split approximately in half k times.

• Since 2k n < 2k+1, then k = log2n (by proposition 1)

• Thus, the number of iterations of the while loop

in a worst-case execution of the algorithm is log2n+1.

• The number of operations in each loop is constant

(doesn’t increase with n).

• Thus, the binary search algorithm is O(log2n) .