1

CS103Unit8bSlides

Algorithms

MarkRedekopp

2

ALGORITHMS

3

HowDoYouFindaWordinaDictionary

• Describean“efficient”method• Assumptions/Guidelines

– Lettarget_word =wordtolookup– Npagesinthedictionary– Eachpagehasthestart andlast wordonthatpagelistedatthetopofthepage

– Assumetheuserunderstandshowtoperformalphabetical(“lexicographic”)comparison(e.g.“abc”issmallerthan“acb”or“abcd”)

apple arc

45

4

Algorithms• Algorithmsareattheheartofcomputer

systems,bothinHWandSW– TheyarefundamentaltoComputerScience

andComputerEngineering• Informaldefinition

– Analgorithmisaprecisewaytoaccomplishataskorsolveaproblem

• Softwareprogramsarecollectionsofalgorithmstoperformdesiredtasks

• Hardwarecomponentsalsoimplementalgorithmsfromsimpletocomplex Hardware Software

5

HumansandComputers

• Humansunderstandalgorithmsdifferentlythancomputers

• Humanseasilytolerateambiguityandabstractconceptsusingcontexttohelp.– “Addapinchofsalt.”Howmuchisapinch?– “MichaelJordancouldsoarlikeaneagle.”– “It’sabearmarket”

• Computersonlyexecutewell-definedinstructions(noambiguity)andoperateondigitalinformationwhichisdefiniteanddiscrete(everythingisexactandnot“closeto”)

6

FormalDefinition

• Foracomputer,“algorithm”isdefinedas...– …anorderedsetofunambiguous,executablestepsthatdefinesaterminatingprocess

• Explanation:– OrderedSteps:thestepsofanalgorithmhaveaparticularorder,notjustanyorder

– Unambiguous:eachstepiscompletelyclearastowhatistobedone

– Executable:Eachstepcanactuallybeperformed– TerminatingProcess:Algorithmwillstop,eventually.(sometimesthisrequirementisrelaxed)

7

AlgorithmRepresentation

• Analgorithmisnotaprogramorprogramminglanguage

• Justasastorymayberepresentedasabook,movie,orspokenbyastory-teller,analgorithmmayberepresentedinmanyways– Flowchart– Pseudocode (English-likesyntaxusingprimitivesthatmostprogramminglanguageswouldhave)

– Aspecificprogramimplementationinagivenprogramminglanguage

8

AlgorithmExample1• List/printallfactorsofanaturalnumber,n

– Howwouldyoucheckifanumberisafactorofn?– Whatistherangeofpossiblefactors?i←1while(i<=n)doif (remainderofn/iiszero)thenListiasafactorofn

i←i+1

• Animprovementi←1while(i<=sqrt(n))doif (remainderofn/iiszero)thenListiandn/iasafactorofn

i←i+1

TF

9

AlgorithmTimeComplexity

• Weoftenjudgealgorithmsbyhowlongtheytaketorunforagiveninputsize

• Algorithmsoftenhavedifferentrun-timesbasedontheinputsize[e.g.#ofelementsinalisttosearchorsort]– Differentinputpatternscanleadtobestandworstcasetimes

– Average-casetimescanbehelpful,butweusuallyuseworstcasetimesforcomparisonpurposes

10

Big-ONotation• Givenaninputtoanalgorithmofsizen,wecanderiveanexpressionin

termsofnforitsworstcaseruntime(i.e.thenumberofstepsitmustperformtocomplete)

• Fromtheexpressionwelookforthedominanttermandsaythatisthebig-O(worst-caseorupper-bound)run-time– Ifanalgorithmwithinputsizeofnrunsinn2 +10n+1000steps,wesaythatit

runsinO(n2)becauseifnislargen2 willdominatetheotherterms

i ← 1while(i <= n) do

if (remainder of n/i is zero) thenList i as a factor of n

i ← i+1

11*n2*n

5n+1= O(n)

1*n1*n

11

Big-ONotation• Givenaninputtoanalgorithmofsizen,wecanderiveanexpressionin

termsofnforitsworstcaseruntime(i.e.thenumberofstepsitmustperformtocomplete)

• Fromtheexpressionwelookforthedominanttermandsaythatisthebig-O(worst-caseorupper-bound)run-time– Ifanalgorithmwithinputsizeofnrunsinn2 +10n+1000steps,wesaythatit

runsinO(n2)becauseifnislargen2 willdominatetheotherterms• Mainsourcesofrun-time:Loops

– Evenworse:Loopswithinloops(i.e.executeallofloop2w/inasingleiterationofloop1,andrepeatforalliterationsofloop1,etc.)

i ← 1while(i <= n) do

if (remainder of n/i is zero) thenList i as a factor of n

i ← i+1

11*n2*n

5n+1= O(n)

1*n1*n

12

AlgorithmExample1• List/printallfactorsofanaturalnumber,n

– Whatisafactor?– Whatistherangeofpossiblefactors?i←1while(i<=n)doif (remainderofn/iiszero)thenListiasafactorofni←i+1

• Animprovementi←1while(i<=sqrt(n))doif (remainderofn/iiszero)thenListiandn/iasafactorofni←i+1

O(n)

13

AlgorithmExample2a• Searchinganorderedlist(array)fora

specificvalue,k,andreturnitsindexor-1ifitisnotinthelist

• SequentialSearch– Startatfirstitem,checkifitisequaltok,repeat

forsecond,third,fourthitem,etc.

i ←0while (i <length(myList))doif (myList[i]equaltok)thenreturn ielsei ←i+1

return -1

2 3 4 6 9 10 13 15 19myListindex 1 2 3 4 5 6 7 80

14

AlgorithmExample2b

• Sequentialsearchdoesnottakeadvantageoftheorderednatureofthelist– Wouldworkthesame(equallywell)onan

orderedorunorderedlist

• BinarySearch– Takeadvantageoforderedlistbycomparingk

withmiddleelementandbasedontheresult,ruleoutallnumbersgreaterorsmaller,repeatwithmiddleelementofremaininglist,etc.

2 3 4 6 9 10 13 15 19Listindex

6 < 9

k = 6

Start in middle

2 3 4 6 9 10 13 15 19Listindex

6 > 4

6 9 10 13 15 19Listindex

6 = 6

2 3 4

1 2 3 4 5 6 7 80

1 2 3 4 5 6 7 80

1 2 3 4 5 6 7 80

15

AlgorithmExample2b• BinarySearch

– Comparekwithmiddleelementoflistandifnotequal,ruleout½ofthelistandrepeatontheotherhalf

– Implementation:• Definerangeofsearchableelements=[start,end)• (i.e.startisinclusive,endisexclusive)

start←0;end←length(List);while (startindexnotequaltoendindex)doi ←(start+end)/2;if (k==List[i])then returni;elseif (k>List[i])then start←i+1;else end←i;return-1;

2 3 4 6 9 11 13 15 19Listindex

2 3 4 6 9 11 13 15 19Listindex

i

k = 11

endstart

i endstart

2 3 4 6 9 11 13 15 19Listindex

endstart i

1 2 3 4 5 6 7 80

1 2 3 4 5 6 7 80

1 2 3 4 5 6 7 80

2 3 4 6 9 11 15 19Listindex

endstarti

1 2 3 4 5 6 7 8013

16

Sorting• Ifwehaveanunorderedlist,sequential

searchbecomesouronlychoice• Ifwewillperformalotofsearchesitmay

bebeneficialtosortthelist,thenusebinarysearch

• Manysortingalgorithmsofdifferingcomplexity(i.e.fasterorslower)

• BubbleSort(simplethoughnotterriblyefficient)– Oneachpassthroughthruthelist,pickupthe

maximumelementandplaceitattheendofthelist.Thenrepeatusingalistofsizen-1(i.e.w/othenewlyplacedmaximumvalue)

7 3 8 6 5 1Listindex

Original1 2 3 4 50

3 7 6 5 1 8Listindex

After Pass 11 2 3 4 50

3 6 5 1 7 8Listindex

After Pass 21 2 3 4 50

3 5 1 6 7 8Listindex

After Pass 31 2 3 4 50

3 1 5 6 7 8Listindex

After Pass 41 2 3 4 50

1 3 5 6 7 8Listindex

After Pass 51 2 3 4 50

17

BubbleSortAlgorithm

7 3 8 6 5 1j i

Pass 1

3 7 8 6 5 1j i

3 7 8 6 5 1j i

3 7 6 8 5 1j i

3 7 6 5 8 1j

3 7 6 5 1 8

swap

no swap

swap

swap

swap

j i

Pass 2

3 7 6 5 1 8j i

3 6 7 5 1 8j i

3 6 5 7 1 8

3 6 5 1 7 8

j

no swap

swap

swap

swap

3 7 6 5 1 8i

Pass n-23 1 5 6 7 8

1 3 5 6 7 8 swap

…

void bsort(int* mylist, int n){int i ;for(i=n-1; i > 0; i--){

for(j=0; j < i; j++){if(mylist[j] > mylist[j+1]) {

swap(j, j+1)} } }

}

i

i

j

18

ComplexityofSearchAlgorithms

• SequentialSearch:Listoflengthn– Worstcase:Searchthroughentirelist– Timecomplexity=an+k

• aissomeconstantfornumberofoperationsweperformintheloopasweiterate

• kissomeconstantrepresentingstartup/finishwork(outsidetheloop)

– SequentialSearch=O(n)

• BinarySearch:Listoflengthn– Worstcase:Continuallydividelistintwo

untilwereachsublistofsize1– Time=a*log2n+k=O(log2n)

• Asngetslarge,binarysearchisfarmoreefficientthansequentialsearch

Dividing by 2 k-times yields:

n / 2k = 1k = log2n

Multiplying by 2 k-times yields: 2*2*2…*2 = 2k

0 5 10 15 20 25 30 35 40 45 500

5

10

15

20

25

30

35

40

45

50

N

Run-tim

e

19

ComplexityofSortAlgorithms

• BubbleSort– 2NestedLoops– Executeouterloopn-1times– Foreachouterloopiteration,

innerlooprunsitimes.– Timecomplexityisproportional

to:n-1+n-2+n-3+…+1=(n2 +n)/2=O(n2)

• OthersortalgorithmscanruninO(n*log2n)

0 2 4 6 8 10 12 14 16 18 200

50

100

150

200

250

300

350

400

N

Run-

time

NN2

N*log2(N)

20

ImportanceofTimeComplexity

N O(1) O(log2n) O(n) O(n*log2n) O(n2) O(2n)

2 1 1 2 2 4 4

20 1 4.3 20 86.4 400 1,048,576

200 1 7.6 200 1,528.8 40,000 1.60694E+60

2000 1 11.0 2000 21,931.6 4,000,000 #NUM!

• Itmakesthedifferencebetweeneffectiveandimpossible• Manyimportantproblemscurrentlycanonlybesolvedwithexponentialrun-time

algorithms(e.g.O(2n)time)…wecalltheseNP=Non-deterministicpolynomialtimealgorithms)[Noknownpolynomial-timealgorithmexists]

• UsuallyalgorithmsareonlypracticaliftheyruninP=polynomialtime(e.g.O(n)orO(n2)etc.)

• OneofthemostpressingopenproblemsinCS:“IsNP=P?”– DoPalgorithmsexistfortheproblemsthatwecurrentlyonlyhaveanNPsolutionfor?