lecture 3 - stanford...
TRANSCRIPT
![Page 1: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/1.jpg)
Lecture3RecurrenceRelationsandtheMasterTheorem!
![Page 2: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/2.jpg)
Announcements!
• HW1isposted!
• DueFriday.
• Sections willbeTuesdays4:30-5:20,room380-381U!
• Theyareoptional.
• Butvaluable!
• SignupforPiazza!
• There’salinkonthecoursewebsite.
• CourseannouncementswillbepostedonPiazza.
![Page 3: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/3.jpg)
Moreannouncements
![Page 4: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/4.jpg)
Lasttime….
• Sorting:InsertionSort andMergeSort
• Analyzingcorrectnessofiterative+recursivealgs
• Via“loopinvariant”andinduction
• Analyzingrunningtimeofrecursivealgorithms
• Bywritingoutatreeandaddingupalltheworkdone.
• Howdowemeasuretheruntimeofanalgorithm?
• Worst-CaseAnalysis
• Big-OhNotation
![Page 5: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/5.jpg)
Today
•RecurrenceRelations!• Howdowemeasuretheruntimearecursivealgorithm?
• LikeIntegerMultiplicationandMergeSort?
• TheMasterMethod• Ausefultheoremsowedon’thavetoanswerthis
questionfromscratcheachtime.
![Page 6: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/6.jpg)
RunningtimeofMergeSort
• Let’scallthisrunningtimeT(n).• whentheinputhaslengthn.
• WeknowthatT(n)=O(nlog(n)).
• Butifwedidn’tknowthat…MERGESORT(A):
n=length(A)
if n≤ 1:
return A
L=MERGESORT(A[:n/2])
R=MERGESORT(A[n/2:])
returnMERGE(L,R)
𝑇 𝑛 ≤ 2 ⋅ 𝑇 𝑛2 + 11 ⋅ 𝑛
Fromlasttime
![Page 7: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/7.jpg)
RecurrenceRelations
• 𝑇 𝑛 = 2 ⋅ 𝑇 *+ + 11 ⋅ 𝑛 isarecurrencerelation.
• ItgivesusaformulaforT(n) intermsofT(lessthann)
• Thechallenge:
GivenarecurrencerelationforT(n),findaclosedformexpressionforT(n).
• Forexample,T(n)=O(nlog(n))
![Page 8: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/8.jpg)
TechnicalitiesIBaseCases
• Formally,weshouldalwayshavebasecaseswithrecurrencerelations.
• 𝑇 𝑛 = 2 ⋅ 𝑇 *+ + 11 ⋅ 𝑛with𝑇 1 = 1
isnotthesameas
• 𝑇 𝑛 = 2 ⋅ 𝑇 *+ + 11 ⋅ 𝑛with𝑇 1 = 1000000000
• However,T(1)=O(1),sosometimeswe’lljustomitit.
WhydoesT(1)=O(1)?
Siggi theStudiousStork
Pluckythe
PedanticPenguin
![Page 9: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/9.jpg)
Onyourpre-lectureexercise
• Youplayedaroundwiththeseexamples(whennisapowerof2):
1. 𝑇 𝑛 = 𝑇 *+ + 𝑛, 𝑇 1 = 1
2. 𝑇 𝑛 = 2 ⋅ 𝑇 *+ + 𝑛, 𝑇 1 = 1
3. 𝑇 𝑛 = 4 ⋅ 𝑇 *+ + 𝑛, 𝑇 1 = 1
• Whataretheclosedforms?
[onboard]
![Page 10: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/10.jpg)
Oneapproachforallofthese
Sizen
n/2n/2
n/4
(Size1)
…
n/4n/4n/4
n/2tn/2tn/2tn/2tn/2tn/2t
…
• The“tree”approach
fromlasttime.
• Addupallthework
doneatallthesub-
problems.
![Page 11: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/11.jpg)
Anotherapproach:
Recursivelyapplytherelationshipabunchuntilyouseeapattern.
• 𝑇 𝑛 = 2 ⋅ 𝑇 *+ + 11 ⋅ 𝑛
• 𝑇 𝑛 = 2 ⋅ 2 ⋅ 𝑇 *1 + 22⋅*
+ + 11 ⋅ 𝑛• 𝑇 𝑛 = 4 ⋅ 𝑇 *
1 + 22 ⋅ 𝑛
• 𝑇 𝑛 = 4 ⋅ 2 ⋅ 𝑇 *3 + 22⋅*
1 + 22 ⋅ 𝑛• 𝑇 𝑛 = 8 ⋅ 𝑇 *
3 + 33 ⋅ 𝑛
• Followingthepattern…
• 𝑻 𝒏 = 𝒏 ⋅ 𝑻 𝟏 + 𝟏𝟏 ⋅ 𝒍𝒐𝒈 𝒏 ⋅ 𝒏 = 𝑶 𝒏 ⋅ 𝐥𝐨𝐠 𝒏
Formally,thisshould
beaccompanied
withaproofthat
thepatternholds!
Morenexttime.
Thisslideskippedinclass,providedhereincase
thiswaymakesmoresensetoyou.
![Page 12: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/12.jpg)
Moreexamples
• Needlesslyrecursiveintegermultiplication
• T(n)=4T(n/2)+O(n)
• T(n)=O(n2)
• Karatsubaintegermultiplication
• T(n)=3T(n/2)+O(n)
• T(n)=O(𝑛?@AB C ≈ n1.6)
• MergeSort
• T(n)=2T(n/2)+O(n)
• T(n)=O(nlog(n))
T(n)=timetosolveaproblemofsizen.
What’sthepattern?!?!?!?!
Thesetwoare
thesameasthe
onesonyour
pre-lecture
exercise.
![Page 13: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/13.jpg)
Themastertheorem
• A formulathatsolvesrecurrenceswhenallofthesub-problemsarethesamesize.
• We’llseeanexampleWednesdaywhennotallproblemsarethesamesize.
• ”Generalized”treemethod.
JedimasterYoda
A useful
formula it is.
Know why it works
you should.
![Page 14: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/14.jpg)
Themastertheorem
• Suppose𝑇 𝑛 = 𝑎 ⋅ 𝑇 *F + 𝑂 𝑛H .Then
Many symbols
those are….
Threeparameters:
a:numberofsubproblems
b:factorbywhichinputsizeshrinks
d:needtodond worktocreateallthe
subproblems andcombinetheirsolutions.
Wecanalsotaken/bto
meaneither*F or
*F and
thetheoremisstilltrue.
![Page 15: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/15.jpg)
TechnicalitiesIIIntegerdivision
• Ifnisodd,Ican’tbreakitupintotwoproblemsofsizen/2.
• However(seeCLRS,Section4.6.2),onecanshowthattheMastertheoremworksfineifyoupretendthatwhatyouhaveis:
• Fromnowonwe’llmostlyignorefloorsandceilingsinrecurrencerelations.
𝑇 𝑛 = 𝑇 𝑛2 + 𝑇 𝑛
2 + 𝑂(𝑛)
𝑇 𝑛 = 2 ⋅ 𝑇 𝑛2 + 𝑂(𝑛)
Pluckythe
PedanticPenguin
![Page 16: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/16.jpg)
Examples(detailsonboard)
• Needlesslyrecursiveintegermult.
• T(n)=4T(n/2)+O(n)
• T(n)=O(n2)
• Karatsubaintegermultiplication
• T(n)=3T(n/2)+O(n)
• T(n)=O(nlog_2(3) ≈ n1.6)
• MergeSort
• T(n)=2T(n/2)+O(n)
• T(n)=O(nlog(n))
• Thatotherone
• T(n)=T(n/2)+O(n)
• T(n)=O(n)
𝑇 𝑛 = 𝑎 ⋅ 𝑇 *F + 𝑂 𝑛H .
a=4
b=2
d=1
a=3
b=2
d=1
a=2
b=2
d=1
a>bd
a>bd
a=bd
✓✓✓
a=1
b=2
d=1
a<bd ✓
![Page 17: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/17.jpg)
Proofofthemastertheorem
• We’lldothesamerecursiontreethingwedidforMergeSort,butbemorecareful.
• Supposethat 𝑇 𝑛 = 𝑎 ⋅ 𝑇 *F + 𝑐 ⋅ 𝑛H.
Pluckythe
PedanticPenguin
Hangon!ThehypothesisoftheMasterTheoremwas
thetheextraworkateachlevelwasO(nd).That’sNOT
thesameaswork<=cnd forsomeconstantc.
That’strue… we’llactuallyproveaweaker
statementthatusesthishypothesisinsteadof
thehypothesisthat𝑇 𝑛 = 𝑎 ⋅ 𝑇 *F + 𝑂 𝑛H .
It’sagoodexercisetomakethisproofwork
rigorouslywiththeO()notation.
Siggi theStudiousStork
![Page 18: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/18.jpg)
Recursiontree
Sizen
n/bn/b
(Size1)
…
n/b2
n/btn/btn/btn/btn/btn/bt
…
Level
Amountof
workatthis
level
0
#
problems
1
2
t
logb(n)
1
a
a2
at
𝑎?@AL *
Sizeof
each
problem
n
n/b
n/b2
n/bt
1
…
n/b
n/b2
n/b2n/b2
n/b2
n/b2
n/b2
𝑇 𝑛 = 𝑎 ⋅ 𝑇 𝑛𝑏 + 𝑐 ⋅ 𝑛H
𝑐 ⋅ 𝑛𝑑
𝑎+𝑐 𝑛𝑏+
H
𝑎𝑐 𝑛𝑏H
𝑎O𝑐 𝑛𝑏O
H
𝑎?@AL * 𝑐
![Page 19: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/19.jpg)
Recursiontree
Sizen
n/bn/b
(Size1)
…
n/b2
n/btn/btn/btn/btn/btn/bt
…
Level
Amountof
workatthis
level
0
#
problems
1
2
t
logb(n)
1
a
a2
at
𝑎?@AL *
Sizeof
each
problem
n
n/b
n/b2
n/bt
1
…
n/b
n/b2
n/b2n/b2
n/b2
n/b2
n/b2
𝑇 𝑛 = 𝑎 ⋅ 𝑇 𝑛𝑏 + 𝑐 ⋅ 𝑛H
𝑐 ⋅ 𝑛𝑑
𝑎+𝑐 𝑛𝑏+
H
𝑎𝑐 𝑛𝑏H
𝑎O𝑐 𝑛𝑏O
H
𝑎?@AL * 𝑐
Totalwork(derivationonboard)isatmost:
𝑐 ⋅ 𝑛H ⋅ P 𝑎𝑏H
OQRSL(*)
OTU
![Page 20: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/20.jpg)
Nowlet’scheckallthecases(onboard)
![Page 21: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/21.jpg)
Evenmoregenerally,forT(n)=aT(n/b)+f(n)…
[FromCLRS]
OllietheOver-AchievingOstrich
Figureouthowtoadapt
theproofwegavetoprove
thismoregeneralversion!
![Page 22: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/22.jpg)
![Page 23: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/23.jpg)
UnderstandingtheMasterTheorem
• Whatdothesethreecasesmean?
• Suppose𝑇 𝑛 = 𝑎 ⋅ 𝑇 *F + 𝑂 𝑛H .Then
![Page 24: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/24.jpg)
Theeternalstruggle
Branchingcausesthenumber
ofproblemstoexplode!
Themostworkisatthe
bottomofthetree!
Theproblemslowerin
thetreearesmaller!
Themostworkisat
thetopofthetree!
![Page 25: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/25.jpg)
Considerourthreewarm-ups
1. 𝑇 𝑛 = 𝑇 *+ + 𝑛
2. 𝑇 𝑛 = 2 ⋅ 𝑇 *+ + 𝑛
3. 𝑇 𝑛 = 4 ⋅ 𝑇 *+ + 𝑛
![Page 26: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/26.jpg)
Firstexample:tallandskinnytree
Sizen
n/2
n/4
n/2t
1
1. 𝑇 𝑛 = 𝑇 *+ + 𝑛, 𝑎 < 𝑏H
• Theamountofworkdoneatthe
top(thebiggestproblem)swamps
theamountofworkdoneanywhere
else.
• T(n)=O(workattop)=O(n)
Mostworkatthe
topofthetree!
WINNER
![Page 27: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/27.jpg)
Thirdexample:bushytree
Sizen
n/2
3. 𝑇 𝑛 = 4 ⋅ 𝑇 *+ + 𝑛, 𝑎 > 𝑏H
• ThereareaHUGEnumberofleaves,andthetotalworkis
dominatedbythetimetodoworkattheseleaves.
• T(n)=O(workatbottom)=O(4depthoftree )=O(n2)
n/2n/2n/2
1 11111 111
1 11111 111
1 11111 111
1 11111 111
1 11111 111
1 111 111
1 1111 111
WINNER
Mostworkat
thebottom
ofthetree!
![Page 28: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/28.jpg)
Secondexample:justright
2. 𝑇 𝑛 = 2 ⋅ 𝑇 *+ + 𝑛, 𝑎 = 𝑏H
• Thebranchingjustbalances
outtheamountofwork.
Sizen
n/2
n/4
1
n/2
n/4n/4n/4
11111 111
• Thesameamountofwork
isdoneateverylevel.
• T(n)=(numberoflevels)*(workperlevel)
• =log(n)*O(n)=O(nlog(n))
1
TIE!
![Page 29: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/29.jpg)
Recap
• The”MasterMethod” makesourliveseasier.
• Butit’sbasicallyjustcodifyingacalculationwecoulddofromscratchifwewantedto.
![Page 30: Lecture 3 - Stanford Universityweb.stanford.edu/class/archive/cs/cs161/cs161.1182/Lectures/Lectu… · Lecture 3 Recurrence Relations and the Master Theorem! Announcements! •HW1](https://reader035.vdocument.in/reader035/viewer/2022062223/5fb486e5692e716b04645954/html5/thumbnails/30.jpg)
NextTime
• Whatifthesub-problemsaredifferentsizes?
• Andwhenmightthathappen?
BEFORE NextTime
• Pre-LectureExercise4!• WhichshouldbeeasierifyoudidPre-LectureExercise3…