comp251: mid-term review - cs.mcgill.cajeromew/comp251material/...avl trees red-black trees • bst...

COMP251:Mid-TermReviewJérôme Waldispühl

SchoolofComputerScienceMcGillUniversity

Overview

Lecture2 HashingLecture3 Heaps&HeapsortLecture4 BSTandAVLtreesLecture5 Red-blacktreesLecture6 DisjointsetsLecture7 Greedyalgorithms(Scheduling,Huffmancoding)Lecture8 Elementarygraphalgorithms

Lecture9 Topologicalsortandstronglyconnectedcomponents

Lecture10 MinimumSpanningTreeLecture11 SinglesourceshortestpathLecture12 BipartitegraphsLecture13 Networkflow1Lecture14 Networkflow2

Techniques

Runningtime

f:runningtime

𝛼·𝑛 :lowerboundonf

𝛽·𝑛 :upperboundonf

• Runningtimeis𝛰 (n)• Runningtimeis𝛺(n)⟹Runningtimeis𝛩(n)

n0

Proofs• Contradiction:Givenaproposition,assumeoppositepropositionistrue,andthenshowsthatitleadstoacontradiction.

• Cutandpaste: Usedwithgraphsandgreedyalgorithms.Oftenusedtoproveanoptimalsolutionofaproblemisbuildfromoptimalsolutionofsub-problem(Optimalsubstructure).Assumeasub-problemisnotoptimal,andreplacewithoptimalsolutiontoshowacontradiction.

• Loopinvariants: Usedtoprovethataloopstructureisdoingwhatitisintendedtodo.Youmustspecify:§ Loopinvariantproperty§ Initialization§ Maintenance§ Termination

OptimalsubstructureLemmaAnysubpath ofashortestpathisashortestpath.

Proof:

Supposethispathp isashortestpathfromu tov.Thenδ(u,v)=w(p)=w(pux)+w(pxy)+w(pyv).Nowsupposethereexistsashorterpathxy.Thenw(p’xy)<w(pxy).w(pux)+w(p’xy)+w(pyv)<w(pux)+w(pxy)+w(pyv)Contradiction ofthehypothesisthatp istheshortestpath!

p’xy

u x y v

pxy

p’xy

Hashing

Resolutionbychaining• InsertiontimeinO(1)ifweinsertattheheadofthelist.• SearchtimeinO(1)timeinaverage,butnottheworstcase.

Hashfunction: h :U→ {0,1,...,m−1}

h(k1)

h(k4)

Openaddressing

index key1 35523 5674 23356 7997

h(282,0)=3

h(282,1)=1

h(282,2)=5

282

Note:Searchmustusethesameprobesequence.

Illustration:Wheretostorekey282?

Full!

✔

Linear&Quadraticprobing

h(k, i) = h '(k)+ i( )modm

h(k, i) = h '(k)+ c1 ⋅ i+ c2 ⋅ i2( )modm

Note:tendencytocreateclusters.

Remarks:• Wemustensurethatwehaveafullpermutationof⟨

0,…,m-1⟩.• Secondaryclustering:2 distinctkeyshavethesame

hʹvalue,iftheyhavethesameprobesequence.

Linearprobing:

Quadraticprobing:

Rotations

yx

A B

C

xy

CB

A

Rightrotation

Leftrotation

Rotations:• Changetreestructure• PreservetheBSTproperty.

Example:rightrotationaty

yx

A B

C

1 yx

A B

C

2

xy

CB

A4

xy

CB

A

3

BST&Self-balancedtrees

x

|hleft-hright|≤1

26

17

30 47

38 50

NilNil Nil

Nil

Nil

Nil Nil

41

AVLtrees Red-blacktrees

• BST(usedtostorekeys)• Runningtimedependentoftheheight⟹ wetrytokeepthetreesbalanced.• AVL&Red-Blacktreesare2typesofself-balancedtrees• ChallengeistokeeptheAVLorRed-Blacktreepropertyvalidaftereachoperation.

InsertRBTree– Example7

3

10 20

11 22

NilNil NilNil

Nil Nil

18

Nil8

NilNil


3

10 20

11 22

NilNilNil

Nil Nil

18

Nil

Insert(T,15)

15

NilNil

8

NilNil


3

10 20

11 22

NilNilNil

Nil Nil

18

Nil

Recolor10,8&11

15

NilNil

8

NilNil


3

10 20

11 22

NilNilNil

Nil Nil

18

Nil

15

NilNil

8

NilNil

Rightrotateat18


3

18Nil Nil

10

20

22

NilNil

Nil

Rightrotateat18(parent&childwithconflictarealigned)

11

Nil 15

NilNil

8

NilNil


3

18Nil Nil

10

20

22

NilNil

Nil

Leftrotateat7

11

Nil 15

NilNil

8

NilNil

InsertRBTree– Example

7

3

Nil Nil

10

Leftrotateat7

18

20

22

NilNil

Nil

11

Nil 15

NilNil

8

NilNil

InsertRBTree– Example

7

3

Nil Nil

10

Recolor10&7(rootmustbeblack!)

18

20

22

NilNil

Nil

11

Nil 15

NilNil

8

NilNil

Case1– uncley isred

• p[p[z]](zʼsgrandparent)mustbeblack,sincezandp[z]arebothredandtherearenootherviolationsofproperty4.•Makep[z]andyblackÞ nowzandp[z]arenotbothred.Butproperty5mightnowbeviolated.•Makep[p[z]]redÞ restoresproperty5.• Thenextiterationhasp[p[z]]asthenewz(i.e.,zmovesup2levels).

z isarightchildhere.Similarstepsifz isaleftchild.

C

A D

Ba

b g

d ez

yp[z]

p[p[z]]

a

b g

d e

new zC

A D

B

Case2– yisblack,z isarightchild

• Leftrotatearoundp[z],p[z]andz switchrolesÞ nowzisaleftchild,andbothzandp[z]arered.• Takesusimmediatelytocase3.

C

A

Ba

b g

z

yp[z]C

B

A

a b

g(new)z

y(new)p[z]D D

δ λ δ λ

Case3– yisblack,z isaleftchild

•Makep[z]blackandp[p[z]]red.• Thenrightrotaterightonp[p[z]](inordertomaintainproperty4).• Nolongerhave2redsinarow.• p[z]isnowblackÞ nomoreiterations.

C

B

A

a b

g

yp[z]

z

D

B

A

a b g

C

Dδ λ

δ λ

p[p[z]]

z

p[z]

Greedyalgorithms

Activity-selectionProblem

012345678910

s6 a6

a5

a4

a3

a2

a1 a7s1

s2

s3

s4

s5

s7f1

f2

f3

f4

f5

f6

f7

i 1 2 3 4 5 6 7si 0 1 2 4 5 6 8fi 2 3 5 6 9 9 10

Activitiessortedbyfinishingtime.

GraphAlgorithms

TopologicalSortWantto“sort” adirectedacyclicgraph(DAG).

B

E

D

C

A

C EDA B

ThinkoforiginalDAGasapartialorder.

Wantatotalorder thatextendsthispartialorder.

Example1

LinkedList:

A B D

C E

1/

Example1

LinkedList:

A B D

C E

1/

2/

Example1

LinkedList:

A B D

C E

1/

2/3

E

2/3

Example1

LinkedList:

A B D

C E

1/4

2/3

E

2/31/4

D

Example1

LinkedList:

A B D

C E

1/4

2/3

E

2/31/4

D

5/

Example1

LinkedList:

A B D

C E

1/4

2/3

E

2/31/4

D

5/

6/

Example1

LinkedList:

A B D

C E

1/4

2/3

E

2/31/4

D

5/

6/7

6/7

C

Example1

LinkedList:

A B D

C E

1/4

2/3

E

2/31/4

D

5/8

6/7

6/7

C

5/8

B

Example1

LinkedList:

A B D

C E

1/4

2/3

E

2/31/4

D

5/8

6/7

6/7

C

5/8

B

9/

Example1

LinkedList:

A B D

C E

1/4

2/3

E

2/31/4

D

5/8

6/7

6/7

C

5/8

B

9/10

9/10

A

MinimumSpanningTrees

MinimumSpanningTree(MST)

a

b

c

e

d

f h

g

i

10

12

9

8

7

3 3

1

8

5

6

2

11

9

• Ithas|V|−1edges.• Ithasnocycles.• Itmightnotbeunique.

Definitions

a

b

c

e

d

f h

g

i

10

12

9

8

7

3 3

1

8

5

6

2

11

9

cut partitionsverticesintodisjointsets,S andV – S.

S V- S

Thisedgecrosses thecut.(oneendpointisinS andtheotherisinV– S.)

Alight edgecrossingcut(maynotbeunique)

Acutrespects AifandonlyifnoedgeinAcrossesthecut.

b

a

c

e

KruskalʼsAlgorithm

1. Startswitheachvertexinitsowncomponent.2. Repeatedlymergestwocomponentsintoonebychoosinga

lightedgethatconnectsthem(i.e.,alightedgecrossingthecutbetweenthem).

3. Scansthesetofedgesinmonotonicallyincreasingorderbyweight.

4. Usesadisjoint-setdatastructuretodeterminewhetheranedgeconnectsverticesindifferentcomponents.

Note:WealsocoveredthePrim’salgorithmtocalculateaMST.

Example

a

b

c

e

d

f h

g

i

10

12

9

8

7

3 3

1

8

5

6

2

11

9

Example

a

b

c

e

d

f h

g

i

10

12

9

8

7

3 3

1

8

5

6

2

11

9

Reject!

Example

a

b

c

e

d

f h

g

i

10

12

9

8

7

3 3

1

8

5

6

2

11

9

Proof:LetTbeaMSTthatincludesA.Case1: (u,v)inT.Weʼredone.Case2: (u,v)notinT.Wehavethefollowing:

u y

x

v

edgeinA

cut

WeshowedgesinT

Safeedge

Theorem1: Let(S,V-S)beanycutthatrespectsA,andlet(u,v) bealightedgecrossing(S,V-S).Then,(u,v)issafeforA.

(x,y)crossescut.LetT´ =T- {(x,y)}È {(u,v)}.Because(u,v)islightforcut,w(u,v)£ w(x,y).Thus,w(T´)=w(T)-w(x,y)+w(u,v)£w(T).Hence,T´ isalsoaMST.So,(u,v)issafeforA.

Singlesourceshortestpaths

Relaxinganedge

RELAX(u,v,w)if d[v]>d[u]+w(u,v) thend[v] ß d[u]+w(u,v)π[v]ßu

4 93 vu

4 7

Relax

4 63 vu

4 6

Dijkstra’s algorithm

DIJKSTRA(V, E,w,s)INIT-SINGLE-SOURCE(V,s)S ← ∅Q ← Vwhile Q ≠ ∅ do

u ← EXTRACT-MIN(Q)S ← S ∪ {u}for each vertex v ∈ Adj[u] do

RELAX(u,v,w)

Example

0

10

5

1

2

6

2 724

3s

z

x

y

t

∞

∞

∞

∞

s t y x zQ

Example

0

10

5

10

5

1

2

6

2 724

3s

z

x

y

t

∞

∞

y t x zQ

Example

0

6

5

9

11

10

5

1

2

6

2 724

3s

z

x

y

t

t x zQ

Example

0

6

5

8

11

10

5

1

2

6

2 724

3s

z

x

y

t

x zQ

Example

0

6

5

8

10

10

5

1

2

6

2 724

3s

z

x

y

t

zQ

Example

0

6

5

8

10

10

5

1

2

6

2 724

3s

z

x

y

t

Q

Example

0

6

5

8

10

10

5

1

2

6

2 724

3s

z

x

y

t

Dijkstra’s algorithm

DIJKSTRA(V, E,w,s):INIT-SINGLE-SOURCE(V,s)S ← ∅Q ← Vwhile Q ≠ ∅ do

u ← EXTRACT-MIN(Q)S ← S ∪ {u}for each vertex v∈Adj[u] do

RELAX(u,v,w)

LoopInvariant:Atthestartofeachiteration:d[v]=δ(s,v)∀v∈ S.

• Variableusedtocalculateshortestpath:d• Propertyusedtocalculateshortestpath:d[v]=δ(s,v)

InitializationInitially,S=∅⟹ True

Termination:StopswhenQ=∅ ⇒d[v]=δ(s,v)∀ v∈ V(byLoopInvariantProperty)

Maintenance:Showthatd[u]=δ(s,u)whenuisaddedtoSineachIteration.

Bipartitegraphs

Example

Z

X

Y

A

C

B Companies

1st 2nd 3rd

Xavier Alphabet Baidu Campbell

Yulia Baidu Alphabet Campbell

Zoran Alphabet Baidu Campbell

Candidates’preferences1st 2nd 3rd

Alphabet Yulia Xavier Zoran

Baidu Xavier Yulia Zoran

Campbell Xavier Yulia Zoran

Companies’preferences

Q:IsX-C,Y-B,Z-Aagoodassignment?

Candidates

Example

Z

X

Y

A

C

B Companies

1st 2nd 3rd

Xavier Alphabet Baidu Campbell

Yulia Baidu Alphabet Campbell

Zoran Alphabet Baidu Campbell

Candidates’preferences1st 2nd 3rd

Alphabet Yulia Xavier Zoran

Baidu Xavier Yulia Zoran



Q:IsX-C,Y-B,Z-Aagoodassignment?A:No!XavierandBaidubothprefertobematchedtogether…

Candidates

Gale-ShapleyalgorithmForeachα∈A,letpref[α]betheorderingofitspreferencesinBForeachβ∈B,letpref[β]betheorderingofitspreferencesinALetmatchingbeasetofcrossingedgesbetweenAandB

matching¬Æwhile thereisα∈Anotyetmatcheddo

β¬pref[α].removeFirst()ifβnotyetmatchedthen

matching¬matchingÈ{(α,β)}else

γ¬β’scurrentmatchif βprefersαoverγ then

matching¬matching-{(γ,β)}È{(α,β)}return matching

Example

Z

X

Y

A

C

BCandidates Companies

1st 2nd 3rd

Xavier Baidu Alphabet Campbell

Yulia Baidu Campbell Alphabet

Zoran Alphabet Campbell Baidu

1st 2nd 3rd

Alphabet Zoran Xavier Yulia

Baidu Yulia Zoran Xavier


Companies’preferencesCandidates’preferences

Example

Z

X

Y

A

C

B Companies

1st 2nd 3rd




1st 2nd 3rd





Candidates

Candidates’preferences

Example

Z

X

Y

A

C

B Companies

1st 2nd 3rd




Men’spreferences1st 2nd 3rd





Candidates

Example

Z

X

Y

A

C

B Companies

1st 2nd 3rd




1st 2nd 3rd





Candidates

Candidates’preferences

Flownetworks

DefinitionsPositiveflow:Afunctionp:V× V→Rsatisfying.

Capacityconstraint:Forallu,v∈ V,0≤p(u,v)≤c(u,v),��

Flowconservation:Forallu∈ V−{s,t}, p(v,u)v∈V∑ = p(u,v)

v∈V∑

Flowintou Flowoutofu

1/2

Positiveflow Capacity

0/2 2/22/3

1/1 1/2Flowin:0+2+1=3Flowout:2+1=3

s

1/3

2/2

1/3

2/2

2/3

0/10/21/3

1/1

2/3

t1/2

• Flowoutofsources ==Flowinthesinkt

• Flow=

• Objective: findmaximumflow

f (s,v)v∈V∑

Maxflow

Example

s

0/2

3/3

0/2

0/3

3/43/3

t

0/2

FlowinG

ResidualgraphGf

s

2

3

2

3

33

t

2

1

Example

ResidualgraphGf

FlowinGf

s

2

3

2

3

33

t

2

1

s

2

0

2

2

20

t

2

0

Example

G

Gf s

2

0

2

2

20

t

2

0

s

0/2

3/3

0/2

0/3

3/43/3

t

0/2

s

2/2

3/3

2/2

2/3

1/43/3

t

2/2

|f|=3

|f|=5

β=2

Question:Howdoweknowifaflowismaximum?

Flowthroughacut

Claim:Givenaflownetwork.LetfbeaflowandA,Bbeas-tcut.Then,

Notation:|f|=fout(A) – fin(A)

f = f (e)− f (e)e∈cut (B,A)∑

e∈cut (A,B)∑

t3

s

5

1

2

4A

B

1

|f|=9-4=5

Maxflow=Mincut

• Ford-Fulkersonterminateswhenthereisnopaths-tintheresidualgraphGf

• Thesetofnodesaccessiblefroms inGf definesacutinG

• Ford-Fulkersonflow=

=capacityofcut(A,B)

c(e)− 0e∈cut (A,B)∑

f = f out (A)− f in (A)

= f (e)e∈cut (A,B)∑ − f (e)

e∈cut (B,A)∑

Example:MinCut

s t

s t

s t

s t

Notacut!

Notacut!

Notamincut!

mincut!

Note:Alledgeshaveacapacityof1.

Example:CalculateMinCut

s t

Tofindamincutcomputeamaxflow.

s t

backwardforward

1

0 0 0 0

1

1

1 1

1

Flow

Residualgraph

Example:CalculateMinCutTofindthecutrunBFS(orDFS)fromsontheresidualgraph.Thereachableverticesdefinethe(min)cut.

Mincut(inG!)

ResidualgraphwithDFS

s t

s t

Announces

• Officehoursextendeduntil3pmtoday.

• Officehours(Carlos&Roman)onwednesday from2pmto4pminTR3110.

• Mid-termexamscheduledat11h30(regularclasshours)inADAMSAuditorium.

• Onecribsheet(2pages)allowed.

comp251: mid-term review - cs.mcgill.cajeromew/comp251material/...avl trees red-black trees • bst...

Documents