ParallelGraphAlgorithms

BasicDefini6ons•  AnundirectedgraphGisapair(V,E),whereVisafinitesetofpointscalledver5cesandEisafinitesetofedges.

•  Anedgee∈ Eisanunorderedpair(u,v),whereuandv∈V.

•  Inadirectedgraph,theedgeeisanorderedpair(u,v).Anedge(u,v)isincidentfrom/outgoingedgeofvertexuandisincidentto/incomingedgeofvertexv.

•  Apathfromavertexvtoavertexuisasequence<v0,v1,v2,…,vk>ofver6ces,wherev0=v,vk=u,and(vi,vi+1)∈ Efori=0,1,…,k-1.

•  Thelengthofapathisdefinedasthenumberofedgesinthepath.

Representa6ons(UndirectedGraphs)• 

Adjacencymatrixrepresenta6on

Adjacencylistrepresenta6on

Problem1:MinimumSpanningTree

•  AspanningtreeofanundirectedgraphGisasub-graphofG,whichisatreecontainingallthever6cesofG.SothespanningtreedoesnotcontainnecessarilyalltheedgesofGbutasubset.

•  Inaweightedgraph,theweightofasub-graphisthesumoftheweightsoftheedgesinthesub-graph.

•  Aminimumspanningtree(MST)foraweightedundirectedgraphisaspanningtreewithminimumweight.

InaPicture

• 

Rela6onshipwithTravelingSalesmanProblem(TSP)

Ø NormallyforTSPcompletegraphsareused(thereisalwaysarouteinbetweentwoci:esnoma;erhowlongittakes)

Ø AnincompletegraphforMSTcanbecompletedbyaddingedgeswithaverylargeweight(notethatthiswillnothaveanyeffectonthesolu6on)

Ø Asolu6onoftheTSPyieldsacyclewithminimalweight.Bydele6nganyedgethiswouldresultinaspanningtree

Ø Soasolu6onofTSPcannothavelessweightthantheweightoftheMST

Ø SotheweightofMSTisalowerboundontheweightofTSP

Sequen6alAlgorithmsforMST•  Borůvka’salgorithm(1926),Kruskal’salgorithm(1956)andPrim’salgorithm(1957)

•  (Historicalnote)Borůvska’salgorithmwasusedin1926toconstructanefficientelectricitynetworkinMoravia(CzechRepublic)*

•  Kruskal’sandPrim’salgorithmarebothbasedontheselec6ngasinglelightestweightedgeineachstepofthealgorithm

*ThealgorithmwasrediscoveredbyChoquetin1938;[4]againbyFlorek,Łukasiewicz,Perkal,Steinhaus,andZubrzycki[5]in1951;andagainbySollin[6]in1965.BecauseSollinwastheonlycomputerscien6stinthislistlivinginanEnglishspeakingcountry,thisalgorithmisfrequentlycalledSollin'salgorithm.

Light-EdgeProperty• 

GivenaweightedundirectedgraphG=(V,E),thenforanycutsetS(SCE),theminimalweightededgeinShastobeanedgeoftheMST

AcutsetsScutsthegraphintotwosetsUandV\UsuchthatanypathfromanodexinUtoanodeyinV\UcontainsanedgefromSProof:AssumewehaveacutsetSwhichcontainsanedgee=(x,y)withminimalweight,whichisnotpartoftheMST.ThenthereisapathPinMST,whichconnectsxandyandwhichdoesnotcontaine.So,becausexandyareonoppositesidesofe,nexttoetheremustbeanedgee’inSwithe’onthepathP.NowaddetotheMST=MST’,theneande’arepartofacycleinMST’.Deletee’fromMST’,andweobtainanotherMSTwithalesserweight(w(e)<w(e’)).Contradic6on.

Kruskal’sAlgorithm

AsdecribedbyKruskalin1956:

“Performthefollowingstepasmany6mesaspossible:AmongtheedgesofGnotyetchosen,choosetheshortestedgewhichdoesnotformanyloopswiththoseedgesalreadychosen”

Prim’sAlgorithm• 

PRIM_MST(V,E,w,r):GivenV,E,andwweightfunc6on,buildMSTstar6ngfromvertexr

• 

Prim'sAlgorithm:ParallelFormula6on

•  Thealgorithmworksinnouteritera6ons-itishardtoexecutetheseitera6onsconcurrently.

•  Theinnerloopisrela6velyeasytoparallelize.Letpbethenumberofprocesses,andletnbethenumberofver6ces.

•  Theadjacencymatrixispar66onedina1-Dblockfashion(columnslices),withdistancevectordpar66onedaccordingly.Seenextslide.

•  Ineachstep,eachprocessorselectsthelocallyclosestnode,followedbyaglobalreduc6ontoselectgloballyclosestnode.

•  ThisnodeisinsertedintoMST,andthechoiceisbroadcastedtoallprocessors.

•  Eachprocessorupdatesitspartofthedvectorlocally.

• 

Computa6onalAspects

•  ThecosttoselecttheminimumentryisO(n/p+logp).•  ThecostofabroadcastisO(logp).•  ThecostoflocalupdateofthedvectorisO(n/p).•  Theparallel6meperitera6onisO(n/p+logp).•  Thetotalparallel6me(nitera6ons)isgivenbyO(n2/p+nlogp).

Borůvka’sAlgorithm(1926)

Whilethereareedgesremaining:(1)  selecttheminimumweightedgeoutofeachvertex

andcontracteachconnectedcomponentdefinedbytheseedgesintoavertex;

(2)  removeselfedges,andwhenthereareredundantedgeskeeptheminimumweightedge;and

(3)  addallselectededgestotheMST.

Notethatthisformula6onisinherentlyparallelwhilecomputerswerenotinventedatthat6me,ormaybebecausecomputerswerenotinventedyet

Example• 

12

7

35

64 4

2

54è è

2

3

1

4

6

Example(otherexecu6onorder)• 

12

7

35

64 3

2

53

7

6

è è

2

3

1

4

6

TheSame!!!!

NotestoBorůvka’sAlgorithm

•  Ateachstepthecontrac6onsofnodesuandvwith(u,v)aminimaledgecanbeexecutedinparallelwiththecontrac6onofnodesxandwwith(x,w)aminimaledge,ifv≠xandu≠w.(Note,u≠xandv≠wautoma6callyholds)

•  Soateachstepatleast½|V|ver6cesareeliminatedèatmostlog(n)stepsarerequired

•  However,alsotheamountofavailableparallelismisreducedbyanhalfauereachstepèunevenloadbalance

InputDataPar66oning•  Recallseparatorsets(nesteddissec6on)forundirectedgraphs,

basedonlevelliza6on(BFS).•  ThesetofnodesVisdividedintoPdisjointsubsetsandseparator

sets: V=V1US2UV2US3…SPUVPP=numberofprocessorsand|Vi|aboutequalforalli

•  DistributetheedgesEsuchthateachprocessorihas Ei={(u,v)|uεViandvεVi},and Leu_Ei={(u,v)|uεSiandvεVi},and Right_Ei={(u,v)|uεViandvεSi+1}

è FirstphaseeveryprocessorcomputesinparallelanMSTforeachEiè Secondthesepar6alMST’sarekni{edtogetherbysynchronizing

thechoiceofminimumweightedgeofLeu_EiwithRight_Ei+1

Problem2:Single-SourceShortestPaths

•  ForaweightedgraphG=(V,E,w,s),thesingle-sourceshortestpathsproblemistofindtheshortestpathsfromavertexs∈Vtoallotherver6cesinV(wistheweigthfunc6onoftheedges).

•  Dijkstra'salgorithmissimilartoPrim'salgorithm.Itmaintainsasetofnodesforwhichtheshortestpathsareknown.

•  Itgrowsthissetbasedonthenodeclosesttosourceusingoneofthenodesinthecurrentshortestpathset.

Dijkstra'sAlgorithm• 

Dijkstra'sAlgorithm:ParallelFormula6on

•  Verysimilartotheparallelformula6onofPrim'salgorithmforminimumspanningtrees.

•  Theweightedadjacencymatrixispar66onedusingthe1-Dblockmapping(columnslicing).

•  Eachprocessselects,locally,thenodeclosesttothesource,followedbyaglobalreduc6ontoselectnextnode.

•  Thenodeisbroadcasttoallprocessorsandthel-vectorupdated.

Problem3:All-PairsShortestPaths

•  GivenaweightedgraphG(V,E,w),theall-pairsshortestpathsproblemistofindtheshortestpathsbetweenallpairsofver6cesvi,vj∈V.

•  Anumberofalgorithmsareknownforsolvingthisproblem:Matrix-Mul6plica6onBasedalgorithm,Dijkstra’salgorithm,Floyd’salgorithm.

Matrix-Mul6plica6onBasedAlgorithm•  Considerthemul6plica6onoftheweightedadjacencymatrixwithitself-except,inthiscase,wereplacethemul6plica6onopera6oninmatrixmul6plica6onbyaddi6on,andtheaddi6onopera6onbyminimiza6on.

•  No6cethattheproductofweightedadjacencymatrixwithitselfreturnsamatrixthatcontainsshortestpathsoflength2betweenanypairofnodes.

•  ItfollowsfromthisargumentthatAncontainsallshortestpaths.

InaPicture• 

Computa6onalAspects•  For(semi)completegraphsandsequen6alexecu6on:

–  Aniscomputedbydoublingpowers-i.e.,asA,A2,A4,A8,andsoon.

–  Weneedlogn(dense)matrixmul6plica6ons,eachtaking6meO(n3).

–  TheserialcomplexityofthisprocedureisO(n3logn).•  For(semi)completegraphsandparallelexecu6on:

–  Eachofthelognmatrixmul6plica6onscanbeperformedinparallel.

–  Wecanusen3/lognprocessorstocomputeeachmatrix-matrixproductin6melogn.

–  Theen6reprocesstakesO(log2n)6me.Notethatforincompletegraphs(leadingtosparsematrices)thiscomplexitydoesnotchangeverymuch,becausesparsexsparsematrixmul6plyveryeasilyleadtofullmatrices.

Dijkstra’sAlgorithmforAll-PairsShortestPaths

Sequen6alExecu6on:– Executeninstancesofthesingle-sourceshortestpathproblem,oneforeachofthensourcever6ces.

– ComplexityisO(n3).ParallelExecu6on:– executeeachofthenshortestpathproblemsonadifferentprocessor(sourcepar66oned),or

– useaparallelformula6onoftheshortestpathproblemtoincreaseconcurrency(sourceparallel)

SourcePar66onedFormula6on

•  Usenprocessors,eachprocessorPifindstheshortestpathsfromvertexvitoallotherver6cesbyexecu6ngDijkstra'ssequen6alsingle-sourceshortestpathsalgorithm.

•  Itrequiresnointerprocesscommunica6on(providedthattheadjacencymatrixisreplicatedatallprocesses).

•  Theparallelrun6meofthisformula6onis:O(n2).O(n2)isthesame6mecomplexityasPrim’salgorithm.

•  Whilethealgorithmiscostop6mal,itcanonlyusenprocessors.

SourceParallelFormula6on

Inthiscase,eachoftheshortestpathproblemsisfurtherexecutedinparallel.Wecanthereforeuseupton2processors.

Floyd’sAlgorithm

•  Foranypairofver6cesvi,vj∈V,considerallpathsfromvitovjwhoseintermediatever6cesbelongtotheset{v1,v2,…,vk}.Letpi(,kj)(ofweightdi(,kj))betheminimum-weightpathamongthem.

•  Ifvertexvkisnotintheshortestpathfromvitovj,thenpi(,kj)isthesameaspi(,kj-1).

•  Ifvkisinpi(,kj),thenwecanbreakpi(,kj)intotwopaths–  onefromvitovkand–  onefromvktovj

Eachofthesepathsusesver6cesfrom{v1,v2,…,vk-1}.

Asaconsequence:

• 

Fromtheseobserva6ons,thefollowingrecurrencerela6onfollows:

Thisequa6onmustbecomputedforeachpairofnodesandfork=1,n.TheserialcomplexityisO(n3).

In(pseudo)code• 

Floyd'sAlgorithm:ParallelExecu6on

•  MatrixD(k)isdividedintopblocksofsize(n/√p)x(n/√p).•  Eachprocessorupdatesitspartofthematrixduringeach

itera6on.•  Tocomputedl(,kr-1)processorPi,jmustgetdl(,kk-1)forallk≠r,

anddk(,kr-1)forallk≠l.•  Ingeneral,duringthekthitera6on,eachofthe√pprocesses

containingpartofthekthrowsendittothe√p-1processesinthesamecolumn.

•  Similarly,eachofthe√pprocessescontainingpartofthekthcolumnsendsittothe√p-1processesinthesamerow.

InaPicture

• 

InaPicture:con6nued

• 

In(pseudo)code• 

Computa6onalAspects

•  Duringeachitera6onofthealgorithm,thekthrowandkthcolumnofprocessorsperformaone-to-allbroadcastalongtheirrows/columns.

•  Thesizeofthisbroadcastisn/√pelements,taking6meΘ((nlogp)/√p).

•  Thesynchroniza6onsteptakes6meΘ(logp).•  Thecomputa6on6meisΘ(n2/p).•  Thetotalparallelrun6me(nstep)ofthe2-Dblockmappingformula6onofFloyd'salgorithmis