graphs - github pages · pre-requisitetree cs2110,sw devmethods cs2150,prog &datarep cs3240,adv...

graphs

1

vertices and edges

vertices or nodes

edges

1

2

34

51

2

34

5

1

2

34

51

2

34

5directed graph

ordigraph

undirected graph

2

example graphs

lots of things can be represented as graphs

3

maps

nodes: intersections?edges: roads?

image: open street map 4

airline routes

image: openflights 5

flowcharts

6

pre-requisite tree

CS 2110, SWDev Methods

CS 2150, Prog& Data Rep

CS 3240, AdvSW Dev Tech

CS 111x, Introto Program'ing

CS 2102,Discrete Math

CS 4102,Algorithms

CS 2330, DigLogic Design

CS 3330,Comp Arch

CS 2190,CS Seminar

CS 4414,Oper Sys

;All CS electives CS 4444,Parallel Comp

CS 4330, AdvComp Arch

CS 4457,Networks

CS 4458,Internet Eng.

CS 4434,Dependable

CS 4620,Compilers

CS 3102,Theory Comp

CS 3205,HCI

Spring only

7

formal definition

graph G: G = (V, E)

V : set of vertices (possibly empty)

E: set of edges — pairs of vertices (possibly empty)directed graph/digraph — ordered pairsundirected graph — unordered pairs

8

paths, etc.

vertices v and w adjacent iff (v, w) ∈ E or (w, v) ∈ E

path: v1, v2, . . . vn such that (vi, vi+1) ∈ E for 1 ≤ i ≤ n

length of path: number of edges in path

simple path: path of distinct vertices

9

weighted graphs

some graphs have weights or costs associated with edges

example motivation:graph representing roads: weight = travel time

weight or cost of a path = sum of weights of edges in path

10

weighted graph example

Charlottesville

Culpeper

DC

Richmond

Fredericksburg46

73

7258

53

35

11

cycles, etc.

cycle: path where length ≥ 1, v1 = vn

undirected graph: …and no repeated edges

AB

CDE X

Y

RQ

Z

AB

CDE X

Y

RQ

Z

AB

CDE X

Y

RQ

Z

A

B

C

D

E

F

G

X

Y

ZS

Q

R

A

B

C

D

E

F

G

X

Y

ZS

Q

R

A

B

C

D

E

F

G

X

Y

ZS

Q

R

A

B

C

D

E

F

G

X

Y

ZS

Q

R

12

loops

(v, v) ∈ E

A

13

graph terminology is not universal

some sources will use slightly different definitions:

walk instead of path

path instead of simple path

closed walk instead of cycle

cycle instead of cycle that is also a simple path

14

connectivity

connected graph: for all x, y ∈ V , there exists a path from x to yN.B: includes 0-length paths

A

B

CD

a connected graphA

B

C

D E

F

a non-connected graph

15

in a directed graph…

DAG — directed acyclic graphno cycles

strongly connected — path from every vertex to every otherimplies cycles (or digraph of 0 or 1 nodes)

weakly connected — would be connected as undirected graph

16

strong/weak connected examples

a strongly connected graphdrawn in two ways

A

B

C

D E

F

G

A

B

CD

EF

G

another stronglyconnected graph

E

F

G H

I

A

B

C

D

a weakly connected graph

a

b

c

d

e

f

g

two strongly connected components

17

trees as graphs

trees are connected, acyclic graphs(with a root chosen)

18

complete graph

complete graph: graph with edges between every pair of distinctvertices

12

3

4

56

7

8

9

10

19

adjacency matrix

A[u][v] =weight if (u, v) ∈ E

0 otherwise

1 2 3 41 0 1 1 12 0 0 1 03 0 0 0 04 1 0 0 0

1

2

3

4

1 2 3 41 9 17 0 02 0 0 13 03 0 0 0 104 0 16 18 0

17

13

16

18

9

1 2

3

4

20

adjacency lists

1

2

3

4

2 3 4 NULL

3 NULL

NULL

1 NULL

1

2

3

4

21

choosing representations

choice:adjacency matrixadjacency listmore?

issues to consider:sizeease of listing edges from nodeease of determining if node X has an edge…

22

variations and alternate representations

adjacency lists might not use linked lists

adjacency matrix can be stored as hashtable (keys=pair of nodes)

…

23

additional information with nodes

often want to store additional information with vertices, edges…

street names, speed limits, …

IP addresses, link speeds, …

…

24

topological sort

only defined for directed acyclic graphorder vertices such that if there is a path from vi to vj, then vj isafter vi

F

H

G

E

C

DB

A

topological sorts:A, F, C, B, D, G, E, H orF, A, H, C, G, B, D, E or…

25

exercise: topological sort

A

B D

C

possible answers: A, B, C, D or A, C, B, D

26

exercise: topological sort

A

B D

C possible answers: A, B, C, D or A, C, B, D

26

no topological sortA

B D

C

27

pre-requisite tree

CS 2110, SWDev Methods

CS 2150, Prog& Data Rep

CS 3240, AdvSW Dev Tech

CS 111x, Introto Program'ing

CS 2102,Discrete Math

CS 4102,Algorithms

CS 2330, DigLogic Design

CS 3330,Comp Arch

CS 2190,CS Seminar

CS 4414,Oper Sys

;All CS electives CS 4444,Parallel Comp

CS 4330, AdvComp Arch

CS 4457,Networks

CS 4458,Internet Eng.

CS 4434,Dependable

CS 4620,Compilers

CS 3102,Theory Comp

CS 3205,HCI

Spring only

28

definition: in-degree

indegree of vertex: number of incoming edges

F0

H1

G2

E2

C2

D2

B2

A0

29

algorithm (simple)

psuedocode:vector<Vertex> topologicalSort(Graph g) {

vector<Vertex> result;for (int i = 0; i < numVertices; ++i) {

Vertex v = g.findVertexOfInDegreeZero();if (did not find v) throw CycleFound();result.push_back(v);for (Vertex w : v.adjacentVertices()) {

g.deleteEdge(v, w);}g.deleteVertex(v);

}return result;

}

30

example

F

H

G

E

C

DB

A

initial in-degree 0 vertices — two choiceschoose one (A — arbitrary),add to result, remove edgesone in-degree 0 vertex: F

result: A,

31

example

F

H

G

E

C

DB

A initial in-degree 0 vertices — two choices

choose one (A — arbitrary),add to result, remove edgesone in-degree 0 vertex: F

result: A,

31

example

F

H

G

E

C

DB

A

initial in-degree 0 vertices — two choices

choose one (A — arbitrary),add to result, remove edges

one in-degree 0 vertex: F

result: A,

31

example

F

H

G

E

C

DB

A

initial in-degree 0 vertices — two choiceschoose one (A — arbitrary),add to result, remove edges

one in-degree 0 vertex: F

result: A,

31

example

F

H

G

E

C

DB

A


result: A, F,

31

example

F

H

G

E

C

DB

A


result: A, F, H,

31

example

F

H

G

E

C

DB

A


result: A, F, H, C,

31

example

F

H

G

E

C

DB

A


result: A, F, H, C, B, G,

31

example

F

H

G

E

C

DB

A


result: A, F, H, C, B, G, D,

31

example

F

H

G

E

C

DB

A


result: A, F, H, C, B, G, D, E,

31

simple topological sort problems

problem: copying the graph?

problem: finding in-degree 0 vertex?scan all vertices and all edges???

32

better pseudocode

vector<Vertex> topologicalSort(Graph g) {vector<Vertex> result;map<Vertex, int> remainingInDegree = g.getInDegrees();

Queue<Vertex> pending;for (Vertex v : g.vertices())

if (remainingInDegree[v] == 0)pending.enqueue(v);

while (!pending.empty()) {Vertex v = pending.dequeue();result.push_back(v);for (Edge e: g.edgesFrom(v)) {

int newDegree = −−remainingInDegree[e.toVertex()];if (newDegree == 0) pending.enqueue(e.toVertex());

}}return result;

} 33

psuedocode idea

track in-degree changes instead of full list of edgesall we care about is in-degree becoming 0

queue: vertices which have in-degree 0 to process

detect cycles? see if result size == number of vertices

34

runtime analysis

assuming |E| edges, |V | vertices, and adjacency listsand in-degree map is constant time (e.g. vertices are 0, 1, 2, …, so it’san array)

step 1: get all in-degreesΘ(|E|) (iterate over edges)

step 2: find + enqueue in-degree 0 verticesΘ(|V |) (iterate over vertices)

step 3: for each vertex, check outgoing edgesΘ(|V | + |E|) (each vertex checked exactly once, each edge checkedexactly once)

overall: Θ(|V | + |E|)35

example

A0

F1

G2

B3

C11

D0

E1

H1

queue: A, D,

result:

36

example

A0

F0

G1

B3

C11

D0

E1

H1

queue: A, D, F,

result: A,

36

example

A0

F0

G1

B3

C11

D0

E0

H0

queue: A, D, F, H, E,

result: A, D,

36

example

A0

F0

G1

B3

C00

D0

E0

H0

queue: A, D, F, H, E, C,

result: A, D, F,

36

example

A0

F0

G1

B2

C00

D0

E0

H0


result: A, D, F, H,

36

example

A0

F0

G1

B1

C00

D0

E0

H0


result: A, D, F, H, E,

36

example

A0

F0

G1

B0

C00

D0

E0

H0

queue: A, D, F, H, E, C, B,

result: A, D, F, H, E, C,

36

example

A0

F0

G0

B0

C00

D0

E0

H0

queue: A, D, F, H, E, C, B, G,

result: A, D, F, H, E, C, B,

36

example

A0

F0

G0

B0

C00

D0

E0

H0

queue: A, D, F, H, E, C, B, G,

result: A, D, F, H, E, C, B, G

36

shortest path

shortest path

lowest {weight,number of edges} path from vertex i to j

37

shortest path applications

map routing

N degrees of separation’

Internet routing

puzzle/game analysis (e.g. rubrik’s cube solutions, …)

38

shortest path algorithm kinds

single pair: path from V to W

single source: for each vertex W , path from V to W

all pairs: for each pair of vertices V, W , path from V to W

39

more formally

given graph G = (V, E) and a vertex s (the source)…

where an edges (v, w) has weight wv,w

for each vertex x find a path v1 = s, v2, . . . , vn = x such that the∑wvi,vi+1 is minimum

40

breadth-first search

shortest path special case: weights = 1

algorithm is breadth-first search

41

special case: breadth-first search on trees

can look at breadth-first search as variation on pre-order traversal

same idea: parents before children

but whole level at a time…

and need to ignore extra paths

42

breadth first search intuition

start with just source

follow edges to first find vertices at distance 1

then use those to find vertices at distance 2, then distance 3, …A

B C D E

G H I J

K LM

key idea: track visited nodesso we don’t check them again(already found the shortest path)

could have list of paths, one per nodebut more compact idea:store one source edge per nodealso called shortest path tree

multiple possible answers!

43

breadth first search pseudocode

void Graph::bfs(Vertex start) {for (Vertex v: vertices) {

v.distance = INFINITY; v.previous = NULL;}Queue frontier;start.distance = 0;frontier.enqueue(start);while (!frontier.isEmpty()) {

Vertex v = q.dequeue();for (Vertex w : verticesWithEdgeFrom(v)) {

if (w.distance == INFINITY) {w.distance = v.distance + 1;w.previous = v;frontier.enqueue(w);

}}

}}

44

BFS runtime?

need to initialize distances to infinity: Θ(|V |) operations

need to check every edge: Θ(|E|) operations

runtime Θ(|V | + |E|)

45

breadth-first search is greedy

greedy algorithms: make the locally optimal choice, never undo

BFS: once one finds a node, one enqueues it oncefind the node later — skip it

why this is okay: find nodes in order of distance

second time ‘visiting’ a node — won’t be a shorter path!

46

add weights: a broken idea

void Graph::BROKEN_shortestPaths(Vertex start) {...while (!frontier.isEmpty()) {Vertex v = q.dequeue();for (Vertex w : verticesWithEdgeFrom(v)) {

// BROKEN!if (w.distance == INFINITY) {

w.distance = v.distance + weightOfEdge(v, w);w.previous = v;frontier.enqueue(w);

}}

}}

}

50

50110

A

B

C

distance 50previous: A

distanceprevious:

47





}}

}}

}

50

50110

A

B

C


distance ∞previous: (none)

47





}}

}}

}

50

50110

A

B

C



47

fix part 1: update to smaller distance


int newDistance = v.distance + weightOfEdge(v, w);if (newDistance < w.distance) {

w.distance = newDistance;w.previous = v;frontier.enqueue(w);

}}

}}

}

problem: now enqueuing nodes multiple timeswant to only visit node once

48

fix part 2: visit nodes once, order by distance

void Graph::SLOW_shortestPaths(Vertex start) {for (Vertex v: vertices) {

v.distance = INFINITY;v.previous = NULL;v.visited = false;

}start.distance = 0;while (!haveUnvisitedNode()) {

Vertex v = findUnvisitedNodeWithSmallestDistance();v.visited = true;for (Vertex w : verticesWithEdgeFrom(v)) {

int newDistance = v.distance + weightOfEdge(v, w);if (newDistance < w.distance) {

w.distance = newDistance;w.previous = v;

}}

}} 49

visiting by distance?

assumption: no negative weights

given this: distance only decreases

and can’t find shorter path from further node!

50

fix part 3: a faster searchvoid Graph::shortestPaths(Vertex start) {

PriorityQueue pq;for (Vertex v: vertices) {

v.distance = INFINITY; v.previous = NULL;}start.distance = 0; pq.insert(0, start);while (!pq.empty()) {

Vertex v = pq.deleteMin();for (Vertex w : verticesWithEdgeFrom(v)) {

int oldDistance = w.distance;int newDistance = v.distance + weightOfEdge(v, w);if (newDistance < oldDistance) {

w.distance = newDistance; w.previous = v;if (oldDistance == INFINITY)

pq.insert(newDistance, w);else

pq.decreaseKey(newDistance, w);}

}}

}51

a note on names

called Dijkstra’s algorithm

52

Dijkstra’s algorithm example 1

21

2

1

2

51

65

1

10

3

A

C

D

B

F

E

G

dist prev pathA 0 — AB ∞ — —C ∞ — —D ∞ — —E ∞ — —F ∞ — —G ∞ — —

D is adjacent —but not a shorter path

F updated from distance 7 (via D)to distance 4 (via C)

53


21

2

1

2

51

65

1

10

3A

C

D

B

F

E

G

dist prev pathA 0 — AB ∞ — —C 2 A A→CD 1 A A→DE ∞ — —F ∞ — —G ∞ — —



53


21

2

1

2

51

65

1

10

3

A

C

D

B

F

E

G

dist prev pathA 0 — AB 6 D A→D→BC 2 A A→CD 1 A A→DE 2 D A→D→EF 7 D A→D→FG 6 D A→D→G



53


21

2

1

2

51

65

1

10

3

A

CD

B

F

E

G

dist prev pathA 0 — AB 6 D A→D→BC 2 A A→CD 1 A A→DE 2 D A→D→EF 4 C A→C→FG 6 D A→D→G


F updated from distance 7 (via D)to distance 4 (via C) 53


21

2

1

2

51

65

1

10

3

A

C

D

B

F

E

G

dist prev pathA 0 — AB 3 E A→D→E→BC 2 A A→CD 1 A A→DE 2 D A→D→EF 4 C A→C→FG 6 D A→D→G



53


7

9

14

10

15

11

2

6

9

A

B

C

G

D

E

dist prev pathA 0 — AB ∞ — —C ∞ — —D ∞ — —E ∞ — —G ∞ — —

54


7

9

14

10

15

11

2

6

9A

B

C

G

D

E

dist prev pathA 0 — AB 7 A A→BC 9 A A→CD ∞ — —E ∞ — —G 14 A A→G

54


7

9

14

10

15

11

2

6

9

A

B

C

G

D

E

dist prev pathA 0 — AB 7 A A→BC 9 A A→CD 22 B A→B→DE ∞ — —G 14 A A→G

54


7

9

14

10

15

11

2

6

9

A

B

C

G

D

E

dist prev pathA 0 — AB 7 A A→BC 9 A A→CD 20 C A→C→DE ∞ — —G 11 C A→C→G

54


7

9

14

10

15

11

2

6

9

A

B

C

G

D

E

dist prev pathA 0 — AB 7 A A→BC 9 A A→CD 20 C A→C→DE 20 G A→C→G→EG 11 C A→C→G

54

Dijkstra’s algorithm runtime

for every vertex (worst case):

find unprocessed vertex with smallest distanceΘ(|V |2) total — if checking every vertexΘ(|V | log |V |) total — if removing from heap

scan all edges of vertex, update distancesΘ(|E|) total — if not maintaining priority queueΘ(|E| log |V |) if updating binary heap

total with binary heap: Θ((|E| + |V |) log |V |)Fibanocci heap instead: Θ(|E| + |V | log |V |)

55

negative weights

example: weight = fuel used; negative weight = refueling

Dijkstra’s algorithm doesn’t workassumption: won’t update a node’s distance after visiting its edges

alternative algorithms do — e.g. Bellman-Ford (Θ(|E||V |) runtime)

negative cost cycles — infinitely small cost!

56

high-level view: dealing with negative weights

Bellman-Ford algorithm

for every node: track shortest known path from sourceinitially: “no known paths”

iterate through all edges updating pathsQ: “can this edge be used to make a better path to source?”

repeat |V | times

57

single-source to single-source+destination

what if want to get from A to Z

solution: Dijkstra’s algorithm from A but stop early — when weproesss Z

gaurentee: won’t update Z’s distance again

58

heuristic shortest path

road map — still slow!

some ideas for speeding up:search highways instead of side-roads earliersearch edges in correct direction earliersearch from both directions, try to meet…

if you take AI — major topic is heuristic searchtaking advantage of ideas like the above…and still getting shortest path, if you want it

59

travelling salesperson problem

given cities, costs to travel between, least-cost trip that:visits each city exactly once, andreturns to the starting city

as a graph:cities = verticescosts = edge weights

assume fully connected graphalternative: first add infinite weight edges between disconnected nodes

60

TSP difficulty

solving TSP exactly is NP-hard

worst case: essentially need to enumerate all possible toursbut, practically solved up to 10000s of cities on ‘real’ maps

obviously doing something smarter…

61

diversion: NP-hard

see also Algorithms

idea: efficient solutions to this problem yield efficient solutions tomany other problems

→ “as hard as” those other problems

other problems ≈ problems whose solutions can be verified inpolynomial time

62

some definitions

Hamiltonian path — path that visits every vertex on a graphexactly once

Hamiltonian cycle — Hamiltonian path that where start node =end node

traveling salesperson problem: find least weight Hamiltonian cycle

63

Hamiltonian cycles and hardness

no known efficient algorithm to detect whether a graph has aHamiltonian cycle

(but easy for complete graphs…)

64

naive TSP algorithm

choose a starting city x1

for each unused next city x2: (n-1 possible)for each unused next city x3: (n-2 possible)

for each unused next city x4: (n-3 possible)…

see if x1, x2, x3, x4, . . . , xn is shorter than anything else

output shortest seen

(N − 1)! factorial runtime = Θ(N !)worse than Θ(2N)

65

naive TSP implementation

vector<Vertex> partial_tour; vector<Vertex> best_tour;void TestTours() {

if (partial_tour.size() == vertices.size()) {partial_tour.push_back(partial_tour[0]);if (weightOf(partial_tour) < weightOf(best_tour)) {

best_tour = partial_tour;}partial_tour.pop_back();

} else {for (Vertex v : vertices − partial_tour) {

partial_tour.push_back(v);TestTours();partial_tour.pop_back(v);

}}

}TSP() {

best_tour = ...; partial_tour = {startNode};TestTours();return best_tour;

}

66

(n-1)! is big

20 cities — > 1016 tours to check

30 cities — > 1030 tours to check

…

67

best gaurenteed TSP algorithm

TSP is NP-hard — no known subexponetial solutionbest general algorithm: Θ(N22N )

20 cities — > 108 operations30 cities — > 1011 operations

uses dynamic programming — covered in 4102

solve subproblems: best way to visit cities 1, 2, 3, 4 starting at 1ending at 4know: if 1, 3, 2, 4 is best for above subproblem, then 1, 3, 2, 4, 5, 1 isshorter than 1, 2, 3, 4, 5, 1can avoid checking 1, 2, 3, 4, 5, 1…

68

TSP heuristics

one idea: branch and bound

still: construct lots and lots of possible tourskeep adding cities

but maintain track extra numbers:the best cost found so farlower bound on the tours we could find with chosen nodes

stop enumerating (return from FindTour early) if lower bound istoo low

69

a lower bound

example lower bound:

if I’ve chosen cities 1, 2, 4, 3 in that order

minimum cost = w(1, 2) + w(2, 4) + w(4, 3) +n∑

i=3min edge from i

if min possible cost > best known cost: stop!

70

other TSP ideas

TSP on real maps — take advantage of geometry

try cities close to each other first

use map distances to compute minimum costs quickly

sometimes can use approximation algorithmsassumption: sufficiently ‘normal’ weights — e.g. A-B shorter than A-C-Bgaurenteed within a certain factor of best solutiongood for pruning very bad solutions quickly

71

TSP records

2006: 85, 900 ‘cities’

distances, etc. from real circuit production problem from the 1980s

72

lab 11

pre-lab: topological sort

in-lab: naive travelling salesperson (map = Tolkein’s middle earth)

post-lab: some acceleration techniques

73

spanning tree definition

given a connected undirected graph G, a spanning tree G′ = (V, E ′) isa subgraph such that:

its edges are a subset of the original graph’s (what subgraph means)

it has the same vertices

it is connected

it has no cycles — i.e. it is a tree

74

spanning tree construction

take a connected graph

repeatedly: remove an edge that does not disconnect the graph

can’t remove any more:now have a spanning tree — same vertices, but is a tree

75

spanning tree examples

A

B

C

D

E

original graph

A

B

C

D

E

A

B

C

D

E

A

B

C

D

E

A

B

C

D

E

spanning treesof graph

76

almost a spanning tree?

EdgwareRoad

StratfordInternational

Star Lane

Stratford High Street

Abbey Road

Leyton

Leytonstone

Stratford

Bromley-By-Bow

WestHam

EastPutney

PutneyBridge

Plaistow

CanaryWharf

NorthGreenwich

CanningTown

DollisHill

Neasden

WillesdenGreen

Archway

TufnellPark

ClaphamSouth

Hampstead

BrentCross

GoldersGreen

ClaphamCommon

ClaphamNorth

Arsenal

HollowayRoad

FinsburyPark

ManorHouse

Brixton

AllSaints

DevonsRoad

Poplar

Blackwall

EastIndia

PuddingMill Lane

HeronQuays

West India Quay

RoyaVictor

Crossharbour &London Arena

Mudchute

SouthQuay

CuttySark

Greenwich

IslandGardens

DeptfordBridge

ElversonRoad

Lewisham

ChiswickPark

BowChurch

Harlesden

WillesdenJunction

KensalGreen

MileEnd

EastActon

NorthActon

Shepherd'sBush

Hammersmith

BowRoad

TurnhamGreen

FulhamBroadway

ParsonsGreen

RavenscourtPark

StepneyGreen

StamfordBrook

GoldhawkRoad

LatimerRoad

FinchleyRoad

SwissCottage

WestHampstead

Kilburn BelsizePark

ChalkFarm

CamdenTown

KentishTown

Stockwell

CaledonianRoad

Highbury &Islington

Limehouse

Westferry

BakerStreet

MaryleboneRegent'sPark

CharingCross

Embankment

PicadillyCircus

Paddington

Elephant &Castle

LambethNorth

Waterloo

QueensPark

KilburnPark

MaidaVale

WarwickAvenue

OxfordCircus

Bank

LiverpoolStreet

St. Paul's

BethnalGreen

BondStreet

MarbleArch

ChanceryLane

Holborn

TottenhamCourt Road

HollandPark

NottingHill Gate Lancaster

GateQueensway

Aldgate

TowerHill

GreatPortland

Street

BarbicanFarringdon

Moorgate

Bayswater Blackfriars

MansionHouseTemple Cannon

Street

Westminster

EustonSquare

King's CrossSt. Pancras

GloucesterRoad

High StreetKensington

SouthKensington

SloaneSquare

VictoriaSt. James'sPark

AldgateEast

Whitechapel

BaronsCourt

WestKensington

Earl'sCourt

Kensington(Olympia)

WestBrompton

CanadaWater

Wapping

Shadwell

LadbrokeGrove

WestbournePark

RoyalOak

St. John'sWood

Bermondsey

LondonBridge

Green Park

Southwark

Angel

OldStreet

Borough

Euston

MorningtonCrescent

LeicesterSquare

Kennington

WarrenStreet

GoodgeStreet

Oval

CoventGarden

Hyde ParkCorner

RussellSquare

Knightsbridge

Pimlico

Vauxhall

MonumentTowerGateway

Zone 1

1km

1mi

Zone 2

Zone 2Zone 2

Zone 2

Zone 1

Zone 3

Zone 3

Zone 3

Zone 3

Wood Lane

EdgwareRoad

White City

Shepherd'sBush Market

https://commons.wikimedia.org/wiki/File:London_Underground_Zone_2.svg 77

https://commons.wikimedia.org/wiki/File:London_Underground_Zone_2.svg

minimum spanning tree

A minimum spanning tree T = (V, E ′) of a weighted graph G isa spanning tree such that

∑e∈E′

weight(e) is smallest.

NB: can be multiple minimum spanning trees

78

minimum spanning tree algorithms

two main algorithms

both greedy — choose edges, then never take that back

tricky part: figuring out what order to choose them in

…and (not this class) proving that’s optimal

79

TSP example (1)

(13 509 us cities)

via U Waterloo: http://www.math.uwaterloo.ca/tsp/ 80

http://www.math.uwaterloo.ca/tsp/

TSP example (2)

(49 603 sites on Nat’l Register of Historic Places)

via U Waterloo: http://www.math.uwaterloo.ca/tsp/ 81

http://www.math.uwaterloo.ca/tsp/

MST example

0.7916524443317322.043065366458222.61021773852578683.0557083800480362

3.122107411784057

3.674469189499639

4.079861813052808

0.8818722233682245

1.8231065073329242

2.1104177339338562.315330611508605

2.3313863574286815

2.9855039258397786

0.84206042195115681.30559299271642741.4023796620684448

2.9894926499283514

3.3437782164331413.80455679430202984.0015906629880895

1.1040695658430248

1.68903358694895612.2647678681598666

3.14674733586472843.81081980498694774.000673598002293

1.2365946527410185

3.1764209776260985

3.393163112099029

3.7539711598729233

3.97046556675163

1.9390952874712843

2.55667250618792832.879881715203522.948960529939049

4.732983889313895

1.00334180946710073.773552581564443

4.725790833225335

2.08479293280193232.3974062900950965

2.908634383802017

3.0657635289780263

3.2564793104537357

1.28023502933616682.6584945366487416

3.6123580832116406

0.75564620077331932.8295266775568666

3.631509992648511

4.0981319042622622.423311884583378

2.9909103749206793

2.074230534431218

3.49783874390409854.384602462511943 3.761199182692469

3.1565560200399294

A

B

C

DE

F

G

H

I

J

K

L

M

N

O

P

Q

82

Prim’s greedy MST algorithm

track: vertices in spanning tree, edges in spanning tree

add a vertex to the spanning tree (arbitrarily)

while not all vertices are in the spanning tree:

pick an edge (u, v) such thatu is already in the spanning treev is not already in the spanning tree(u, v) has the smallest weight of all possible edges

add the edge and v to the spanning tree

83

Prim’s algorithm example

2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

(A, D)(A, D), (A, B)(A, D), (A, B), (C, D)(A, D), (A, B), (C, D), (D, G)(A, D), (A, B), (C, D), (D, G), (F, G)(A, D), (A, B), (C, D), (D, G), (F, G), (E, G)(A, D), (A, B), (C, D), (D, G), (F, G), (E, G)

84


2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

(A, D)

(A, D), (A, B)(A, D), (A, B), (C, D)(A, D), (A, B), (C, D), (D, G)(A, D), (A, B), (C, D), (D, G), (F, G)(A, D), (A, B), (C, D), (D, G), (F, G), (E, G)(A, D), (A, B), (C, D), (D, G), (F, G), (E, G)

84


2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

(A, D)

(A, D), (A, B)

(A, D), (A, B), (C, D)(A, D), (A, B), (C, D), (D, G)(A, D), (A, B), (C, D), (D, G), (F, G)(A, D), (A, B), (C, D), (D, G), (F, G), (E, G)(A, D), (A, B), (C, D), (D, G), (F, G), (E, G)

84


2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

(A, D)(A, D), (A, B)

(A, D), (A, B), (C, D)

(A, D), (A, B), (C, D), (D, G)(A, D), (A, B), (C, D), (D, G), (F, G)(A, D), (A, B), (C, D), (D, G), (F, G), (E, G)(A, D), (A, B), (C, D), (D, G), (F, G), (E, G)

84


2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

(A, D)(A, D), (A, B)(A, D), (A, B), (C, D)

(A, D), (A, B), (C, D), (D, G)

(A, D), (A, B), (C, D), (D, G), (F, G)(A, D), (A, B), (C, D), (D, G), (F, G), (E, G)(A, D), (A, B), (C, D), (D, G), (F, G), (E, G)

84


2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

(A, D)(A, D), (A, B)(A, D), (A, B), (C, D)(A, D), (A, B), (C, D), (D, G)

(A, D), (A, B), (C, D), (D, G), (F, G)

(A, D), (A, B), (C, D), (D, G), (F, G), (E, G)(A, D), (A, B), (C, D), (D, G), (F, G), (E, G)

84


2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

(A, D)(A, D), (A, B)(A, D), (A, B), (C, D)(A, D), (A, B), (C, D), (D, G)(A, D), (A, B), (C, D), (D, G), (F, G)

(A, D), (A, B), (C, D), (D, G), (F, G), (E, G)

(A, D), (A, B), (C, D), (D, G), (F, G), (E, G)

84


2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

(A, D)(A, D), (A, B)(A, D), (A, B), (C, D)(A, D), (A, B), (C, D), (D, G)(A, D), (A, B), (C, D), (D, G), (F, G)(A, D), (A, B), (C, D), (D, G), (F, G), (E, G)

(A, D), (A, B), (C, D), (D, G), (F, G), (E, G)84


A

B

C

DE

F

G

H

I

J

K

L

M

N

O

P

Q

85

Prim’s algorithm runtime

spanning tree will have |V | − 1 edgeseach edge added connects a new vertex

choosing each edgenaive — scan all edges each time |E| workbetter — maintain priority queue of vertices, priority=cost of best edge

up to |E| inserts or decreaseKeys (update best edge for vertex)

max size of priority queue: |V | − 1

Θ(|E| log |V |) time with binary heapΘ(|E| + |V | log |V |) time with Fibanocci heap

86

Prim’s algorithm pseudocode

set<Edge> used_edges; // where result goespriority_queue<Vertex> pending_vertices;map<Vertex, Edge> best_edge_to;for (Vertex v : vertices) {

pending_vertices.insert(INFINITY, v);}pending_vertices.decreaseKey(0, start_vertex);while (!pending_vertices.empty()) {

Vertex v = pending_vertices.deleteMin();used_edges.insert(best_edge_to[v]);for (Edge e : edgesFrom(v)) {

if (e.cost < best_edge_to[e.to].cost) {best_edge_to[e.to] = e;if (e.to in pending_vertices)

pending_vertices.decreaseKey(e.cost, e.to);}

}}

87

Kruskal’s greedy MST algorithm

track: edges in spanning tree

while spanning tree has less than |V | − 1 edges:

pick a minimum weight edge (u, v) such thatadding it to the spanning tree would not create a cycle

add the edge to the spanning tree

88

Kruskal’s algorithm example

2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

(A, D)(A, D), (F, G)(A, D), (F, G), (A, B)(A, D), (F, G), (A, B), (C, D)(A, D), (F, G), (A, B), (C, D), (D, G)(A, D), (F, G), (A, B), (C, D), (D, G), (E, G)(A, D), (F, G), (A, B), (C, D), (D, G), (E, G)

89


2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

(A, D)

(A, D), (F, G)(A, D), (F, G), (A, B)(A, D), (F, G), (A, B), (C, D)(A, D), (F, G), (A, B), (C, D), (D, G)(A, D), (F, G), (A, B), (C, D), (D, G), (E, G)(A, D), (F, G), (A, B), (C, D), (D, G), (E, G)

89


2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

(A, D)

(A, D), (F, G)

(A, D), (F, G), (A, B)(A, D), (F, G), (A, B), (C, D)(A, D), (F, G), (A, B), (C, D), (D, G)(A, D), (F, G), (A, B), (C, D), (D, G), (E, G)(A, D), (F, G), (A, B), (C, D), (D, G), (E, G)

89


2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

(A, D)(A, D), (F, G)

(A, D), (F, G), (A, B)

(A, D), (F, G), (A, B), (C, D)(A, D), (F, G), (A, B), (C, D), (D, G)(A, D), (F, G), (A, B), (C, D), (D, G), (E, G)(A, D), (F, G), (A, B), (C, D), (D, G), (E, G)

89


2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

(A, D)(A, D), (F, G)(A, D), (F, G), (A, B)

(A, D), (F, G), (A, B), (C, D)

(A, D), (F, G), (A, B), (C, D), (D, G)(A, D), (F, G), (A, B), (C, D), (D, G), (E, G)(A, D), (F, G), (A, B), (C, D), (D, G), (E, G)

89


2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

(A, D)(A, D), (F, G)(A, D), (F, G), (A, B)(A, D), (F, G), (A, B), (C, D)

(A, D), (F, G), (A, B), (C, D), (D, G)

(A, D), (F, G), (A, B), (C, D), (D, G), (E, G)(A, D), (F, G), (A, B), (C, D), (D, G), (E, G)

89


2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

(A, D)(A, D), (F, G)(A, D), (F, G), (A, B)(A, D), (F, G), (A, B), (C, D)(A, D), (F, G), (A, B), (C, D), (D, G)

(A, D), (F, G), (A, B), (C, D), (D, G), (E, G)

(A, D), (F, G), (A, B), (C, D), (D, G), (E, G)

89


2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

(A, D)(A, D), (F, G)(A, D), (F, G), (A, B)(A, D), (F, G), (A, B), (C, D)(A, D), (F, G), (A, B), (C, D), (D, G)(A, D), (F, G), (A, B), (C, D), (D, G), (E, G)

(A, D), (F, G), (A, B), (C, D), (D, G), (E, G)89


A

B

C

DE

F

G

H

I

J

K

L

M

N

O

P

Q

90

Kruskal: tracking sets (1)

set ABCD

set FG

set E

2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

track sets of edgessame set — already connectedgoal: add edges that connect distinct sets

91

Kruskal: tracking sets (2)

set ABCD

set FG

set E set E

set ABCDFG

2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

92

Kruskal pseudocode

SetTracker setTracker;for (Vertex v : vertices) {

setTracker.createNewSetFor(v);}vector<Edge> result;for (Edge e : sortByWeight(edges)) {

// check if adding edge would connect unconnected setsif (setTracker.setIdOf(e.from) != setTracker.setIdOf(e.to)) {

result.push_back(e);setTracker.mergeSets(

setTracker.setIdOf(e.from),setTracker.setIdOf(e.to)

);}if (result.size() == vertices.size() − 1) break;

}return result;

93

Kruskal runtime

need to sort all edges (|E| log |E| time)

for each edge: (|E| times)two “find the set something is in” operations

for each edge added: (|V | − 1 times)one “merge two sets” operations

overall: Θ(|E| log |E|) = Θ(|E| log |V |) timeaside: log |V | ∈ Θ(log |E|) since |V |2 ≥ |E| ≥ |V | − 1

94

union-find data structure

SetTracker called a “union-find datastructure” or “disjoint-setdatastructure”

best implementation: slightly worse than amortized constant timeper operation

amortized O(α(n)) time where α(n) is the inverse of the Ackermannfunctionα(n) is asymptotically smaller than log(n)

95

Kruskal runtime

need to sort all edges (|E| log |E| time)

for each edge: (|E| times) O(|E|α(|V|))two “find the set something is in” operations

for each edge added: (|V | − 1 times) O(|V|α(|V|))one “merge two sets” operations

overall: Θ(|E| log |E|) = Θ(|E| log |V |) timeaside: log |V | ∈ Θ(log |E|) since |V |2 ≥ |E| ≥ |V | − 1

96

implementing union-find: naive/slow

map<Vertex, Vertex> parentOf;MakeInitialSets() {

for (Vertex v : vertices)parentOf[v] = v;

}// Each set represented by its "root" vertexVertex FindSetOf(Vertex v) {

if (v == parentOf[v]) {return v;

} else {return FindSetOf(parentOf[v]);

}}UnionSets(Vertex u, Vertex v) {

parentOf[v] = u;} 97

union-find graphs

set ABCDid = A

set FGid = F

set Eid = E

set Eid = E

set ABCDFGid = F

2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

2

4

1

3

10

25

7

8

4

6

1

A

B

C

D

E

F

G

Union(IdOf(D),IdOf(G)

);

98

implementing union-find: path compression

...FindSetOf(Vertex v) {

if (v == parentOf[v]) {return v;

} else {parentOf[v] = FindSetOf(parentOf[v]);return parentOf[v];

}}

shortcut future searches for loop

99

implementing union-find: union by size

map<Vertex, int> sizeOf; // SetId -> # of vertices in setMakeInitialSets() {

for(...)sizeOf[v] = 1;

}

UnionOf(Vertex u, Vertex v) {if (sizeOf[u] > sizeOf[v]) {

(u,v) = (v,u);}// attach lower size to higher sizeparentOf[u] = v;

// update sizesizeOf[v] += sizeOf[u];

} 100

graph summary (1)

directed (digraph) versus undirected

topological sort — ordering of vertices in digraphintuition: find vertex w/ no in-edge, delete

shortest path — minimum edges from one vertex to anotherunweighted: breadth-first search — queue — distance 1 then 2 then 3weighted: Dijkstra’s — priority queue — visit veritices ordered by bestdistance

101

graphs summary (2)

traveling salesperson problem — minimum ‘tour’ — visit all, thenreturn

NP-hard — essentially “try everything” worst casespeedup: stop search early if not better than known bestspeedup: avoid rechecking subproblems (e.g. shortest path from A to Dvisiting A,B,C,D)speedup: heuristics

spanning tree — tree (no cycles) connecting all vertices ofconnected graphminimum spanning tree — spanning tree with min sum of edgeweights

finding: greedy — choose smallest edges first102

aside: MST to approximate TSP

TSP special case: triangle rule applies

w(a → b) ≤ w(a → c) + w(c → b)

one “good” solution: find MST

do an (e.g.) pre-order traversal of the tree

use that as the tour

A

B C

D E

103

MST as good TSP approx

context: triangle rule, use MST pre-order traversal as TSP

worst weight: 2 × MSTedges not in MST: weight not worse than path through treeresult: use every edge twice (to get to node, to get back)

observation: best TSP - one edge = a spanning tree

→ weight of MST ≥ best TSP solution (= some ST + one edge)

→ above TSP — at most 2x as bad as best

A

B C

104

MST to TSP example

A

B

C

DE

F

G

pre-order traversalfrom root F:F,B, E, A,G, D, C,

1

23

45

6

7

105

MST to TSP example

A

B

C

DE

F

Gpre-order traversalfrom root F:F,B, E, A,G, D, C,

1

23

45

6

7

105

graphs - github pages · pre-requisitetree cs2110,sw devmethods cs2150,prog &datarep cs3240,adv...

Documents