COSC 3101N J. Elder

Assignment 2 Remarking

Assignment 2 Marks

y = 0.995x + 8.47

R2 = 0.8558

0

10

20

30

40

50

60

70

80

90

100

0 50 100

Old Mark

New

Mar

k

COSC 3101N J. Elder

Section V. Graph Algorithms

COSC 3101N J. Elder

Breadth-First Search

Idea: send out search ‘wave’ from s.

Keep track of progress by colouring vertices: Undiscovered vertices are coloured white

Just discovered vertices (on the wavefront) are coloured grey.

Previously discovered vertices (behind wavefront) are coloured black.

Graph ( , ) (directed or undirected) aI nput: source vertex nd .V E sG V

Ouput:

[ ] distance f rom to , .

[ ] such that ( , ) is last edge on shortest path f rom to .

d v s v v V

v u u v s v

COSC 3101N J. Elder

Breadth-First Search Algorithm

Each vertex assigned finite d value at most once.

d values assigned are monotonically increasing over time.

Q contains vertices with d values {i, …, i, i+1, …, i+1}

Time = O(V+E)

COSC 3101N J. Elder

Depth-First Search

Explore every edge, starting from different vertices if necessary.

As soon as vertex discovered, explore from it.

Keep track of progress by colouring vertices: White: undiscovered vertices

Grey: discovered, but not finished (still exploring from it)

Black: finished (found everything reachable from it).

Graph ( , ) (directed or uI nput ndire: c ) tedG V E

2 timestamps on each vertex:

[ ] discovery time.

[ ] fi nishing time.

Ouput:

d v

f v 1 [ ] [ ] 2| |d v f v V

COSC 3101N J. Elder

Depth-First Search Algorithm

total work = | [ ] | ( )v V

Adj v E

total work = ( )V

Thus running time = ( )V E

COSC 3101N J. Elder

Why is BFS O(S+V), while DFS is (S+V)?

Remember: BFS(G,s) finds the shortest paths from a specified vertex s.

If G is either undirected and not connected, or directed and not strongly connected, not all vertices of the graph will be visited in BFS.

In contrast, DFS(G) visits all vertices, restarting at multiple source vertices if necessary.

COSC 3101N J. Elder

An Application of DFS: Topological Sorting

COSC 3101N J. Elder

Strongly Connected Components

Given directed graph ( , )

A (SCC) of G

is a maximal set of vertices such that

, , is reachable f rom and vice-ver

strongly connected compone t

sa.

n

G V E

C V

u v C v u

COSC 3101N J. Elder

Component Graph

( , )SCC SCC SCCG V E

has one vertex f or each SCC in .SCCV G

has an edge if there is an edge between the corresponding SCCs in .SCCE G

G

SCCG

COSC 3101N J. Elder

Lemma

is a DAG.SCCG

I n other words,

Let and be distinct SCCs in C C G

Let , , ,u v C u v C

Suppose there is a path f rom to in .u u G

Then there cannot also be a path f rom to in .v v G

COSC 3101N J. Elder

SCC Algorithm

COSC 3101N J. Elder

Lemma

Let and be distinct SCCs in ( , ). C C G V E

Suppose there is an edge ( , ) such that and .u v E u C v C

Then ( ) ( ).f C f C

Similarly,

Suppose there is an edge ( , ) such that and .Tu v E u C v C

Then ( ) ( ).f C f C

COSC 3101N J. Elder

So why does the SCC Algorithm work?

GC Cu v

TGC Cu v

The second DFS on starts with SCC such that ( ) is maximum.TG C f C

Since ( ) ( ) , there are no edges f rom to in .Tf C f C C C C C G

Thus DFS will visit only vertices in .C

The next root chosen in the second DFS is in SCC such that f (C ) is maximum

over all SCCs other than .

C

C

DFS visits all vertices in , but the only edges out of go to , which we've already visited.C C C

COSC 3101N J. Elder

Minimum Spanning Trees

Example Problem You are planning a new terrestrial telecommunications network to

connect a number of remote mountain villages in a developing country.

The cost of building a link between pairs of neighbouring villages (u,v) has been estimated: w(u,v).

You seek the minimum cost design that ensures each village is connected to the network.

The solution is called a minimum spanning tree.

COSC 3101N J. Elder

Building the Minimum Spanning Tree

Iteratively construct the set of edges A in the MST.

Initialize A to {}

As we add edges to A, maintain a Loop Invariant: A is a subset of some MST

Maintain loop invariant and make progress by only adding safe edges.

An edge (u,v) is called safe for A iff A{u,v}) is also a subset of some MST.

COSC 3101N J. Elder

Finding a safe edge

Idea: Every 2 disjoint subsets of vertices must be connected by at least one edge.

Which one should we choose?

COSC 3101N J. Elder

Some definitions

A cut (S,V-S) is a partition of vertices into disjoint sets V and S-V.

Edge (u,v)E crosses cut (S, V-S) if one endpoint is in S and the other is in V-S.

A cut respects A iff no edge in A cross the cut.

An edge is a light edge crossing a cut iff its weight is minimum over all edges crossing the cut.

COSC 3101N J. Elder

Theorem

Let A be a subset of some MST

(S,V-S) be a cut that respects A

(u,v) be a light edge crossing (S,V-S)

Then (u,v) is safe for A.

COSC 3101N J. Elder

Proof

Let T be an MST that includes A.

If T contains (u,v) then we’re done.

Suppose T does not contain (u,v) Can construct different MST T' that includes A{u,v})

The edge (u,v) forms a cycle with the edges on the path from u to v in T.

There is at least one edge in T that crosses the cut: let that edge be (x,y)

(x,y) is not in A, since the cut (S,V-S) respects A.

Form new spanning tree T' by deleting (x,y) from T and adding (u,v).

w(T') w(T), since w(u,v) w(x,y) T' is an MST.

A T', since A T and (x,y)A A{u,v}) T'

Thus (u,v) is safe for A.

All edges shown (except (u,v)) are in T

COSC 3101N J. Elder

Kruskal’s Algorithm for computing MST

Starts with each vertex being its own component.

Repeatedly merges two components into one by choosing the light edge that crosses the cut between them.

Scans the set of edges in monotonically increasing order by weight.

Uses a disjoint-set data structure to determine whether an edge connects vertices in different components.

COSC 3101N J. Elder

Kruskal’s Algorithm for computing MST

Running Time =

O(ElogE)

= O(ElogV)

COSC 3101N J. Elder

Prim’s Algorithm

Build one tree

Start from arbitrary root r

At each step, add light edge crossing cut (VA, V- VA), where VA = vertices that A is incident on.

COSC 3101N J. Elder

Finding light edges quickly

All vertices not in the partial MST formed by A reside in a min-priority queue.

Each object is a vertex v in V-VA

Key of v is minimum weight of any edge (u,v), u VA.

Each vertex knows its parent in partial MST by [v].

As algorithm progresses, {( , [ ]) : { } }.A v v v V r Q

COSC 3101N J. Elder

Prim’s Algorithm

Running Time =

O(Elog(V))

( )O V

Executed | | timesV

(log )O V

Executed |E | times

(log )O V

COSC 3101N J. Elder

Example: Prim's Algorithm

COSC 3101N J. Elder

Single-Source Shortest Paths

COSC 3101N J. Elder

The Problem

What is the shortest driving route from Toronto to Ottawa? (e.g. MAPQuest, Microsoft Streets and Trips)

Input: Directed graph G=(V,E) Weight function w: ER

0 1 11

Weight of path , ,..., ( , )k

iiki

p v v v w v v

Shortest-path weight f rom to :u v

min{ ( ) : } if a path ,( , ) otherwise.

p

w p u v u vu v

Shortest path f rom to is any path such that ( ) ( , ).u v p w p u v

COSC 3101N J. Elder

Example

COSC 3101N J. Elder

Shortest path variants

Single-source shortest-paths problem: – the shortest path from s to each vertex v. (e.g. BFS)

Single-destination shortest-paths problem: Find a shortest path to a given destination vertex t from each vertex v.

Single-pair shortest-path problem: Find a shortest path from u to v for given vertices u and v.

All-pairs shortest-paths problem: Find a shortest path from u to v for every pair of vertices u and v.

COSC 3101N J. Elder

Negative-weight edges

OK, as long as no negative-weight cycles are reachable from the source.

If we have a negative-weight cycle, we can just keep going around it, and get w(s, v) = −∞ for all v on the cycle.

But OK if the negative-weight cycle is not reachable from the source.

Some algorithms work only if there are no negative-weight edges in the graph.

COSC 3101N J. Elder

Optimal substructure

Lemma: Any subpath of a shortest path is a shortest path Proof: Cut and paste.

Now suppose there exists a shorter path .xyp

x y

Then ( ) ( ).xy xyw p w p

Construct p:

Then ( ) ( ) ( ) ( )ux xy yvw p w p w p w p ( ) ( ) ( )ux xy yvw p w p w p ( ).w p

So p wasn't a shortest path af ter all!

Suppose this path is a shortest path f rom to .p u v

Then ( , ) ( ) ( ) ( ) ( ).ux xy yvu v w p w p w p w p

COSC 3101N J. Elder

Cycles

Shortest paths can’t contain cycles:

Already ruled out negative-weight cycles.

Positive-weight: we can get a shorter path by omitting the cycle.

Zero-weight: no reason to use them assume that our solutions won’t use them.

COSC 3101N J. Elder

Output of a single-source shortest-path algorithm

For each vertex v in V:

d[v] = δ(s, v).

Initially, d[v]=∞.

Reduces as algorithms progress. But always maintain d[v] ≥ δ(s, v).

Call d[v] a shortest-path estimate.

π[v] = predecessor of v on a shortest path from s.

If no predecessor, π[v] = NIL.

π induces a tree — shortest-path tree.

COSC 3101N J. Elder

Initialization

All shortest-paths algorithms start with the same initialization:

INIT-SINGLE-SOURCE(V, s)

for each v in V

do d[v]←∞

π[v] ← NIL

d[s] ← 0

COSC 3101N J. Elder

Relaxing an edge

Can we improve shortest-path estimate for v by going through u and taking (u,v)?

RELAX(u, v,w)if d[v] > d[u] + w(u, v) then

d[v] ← d[u] + w(u, v)π[v]← u

COSC 3101N J. Elder

General single-source shortest-path strategy

1. Start by calling INIT-SINGLE-SOURCE2. Relax Edges

Algorithms differ in the order in which edges are taken andhow many times each edge is relaxed.

COSC 3101N J. Elder

Example: Single-source shortest paths in a directed acyclic graph (DAG)

Since graph is a DAG, we are guaranteed no negative-weight cycles.

COSC 3101N J. Elder

Algorithm

Time: ( )V E

COSC 3101N J. Elder

Example

COSC 3101N J. Elder

Example

COSC 3101N J. Elder

Example

COSC 3101N J. Elder

Example

COSC 3101N J. Elder

Example

COSC 3101N J. Elder

Example

COSC 3101N J. Elder

Correctness: Path relaxation property (Lemma 24.15)

0 1 0Let , , . . . , be a shortest path f rom .k kp v v v s v to v

0 1 1 2 -1I f we relax, in order, ( , ), ( , ), . . . , ( , ), k kv v v v v v

even intermixed with other relaxations,

then [ ] ( , ).k kd v s v

COSC 3101N J. Elder

Correctness of DAG Shortest Path Algorithm

Because we process vertices in topologically sorted order, edges of any path are relaxed in order of appearance in the path.

Edges on any shortest path are relaxed in order.

By path-relaxation property, correct.

COSC 3101N J. Elder

Example: Dijkstra’s algorithm

Applies to general weighted directed graph (may contain cycles).

But weights must be non-negative.

Essentially a weighted version of BFS. Instead of a FIFO queue, uses a priority queue.

Keys are shortest-path weights (d[v]).

Maintain 2 sets of vertices: S = vertices whose final shortest-path weights are determined.

Q = priority queue = V-S.

COSC 3101N J. Elder

Dijkstra’s algorithm

Observations: Looks a lot like Prim’s algorithm, but computing d[v], and

using shortest-path weights as keys. Dijkstra’s algorithm can be viewed as greedy, since it

always chooses the “lightest” (“closest”?) vertex in V − S to add to S.

COSC 3101N J. Elder

Dijkstra’s algorithm: Analysis

Analysis: Using binary heap, each operation takes O(logV)

time O(ElogV)

COSC 3101N J. Elder

Example

COSC 3101N J. Elder

Example

COSC 3101N J. Elder

Example

COSC 3101N J. Elder

Example

COSC 3101N J. Elder

Example

COSC 3101N J. Elder

Example

COSC 3101N J. Elder

Correctness of Dijkstra’s algorithm

Loop invariant: At the start of each iteration of the while loop, d[v] = δ(s, v) for all v in S.

Initialization: Initially, S is empty, so trivially true.

Termination: At end, Q is empty S = V d[v] = δ(s, v) for all v in V.

Maintenance: Need to show that d[u] = δ(s, u) when u is added to S in each

iteration.

Proof is by contradiction (see text).