principle of algorithms x -...

Algorithm Design and ImplementationPrinciple of Algorithms X

Network Flow I

Guoqiang Li

School of Software, Shanghai Jiao Tong University

1

Max-Flow and Min-Cut Problems

1

A Flow network

A flow network is a tuple G = (V ,E , s, t, c).

• Diagraph (V ,E) with source s ∈ V and sink t ∈ V .

• Capacity c(e) > 0 for each e ∈ E .

Intuition. Material flowing through a transportation network, which originates

at source and is sent to sink.

2

Minimum-cut problem

An st-cut (cut) is a partition (A,B) of the nodes with s ∈ A and t ∈B.

Definition

Its capacity is the sum of the capacities of the edges from A to B.

cap(A,B) =∑

e out of A

c(e)

Definition

3

Minimum-cut problem

An st-cut (cut) is a partition (A,B) of the nodes with s ∈ A and t ∈B.

Definition

Its capacity is the sum of the capacities of the edges from A to B.

cap(A,B) =∑

e out of A

c(e)

Definition

4

Minimum-cut problem

Min-cut problem. Find a cut of minimum capacity.

5

Quiz 1

Which is the capacity of the given st-cut?

A. 11 (20 + 25− 8− 11− 9− 6)

B. 34 (8 + 11 + 9 + 6)

C. 45 (20 + 25)

D. 79 (20 + 25 + 8 + 11 + 9 + 6)

6

Maximum-flow problem

An st-flow(flow) f is a function that satisfies:

• For each e ∈ E : 0 ≤ f (e) ≤ c(e)

• For each v ∈ V − {s, t}:∑

e in to v

f (e) =∑

e out of v

f (e)

Definition

7

Maximum-flow problem

An st-flow(flow) f is a function that satisfies:

• For each e ∈ E : 0 ≤ f (e) ≤ c(e)

• For each v ∈ V − {s, t}:∑

e in to v

f (e) =∑

e out of v

f (e)

Definition

The value of a flow f is: val(f ) =∑

e out of s

f (e)−∑

e in to s

f (e)

Definition

8

Maximum-flow problem

Max-flow problem. Find a flow of maximum value.

9

Ford-Fulkerson Algorithm

9

Toward a max-flow algorithm

Greedy algorithm.

• Start with f (e) = 0 for each edge e ∈ E .

• Find an s t path P where each edge has f (e) < c(e).

• Augment flow along path P.

• Repeat until you get stuck.

10

Toward a max-flow algorithm

Greedy algorithm.

• Start with f (e) = 0 for each edge e ∈ E .

• Find an s t path P where each edge has f (e) < c(e).

• Augment flow along path P.

• Repeat until you get stuck.

11

Toward a max-flow algorithm

Greedy algorithm.

• Start with f (e) = 0 for each edge e ∈ E .

• Find an s t path P where each edge has f (e) < c(e).

• Augment flow along path P.

• Repeat until you get stuck.

12

Toward a max-flow algorithm

Greedy algorithm.

• Start with f (e) = 0 for each edge e ∈ E .

• Find an s t path P where each edge has f (e) < c(e).

• Augment flow along path P.

• Repeat until you get stuck.

13

Toward a max-flow algorithm

Greedy algorithm.

• Start with f (e) = 0 for each edge e ∈ E .

• Find an s t path P where each edge has f (e) < c(e).

• Augment flow along path P.

• Repeat until you get stuck.

14

Toward a max-flow algorithm

Greedy algorithm.

• Start with f (e) = 0 for each edge e ∈ E .

• Find an s t path P where each edge has f (e) < c(e).

• Augment flow along path P.

• Repeat until you get stuck.

15

Toward a max-flow algorithm

Greedy algorithm.

• Start with f (e) = 0 for each edge e ∈ E .

• Find an s t path P where each edge has f (e) < c(e).

• Augment flow along path P.

• Repeat until you get stuck.

16

Why the greedy algorithm fails

Q. Why does the greedy algorithm fail?

A. Once greedy algorithm increases flow on an edge, it never decreases it.

Ex. Consider flow network G .

• The unique max flow has f ∗(v ,w) = 0.

• Greedy algorithm could choose s → v → w → t as first augmenting path.

Bottom line. Need some mechanism to undo a bad decision.

17

Residual network

Original edge e = (u, v) ∈ E .

• Flow f (e).

• Capacity c(e)

Reverse edge e reverse = (v , u)

• Undo flow sent.

Residual capacity

cf (e) =

{c(e)− f (e) if e ∈ E

f (e) if e reverse ∈ E

Residual network Gf = (V ,Ef , s, t, cf )

• Ef = {e : f (e) < c(e)} ∪ {e reverse : f (e) > 0}.

• Key property: f ′ is a flow in Gf iff f + f ′ is a flow in G

18

Augmenting path

An augmenting path is a simple s t path in the residual network Gf .

Definition

The bottleneck capacity of an augmenting path P is the minimum

residual capacity of any edge in P.

Definition

19

Augmenting path

Key Property. Let f be a flow and let P be an augmenting path in Gf . After

calling f ′ ← Augment(f , c,P), the resulting f ′ is a flow and

val(f ′) = val(f ) + bottleneck(Gf ,P).

Augment(f ,c,P)

δ ← bottleneck capacity of augmenting path P;

for each edge e ∈ P do

if (e ∈ E) then f (e)← f (e) + δ;

elsef (e reverse )← f (e reverse )− δ

end

end

Return f ;

20

Network flow: quiz 2

Which is the augmenting path of highest bottleneck capacity?

1. A→ F → G → H

2. A→ B → C → D → H

3. A→ F → B → G → H

4. A→ F → B → G → C → D → H

21

Ford–Fulkerson algorithm

Ford–Fulkerson augmenting path algorithm.

• Start with f (e) = 0 for each edge e ∈ E .

• Find an s t path P in the residual network Gf .

• Augment flow along path P.

• Repeat until you get stuck.

Ford–Fulkerson(G)

for each edge e ∈ E dof (e)← 0

end

Gf ← residual network of G with respect to flow f ;

while there exists an s t path P in Gf do

f ← Augment(f ,c,P);

Update(Gf );

end

Return f ;

22

Max-Flow Min-Cut Theorem

22

Relationship between flows and cuts

Let f be any flow and let (A,B) be any cut. Then, the value of the

flow f equals the net flow across the cut (A,B).

val(f ) =∑out of A

f (e)−∑

e in to A

f (e)

Lemma

23

Relationship between flows and cuts

Let f be any flow and let (A,B) be any cut. Then, the value of the

flow f equals the net flow across the cut (A,B).

val(f ) =∑out of A

f (e)−∑

e in to A

f (e)

Lemma

24

Relationship between flows and cuts

Let f be any flow and let (A,B) be any cut. Then, the value of the

flow f equals the net flow across the cut (A,B).

val(f ) =∑out of A

f (e)−∑

e in to A

f (e)

Lemma

25

Network flow: quiz 3

Which is the net flow across the given cut?

1. 11 (20 + 25− 8− 11− 9− 6)

2. 26 (20 + 22− 8− 4− 4)

3. 42 (20 + 22)

4. 45 (20 + 25)

26

Relationship between flows and cuts

Let f be any flow and let (A,B) be any cut. Then, the value of the

flow f equals the net flow across the cut (A,B).

val(f ) =∑out of A

f (e)−∑

e in to A

f (e)

Lemma

Proof.

val(f ) =∑

e out of s

f (e)−∑

e in to s

f (e)

=∑v∈A

( ∑e out of v

f (e)−∑

e in to v

f (e)

)=

∑e out of A

f (e)−∑

e in to A

f (e).

27

Relationship between flows and cuts

Weak Duality Let f be any flow and (A,B) be any cut. Then, val(f ) ≤cap(A,B).

Theorem

Proof.val(f ) =

∑e out of A

f (e)−∑

e in to A

f (e)

≤∑

e out of A

f (e)

≤∑

e out of A

c(e)

= cap(A,B)

28

Certificate of optimality

Let f be a flow and let (A,B) be any cut. If val(f ) = cap(A,B), then

f is a max flow and (A,B) is a min cut.

Corollary

Proof.• For any flow f ′ : val (f ′) ≤ cap(A,B) = val(f ).

• For any cut (A′,B ′) : cap (A′,B ′) ≥ val(f ) = cap(A,B)

29

Max-flow min-cut theorem

Value of a max flow = capacity of a min cut.

Max-Flow Min-Cut Theorem

30

Max-flow min-cut theorem

Value of a max flow = capacity of a min cut.

Max-Flow Min-Cut Theorem

A flow f is a max flow iff no augmenting paths.

Augmenting path theorem

Proof. The following three conditions are equivalent for any flow f :

i. There exists a cut (A,B) such that cap(A,B) = val(f ).

ii. f is a max flow.

iii. There is no augmenting path with respect to f .

[i⇒ ii] This is the weak duality corollary.

31

Max-flow min-cut theorem

Value of a max flow = capacity of a min cut.

Max-Flow Min-Cut Theorem

A flow f is a max flow iff no augmenting paths.

Augmenting path theorem

Proof. The following three conditions are equivalent for any flow f :

i. There exists a cut (A,B) such that cap(A,B) = val(f ).

ii. f is a max flow.

iii. There is no augmenting path with respect to f .

[ii⇒ iii] We prove contrapositive: ¬ iii ⇒ ¬ii.

• Suppose that there is an augmenting path with respect to f .

• Can improve flow f by sending flow along this path.

• Thus, f is not a max flow.

32

Max-flow min-cut theorem

[ iii ⇒ i ]

• Let f be a flow with no augmenting paths.

• Let A be set of nodes reachable from s in residual network Gf .

• By definition of A : s ∈ A.

• By definition of flow f : t /∈ A.

val(f ) =∑

e out of A

f (e)−∑

e in to A

f (e)

=∑

e out of A

c(e)− 0

= cap(A,B)

33

Capacity-Scaling Algorithm

33

Analysis of Ford–Fulkerson algorithm (when capacities are integral)

Assumption. Every edge capacity c(e) is an integer between 1 and C .

Integrality invariant. Throughout Ford-Fulkerson, every edge flow f (e) and

residual capacity cf (e) is an integer.

Proof. By induction on the number of augmenting paths.

Ford–Fulkerson terminates after at most val (f ∗) ≤ nC

augmenting paths, where f ∗ is a max flow.

Theorem

Proof. Each augmentation increases the value of the flow by at least 1.

34

Analysis of Ford–Fulkerson algorithm (when capacities are integral)

The running time of Ford–Fulkerson is O(mnC).

Corollary

Proof. Can use either BFS or DFS to find an augmenting path in O(m) time.

There exists an integral max flow f ∗Integrality Theorem

Proof. Since Ford–Fulkerson terminates, theorem follows from integrality

invariant.

35

Ford–Fulkerson: exponential example

Q. Is generic Ford–Fulkerson algorithm poly-time in input size?

A. No. If max capacity is C , then algorithm can take ≥ C iterations.

• s → v → w → t

• s → w → v → t

• s → v → w → t

• s → w → v → t

• . . .

• s → v → w → t

• s → w → v → t

36

Network flow: quiz 4

The Ford–Fulkerson algorithm is guaranteed to terminate if the edge

capacities are . . .

A. Rational numbers.

B. Real numbers.

C. Both A and B.

D. Neither A nor B.

37

Choosing good augmenting paths

Use care when selecting augmenting paths.

• Some choices lead to exponential algorithms.

• Clever choices lead to polynomial algorithms.

Pathology. When edge capacities can be irrational, no guarantee that

Ford–Fulkerson terminates (or converges to a maximum flow)!

Goal. Choose augmenting paths so that:

• Can find augmenting paths efficiently.

• Few iterations.

38

Choosing good augmenting paths

Choose augmenting paths with:

• Max bottleneck capacity(“fattest”).

• Sufficiently large bottleneck capacity.

• Fewest edges.

39

Capacity-scaling algorithm

Overview. Choosing augmented paths with large bottleneck capacity.

• Maintain scaling parameter ∆.

• Let Gf (∆) be the part of the residual network containing only those

edges with capacity ≥ ∆.

• Any augmenting path in Gf (∆) has bottleneck capacity ≥ ∆.

40

Capacity-scaling algorithm

Capacity-Scaling(G)

for each edge e ∈ E dof (e)← 0

end

∆← largest power of 2 ≤ C ;

while ∆ ≥ 1 do

Gf (∆)← ∆-residual network of G with respect to flow f ;

while there exists an s t path P in Gf (∆) do

f ← Augment(f , c, P);

Update(G∆(f ));

end

∆ = ∆/2;

end

Return f ;

41

Capacity-scaling algorithm: proof of correctness

Assumption. All edge capacities are integers between 1 and C .

Invariant. The scaling parameter ∆ is a power of 2.

Proof. Initially a power of 2; each phase divides ∆ by exactly 2.

Integrality invariant. Throughout the algorithm, every edge flow f (e) and

residual capacity cf (e) is an integer.

Proof. Same as for generic Ford–Fulkerson.

42

Capacity-scaling algorithm: proof of correctness

If capacity-scaling algorithm terminates, then f is a max flow.

Theorem

Proof.

• By integrality invariant, when ∆ = 1 ⇒ Gf (∆) = Gf

• Upon termination of ∆ = 1 phase, there are no augmenting paths.

• Result follows augmenting path theorem.

43

Capacity-scaling algorithm: analysis of running time

There are 1 + blog2 Cc scaling phases.

Lemma 1

There are ≤ 2m augmentations per scaling phase.

Lemma 2

Let f be the flow at the end of a ∆-scaling phase.

Then, the max-flow value ≤ val(f ) + m∆.

Lemma 3

44

Capacity-scaling algorithm: analysis of running time

The capacity-scaling algorithm takes O(m2logC) time.

Theorem

Proof.

• Lemma 1+ Lemma 2 ⇒ O(m log C) augmentations.

• Finding an augmenting path takes O(m) time.

45

Capacity-scaling algorithm: analysis of running time

There are 1 + blog2 Cc scaling phases.

Lemma 1

Proof. Initially C/2 < ∆ ≤ C ; ∆ decreases by a factor of 2 in each iteration.

46

Capacity-scaling algorithm: analysis of running time

There are ≤ 2m augmentations per scaling phase.

Lemma 2

Proof.

• Let f be the flow at the beginning of a ∆-scaling phase.

• Lemma 3 ⇒ max-flow value ≤ val(f ) + m(2∆).

• Each augmentation in a ∆-phase increases val(f ) by at least ∆.

47

Capacity-scaling algorithm: analysis of running time

Let f be the flow at the end of a ∆-scaling phase.

Then, the max-flow value ≤ val(f ) + m∆.

Lemma 3

Proof.• We show there exists a cut (A,B) such that cap(A,B) ≤ val(f ) + m∆.

• Choose A to be the set of nodes reachable from s in Gf (∆).

• By definition of A : s ∈ A.

• By definition of flow f : t /∈ A.

val(f ) =∑

e out of A

f (e)−∑

e in to A

f (e)

≥∑

e out of A

(c(e)− ∆)−∑

e in to A

∆

≥∑

e out of A

c(e)−∑

e out of A

∆−∑

e in to A

∆

≥ cap(A,B)−m∆

48

Shortest Augmenting Paths

48

Shortest augmenting path

Q. How to choose next augmenting path in Ford–Fulkerson?

A. Pick one that uses the fewest edges.

Shortest-Augmenting-Path(G)

for each edge e ∈ E dof (e)← 0

end

Gf ← residual network of G with respect to flow f ;

while there exists an s t path in Gf do

P ← BFS((Gf ));

f ← Augment(f , c, P);

Update(Gf );

end

Return f ;

49

Shortest augmenting path: overview of analysis

The length of a shortest augmenting path never decreases.

Lemma 1

After at most m shortest-path augmentations, the length of a shortest

augmenting path strictly increases.

Lemma 2

The shortest-augmenting-path algorithm takes O(m2n) time.

Theorem

Proof.

• O(m) time to find a shortest augmenting path via BFS.

• There are ≤ mn augmentations

- at most m augmenting paths of length k ←− Lemma 1 + Lemma 2

- at most n − 1 different lengths

50

Shortest augmenting path: analysis

Given a digraph G = (V ,E) with source s, its level graph is defined by:

• `(v) = number of edges in shortest s v path.

• LG = (V ,EG ) is the subgraph of G that contains only those edges

(v ,w) ∈ E with `(w) = `(v) + 1.

Definition

51

Network flow: quiz 5

Which edges are in the level graph of the following digraph?

A. D → F

B. E → F

C. Both A and B.

D. Neither A nor B.

52

Shortest augmenting path: analysis

Given a digraph G = (V ,E) with source s, its level graph is defined by:

• `(v) = number of edges in shortest s v path.

• LG = (V ,EG ) is the subgraph of G that contains only those edges

(v ,w) ∈ E with `(w) = `(v) + 1.

Definition

Key property. P is a shortest s v path in G iff P is an s v path in LG .

53

Shortest augmenting path: analysis

The length of a shortest augmenting path never decreases.

Lemma 1

Proof.

• Let f and f ′ be flow before and after a shortest-path augmentation.

• Let LG and LG ′ be level graphs of Gf and Gf ′ .

• Only back edges added to Gf ′

(any s t path that uses a back edge is longer than previous length)

54

Shortest augmenting path: analysis

After at most m shortest-path augmentations, the length of a shortest

augmenting path strictly increases.

Lemma 2

Proof.

• At least one (bottleneck) edge is deleted from LG per augmentation.

• No new edge added to LG until shortest path length strictly increases.

55

Shortest augmenting path: review of analysis

The length of a shortest augmenting path never decreases.

Lemma 1

After at most m shortest-path augmentations, the length of a shortest

augmenting path strictly increases.

Lemma 2

The shortest-augmenting-path algorithm takes O(m2n) time.

Theorem

56

Shortest augmenting path: improving the running time

Note. Θ(mn) augmentations necessary for some flow networks.

• Try to decrease time per augmentation instead.

• Simple idea ⇒ O(mn2) [Dinitz 1970]

• Dynamic trees ⇒ O(mn log n) [Sleator–Tarjan 1983]

57

Augmenting-path algorithms: summary

year method # augmentations running time

1955 augmenting path nC O(mnC)

1972 fattest path m log(mC) O(m2 log n log(mC)

)1972 capacity scaling m logC O

(m2 logC

)1985 improved capacity scaling m logC O(mn logC)

1970 shortest augmenting path mn O(m2n

)1970 level graph mn O

(mn2

)1983 dynamic trees mn O(mn log n)

augmenting-path algorithms with m edges, n nodes, and integer capacities between 1 and C

58

Maximum-flow algorithms: theory highlights

year method worst case discovered by

1951 simplex O(mn2C

)Dantzig

1955 augmenting paths O(mnC) Ford–Fulkerson

1970 shortest augmenting paths O(mn2

)Edmonds–Karp, Dinitz

1974 blocking flows O(n3)

Karzanov

1983 dynamic trees O(mn log n) Sleator–Tarjan

1985 improved capacity scaling O(mn logC) Gabow

1988 push–relabel O(mn log

(n2/m

))Goldberg–Tarjan

1998 binary blocking flows O(m3/2 log

(n2/m

)logC

)Goldberg–Rao

2013 compact networks O(mn) Orlin

2014 interior-point methods O(mm1/2 logC

)Lee–Sidford

2016 electrical flows O(m10/7C1/7

)Madry

20xx ???

augmenting-path algorithms with m edges, n nodes, and integer capacities between 1 and C

59