Lower Bounds on the Communication of Distributed Graph Algorithms:

Progress and Obstacles

Rotem OshmanADGA 2013

Overview: Network ModelsCONGESTED CLIQUE

ASYNC MESSAGE-PASSING

LOCAL

CONGEST / general network

X

Talk Overview

I. Lower bound techniquesa. CONGEST general networks: reductions from 2-

party communication complexityb. Asynchronous message passing: reductions from

multi-party communication complexityII. Obstacles on proving lower bounds for the

congested clique

Communication Complexity

𝑋 𝑌

= ?

Example: DISJOINTNESS

𝑋⊆ {1 ,…,𝑛} 𝑌⊆ {1 ,…,𝑛}

𝑋∩𝑌=∅ ?

bitsneeded

[Kalyanasundaram and Schnitger, Razborov ’92]

DISJ :

Applying 2-Party Communication Complexity Lower Bounds

Textbook reduction:Given algorithm for solving task …

Solution for answer for DISJOINTNESS

bits

𝑌𝑋

Based on

Based on

Simulate

Example: Spanning Trees

• Setting: directed, strongly-connected network• Communication by local broadcast with

bandwidth • UIDs • Diameter 2• Question: how many rounds to find a rooted

spanning tree?

New Problem: PARTITION

• Inputs: , with the promise that

• Goal: Alice outputs ,Bob outputs such that partition .

𝑋 𝑌

The PARTITION Problem

• Trivial algorithm:– Alice sends her input to Bob– Alice outputs all tasks in her input– Bob outputs all remaining tasks

• Communication complexity: bits• Lower bound?

Reduction from DISJ to PARTITION

• Given input for DISJ :– Notice: iff – To test whether :• Try to solve PARTITION on • Ensure • Check if is a partition of : Alice sends Bob hash(), Bob

compares it to hash()

4 65

From PARTITION to Spanning Tree

a b

1 2 3

𝑋={1,2,3 } 𝑌={2,4,5,6 }

Given a spanning tree algorithm …

4 65

From PARTITION to Spanning Tree

a b

1 2 3

𝑋={1,2,3 } 𝑌={2,4,5,6 }

Simulating one round of :

Node a’s message

Node b’s message

4 65

From PARTITION to Spanning Tree

a b

1 2 3

𝑋={1,2,3 } 𝑌={2,4,5,6 }

When outputs a spanning tree:

From PARTITION to Spanning Tree

• If runs for rounds, we use bits

• One detail: randomness– Solution: Alice and Bob use public randomness

When Two Players Just Aren’t Enough

• No bottlenecks in the network

When Two Players Just Aren’t Enough

• Too much information revealed

Multi-Player Communication Complexity

• Communication by shared blackboard• Number-on-forehead• Number-in-hand

??

The Message-Passing Model

• players• Private channels• Private -bit inputs • Private randomness

• Goal: compute • Cost: total communication

The Coordinator Model

• players, one coordinator• The coordinator has no input

Message-Passing vs. Coordinator

≈

Prior Work on Message-Passing

• For players with -bit inputs…• Phillips, Verbin, Zhang ’12:– for bitwise problems (AND/OR, MAJ, …)

• Woodruff, Zhang ‘12, ‘13:– for threshold and graph problems

• Braverman, Ellen, O., Pitassi, Vaikuntanathan ‘13: for

Set Disjointness

Disj𝑛 ,𝑘    =¿ 𝑖=1¿𝑛¿ 𝑗=1¿𝑘 𝑋 𝑖𝑗¿

?

𝑋 1𝑋 2

𝑋 3

𝑋 4𝑋 5

Notation

• : randomized protocol– Also, the protocol’s transcript– : player ’s view of the transcript

• worst-case communication of

in the worst case

Entropy and Mutual Information

• Entropy:

• A lossless encoding of requires bits• Conditional entropy:

Entropy and Mutual Information

• Mutual information:

• Conditional mutual information:

Information Cost for Two Players[Chakrabarti, Shi, Wirth, Yao ’01], [Bar-Yossef, Jayram, Kumar, Sivakumar ‘04], [Braverman, Rao ‘10], …

Fix a distribution , • External information cost:

• Internal information cost:

Extension to the coordinator model:

Why is Info Complexity Nice?

• Formalizes a natural notion– Analogous to causality/knowledge

• Admits direct sum theorem:

“The cost of solving independent copies of problem is times the cost of

solving ”

Example

Example (Work in Progress)

• Triangle detection in general congested graphs• “Is there a triangle” =

”is a triangle”

Application of DISJ Lower Bound

• Open problem from Woodruff & Zhang ‘13:– Hardness of computing the diameter of a graph

• We can show: bits to distinguish diameter 3 from diameter

• Reduction from DISJ : given ,– Notice: disjoint iff

Application of DISJ Lower Bound

• Diameter • Diameter

𝑝1 𝑝3

𝑝2

𝑝4

1

32 4

5

6

𝑋 3

Part II: The Power of the Congested Clique

CONGESTED CLIQUE

Conversion from Boolean Circuit

• Suppose we have a Boolean circuit – Any type of gate, inputs– Fan-in – Depth = , #gates and wires =

• Step 1: reduce the fan-out to – Convert large fan-out gates to “copying tree”– Blowup: depth, size

• Step 2: convert to a layered circuit

Conversion from Boolean Circuit

• Now we have a layered circuit of depth and size = – With fan-in and fan-out

• Design a CONGEST protocol:– Fix partition of inputs of size each– Assign each gate to a random CONGEST node– Simulate the circuit layer-by-layer

Simulating a Layer

• If node “owns” gate on layer , it sends ’s output to the nodes that need it on layer

• Size of layer size of layer • What is the load on edge ?– For each wire from layer to layer ,

– At most wires in total– By Chernoff, w.h.p. the load is

Conversion from Boolean Circuit

• A union-bound finishes the proof• Corollary: explicit lower bounds in the

congested clique imply explicit lower bounds on Boolean circuits with polylogarithmic depth and nearly-linear size.

• Even worse:– Reasons to believe even bound hard

ConclusionCONGESTED CLIQUE

ASYNC MESSAGE-PASSING

LOCAL

CONGEST / general network

X