ryan o’donnell carnegie mellon university. part 1: a. fourier expansion basics b. concepts: bias,...
TRANSCRIPT
Ryan O’Donnell
Carnegie Mellon University
Part 1:
A. Fourier expansion basics
B. Concepts:
Bias, Influences, Noise Sensitivity
C. Kalai’s proof of Arrow’s Theorem
10 Minute Break
Part 2:
A. The Hypercontractive Inequality
B. Algorithmic Gaps
Sadly no time for:
Learning theory
Pseudorandomness
Arithmetic combinatorics
Random graphs / percolation
Communication complexity
Metric / Banach spaces
Coding theory
etc.
1A. Fourier expansion basics
f : {0,1}n {0,1}
f : {−1,+1}n {−1,+1}
ℝ3
(+1,+1,+1)
(−1,−1,−1)
(+1,+1,−1)
(+1,−1,+1)
(−1,+1,+1)
−1
−1
−1
+1
+1
+1
+1
−1
ℝ3
(+1,+1,+1)
(−1,−1,−1)
+1
+1
+1
+1−1
−1
−1
−1
ℝ3
(+1,+1,+1)
(−1,−1,−1)
−1
−1
−1
+1−1
−1
−1
−1
ℝ3
(+1,+1,+1)
(−1,−1,−1)
+1
+1
+1
+1+1
+1
+1
−1
ℝ3
(+1,+1,+1)
(−1,−1,−1)
+1
+1
+1
+1+1
+1
+1
+1
ℝ3
(+1,+1,+1)
(−1,−1,−1)
−1
−1
−1
−1−1
−1
−1
−1
ℝ3
(+1,+1,+1)
(−1,−1,−1)
−1
−1
+1
+1−1
+1
+1
−1
ℝ3
(+1,+1,+1)
(−1,−1,−1)
−1
+1
−1
+1+1
−1
+1
−1
ℝ3
(+1,+1,+1)
(−1,−1,−1)
+1
−1
−1
+1+1
+1
−1
−1
ℝ3
(+1,+1,+1)
(−1,−1,−1)
−1
−1
−1
+1
+1
+1
+1
−1
(+1,+1,+1)
+1+1
+1+1
+1+1+1+1
−1−1
−1−1
−1−1−1−1
+1
+1
+1
+1
−1
−1 −1
−1
(+1,+1,−1)
(+1,−1,−1)
=
=
Proposition:
Every f : {−1,+1}n {−1,+1} can be
expressed as a multilinear polynomial,
That’s it. That’s the “Fourier expansion” of f.
(uniquely)
(indeed, → ℝ)
Proposition:
Every f : {−1,+1}n {−1,+1} can be
expressed as a multilinear polynomial,
That’s it. That’s the “Fourier expansion” of f.
(uniquely)
(indeed, → ℝ)
⇓
Rest: 0
Why?
Coefficients encode useful information.
When?
1. Uniform probability involved
2. Hamming distances relevant
Parseval’s Theorem:
Let f : {−1,+1}n {−1,+1}.
Then
avg { f(x)2 }
“Weight” of f on S ⊆ [n]
=
{2}{1}
∅
{3}
{1,3}{1,2} {2,3}
{1,2,3}
{2}{1}
∅
{3}
{1,3}{1,2} {2,3}
{1,2,3}
{2}{1}
∅
{3}
{1,3}{1,2} {2,3}
{1,2,3}
{2}{1}
∅
{3}
{1,3}{1,2} {2,3}
{1,2,3}
{2}{1}
∅
{3}
{1,3}{1,2} {2,3}
{1,2,3}
{2}{1}
∅
{3}
{1,3}{1,2} {2,3}
{1,2,3}
1B. Concepts:
Bias, Influences, Noise Sensitivity
Social Choice:
Candidates ±1
n voters
Votes are random
f : {−1,+1}n {−1,+1}
is the “voting rule”
Bias of f:
avg f(x) = Pr[+1 wins] − Pr[−1 wins]
Fact:
Weight on ∅ = measures “imbalance”.
Influence of i on f:
Pr[ f(x) ≠ f(x(⊕i)) ]
= Pr[voter i is a swing voter]
Fact:
{2}{1}
∅
{3}
{1,3}{1,2} {2,3}
{1,2,3}
Maj(x1,x2,x3)
+1
+1
+1+1
−1
−1
−1−1
Infi(f) = Pr[ f(x) ≠ f(x(⊕i)) ]
+1
+1
+1+1
−1
−1
−1−1
Infi(f) = Pr[ f(x) ≠ f(x(⊕i)) ]
avg Infi(f) = frac. of edges which
are cut edges
LMN Theorem:
If f is in AC0
then avg Infi(f)
⇒ avg Infi(Parityn) = 1
⇒ Parity ∉ AC0
⇒ avg Infi(Majn) =
⇒ Majority ∉ AC0
KKL Theorem:
If Bias(f) = 0,
then
Corollary:
Assuming f monotone,
−1 or +1 can bribe o(n) voters
and win w.p. 1−o(1).
Noise Sensitivity of f at ϵ:
NSԑ(f) = Pr[wrong winner wins],
when each vote misrecorded w/prob ϵ
f(
f(
)
)
+ − + + − − + − −
− − + + + + + − −
Learning Theory principle:
[LMN’93, …, KKMS’05]
If all f ∈ C have small NSԑ(f)
then C is efficiently learnable.
{2}{1}
∅
{3}
{1,3}{1,2} {2,3}
[3]
Proposition:
for small ԑ,
with Electoral College:
ϵ 10
1
1C. Kalai’s proof of Arrow’s Theorem
Ranking 3 candidates
Condorcet [1775] Election:
=> (x_i, y_i, z_i) are Not All Equal (no 111 -1-1-1)
Condorcet: Try f = Maj. Outcome can be “irrational” A > B > C > A. [easy
eg]
Maybe some other f?
A > B?
B > C?
C > A?
Ranking 3 candidates
Condorcet [1775] Election:
=> (x_i, y_i, z_i) are Not All Equal (no 111 -1-1-1)
Condorcet: Try f = Maj. Outcome can be “irrational” A > B > C > A. [easy
eg]
Maybe some other f?
• • • • • •
A > B?
B > C?
C > A?
“C >
A >
B”
“A >
B >
C”
“B >
C >
A”
+
−
+
+
+
−
+
+
−
+
−
−
−
+
−
−
−
+
−
+
−
+
−
+
−
+
+
Ranking 3 candidates
Condorcet [1775] Election:
=> (x_i, y_i, z_i) are Not All Equal (no 111 -1-1-1)
Condorcet: Try f = Maj. Outcome can be “irrational” A > B > C > A. [easy
eg]
Maybe some other f?
• • • • • •
A > B?
B > C?
C > A?
“C >
A >
B”
“A >
B >
C”
“B >
C >
A”
f( )f( )f( )
=+=+=−
Society: “A > B > C”
+
−
+
+
+
−
+
+
−
+
−
−
−
+
−
−
−
+
−
+
−
+
−
+
−
+
+
Ranking 3 candidates
Condorcet [1775] Election:
=> (x_i, y_i, z_i) are Not All Equal (no 111 -1-1-1)
Condorcet: Try f = Maj. Outcome can be “irrational” A > B > C > A. [easy
eg]
Maybe some other f?
• • • • • •
A > B?
B > C?
C > A?
“C >
A >
B”
“A >
B >
C”
“B >
C >
A”
f( )f( )f( )
=+=+=−
Society: “A > B > C”
+
−
+
+
+
−
+
+
−
+
−
+
−
+
−
−
−
+
−
+
−
+
−
+
−
+
+
Ranking 3 candidates
Condorcet [1775] Election:
=> (x_i, y_i, z_i) are Not All Equal (no 111 -1-1-1)
Condorcet: Try f = Maj. Outcome can be “irrational” A > B > C > A. [easy
eg]
Maybe some other f?
• • • • • •
“C >
A >
B”
“A >
B >
C”
“B >
C >
A”
Society: “A > B > C”
A > B?
B > C?
C > A?
f( )f( )f( )
=+=+=+
+
−
+
+
+
−
+
+
−
+
−
+
−
+
−
−
−
+
−
+
−
+
−
+
−
+
+
Ranking 3 candidates
Condorcet [1775] Election:
=> (x_i, y_i, z_i) are Not All Equal (no 111 -1-1-1)
Condorcet: Try f = Maj. Outcome can be “irrational” A > B > C > A. [easy
eg]
Maybe some other f?
• • • • • •
“C >
A >
B”
“A >
B >
C”
“B >
C >
A”
Society: “A > B > C > A”?A > B?
B > C?
C > A?
f( )f( )f( )
=+=+=+
+
−
+
+
+
−
+
+
−
+
−
+
−
+
−
−
−
+
−
+
−
+
−
+
−
+
+
Arrow’s Impossibility Theorem [1950]:
If
f : {−1,+1}n {−1,+1} never gives
irrational outcome in Condorcet
elections,
then
f is a Dictator or a negated-Dictator.
Gil Kalai’s Proof [2002]:
• • • • • •
“C >
A >
B”
“A >
B >
C”
“B >
C >
A”
A > B?
B > C?
C > A?
f( )f( )f( )
=+=+=−
+
−
+
+
+
−
+
+
−
+
−
−
−
+
−
−
−
+
−
+
−
+
−
+
−
+
+
• • • • • •
“C >
A >
B”
“A >
B >
C”
“B >
C >
A”
A > B?
B > C?
C > A?
f( )f( )f( )
=+=+=−
+
−
+
+
+
−
+
+
−
+
−
−
−
+
−
−
−
+
−
+
−
+
−
+
−
+
+
Gil Kalai’s Proof:
Gil Kalai’s Proof:
Gil Kalai’s Proof, concluded:
f never gives irrational outcomes ⇒ equality
⇒ all Fourier weight “at level 1”
⇒ f(x) = ±xj for some j (exercise).
⇓
Guilbaud’s Theorem [1952]
Guilbaud’s Number ≈ .912
Corollary of “Majority Is Stablest” [MOO05]:
If Infi(f) ≤ o(1) for all i,
then
Pr[rational outcome with f]
Part 2:
A. The Hypercontractive Inequality
B. Algorithmic Gaps
2A. The Hypercontractive Inequality
AKA Bonami-Beckner Inequality
all use “Hypercontractive Inequality”
Hoeffding Inequality:
Let
F = c0 + c1 x1 + c2 x2 + ··· + cn xn,
where xi’s are indep., unif. random ±1.
Mean: μ = c0 Variance:
Hoeffding Inequality:
Let
F = c0 + c1 x1 + c2 x2 + ··· + cn xn,
Mean: μ = Variance:
Hypercontractive Inequality*:
Let
Then for all q ≥ 2,
Hypercontractive Inequality:
Let
Then F is a “reasonabled” random variable.
Hypercontractive Inequality:
Let
Then for all q ≥ 2,
Hypercontractive Inequality:
Let
Then
“q = 4” Hypercontractive Inequality:
Let
Then
“q = 4” Hypercontractive Inequality:
Let
all use Hypercontractive Inequality
just use “q = 4” Hypercontractive Inequality
“q = 4” Hypercontractive Inequality:
Let F be degree d over n i.i.d. ±1 r.v.’s.
Then
Proof [MOO’05]: Induction on n.
Obvious step.
Use induction hypothesis.
Use Cauchy-Schwarz on the obvious thing.
Use induction hypothesis.
Obvious step.
2B. Algorithmic Gaps
Opt
best poly-timeguarantee
ln(N)
“Set-Cover is NP-hard to
approximate to factor ln(N)”
Opt
LP-Rand-Roundingguarantee
ln(N)
“Factor ln(N) Algorithmic Gap
for LP-Rand-Rounding”
Opt(S)
LP-Rand-Rounding(S)
ln(N)
“Algorithmic Gap Instance S
for LP-Rand-Rounding”
Algorithmic Gap instances
are often “based on” {−1,+1}n.
Sparsest-Cut:
Algorithm: Arora-Rao-Vazirani SDP.
Guarantee: Factor
Opt = 1/n
Opt = 1/n
Opt = 1/n
Opt = 1/n
f(x) = sgn( )
Opt = 1/n
f(x) = sgn(r1x1 + ••• + rnxn)
ARV gets
Opt = 1/n
ARV gets
gap:
Algorithmic Gaps → Hardness-of-Approx
LP / SDP-rounding Alg. Gap instance
• n optimal “Dictator” solutions
• “generic mixture of Dictators” much worse
+ PCP technology
= same-gap hardness-of-approximation
Algorithmic Gaps → Hardness-of-Approx
LP / SDP-rounding Alg. Gap instance
• n optimal “Dictator” solutions
• “generic mixture of Dictators” much worse
+ PCP technology
= same-gap hardness-of-approximation
KKL / Talagrand Theorem:
If f is balanced,
Infi(f) ≤ 1/n.01 for all i,
then
avg Infi(f) ≥
Gap: Θ(log n) = Θ(log log N).
[CKKRS05]: KKL + Unique Games Conjecture
⇒ Ω(log log log N) hardness-of-approx.
2-Colorable 3-Uniform hypergraphs:
Input: 2-colorable, 3-unif. hypergraph
Output: 2-coloring
Obj: Max. fraction of legally
colored hyperedges
2-Colorable 3-Uniform hypergraphs:
Algorithm: SDP [KLP96].
Guarantee:
[Zwick99]
Algorithmic Gap Instance
Vertices: {−1,+1}n
6n hyperedges:{ (x,y,z) : poss. prefs in
a Condorcet
election}
(i.e., triples s.t. (xi,yi,zi) NAE for all i)
Elts: {−1,+1}n Edges: Condorcet votes (x,y,z)
2-coloring = f : {−1,+1}n → {−1,+1}
frac. legally colored hyperedges
= Pr[“rational” outcome with f]
Instance 2-colorable? ✔
(2n optimal solutions: ±Dictators)
Elts: {−1,+1}n Edges: Condorcet votes (x,y,z)
SDP rounding alg. may output
Random weighted majority also
rational-with-prob.-.912! [same CLT arg.]
f(x) = sgn(r1x1 + ••• + rnxn)
Algorithmic Gaps → Hardness-of-Approx
LP / SDP-rounding Alg. Gap instance
• n optimal “Dictator” solutions
• “generic mixture of Dictators” much worse
+ PCP technology
= same-gap hardness-of-approximation
Corollary of Majority Is Stablest:
If Infi(f) ≤ o(1) for all i,
then
Pr[rational outcome with f]
Cor: this + Unique Games Conjecture
⇒ .912 hardness-of-approx*
2C. Future Directions
Develop the “structure vs. pseudorandomness”
theory for Boolean functions.
