expander graphs, randomness extractors and list-decodable codes
DESCRIPTION
Expander Graphs, Randomness Extractors and List-Decodable Codes. Salil Vadhan Harvard University Joint work with Venkat Guruswami (UW) & Chris Umans (Caltech). Connections in Pseudorandomness. List-Decodable Error-Correcting Codes. Pseudorandom Generators. [PV05,GR06]. This Work. - PowerPoint PPT PresentationTRANSCRIPT
Expander Graphs,Randomness Extractors
and List-Decodable Codes
Salil VadhanHarvard University
Joint work with Venkat Guruswami (UW) & Chris Umans (Caltech)
[GW94,WZ95,TUZ01,RVW00,CRVW02]
Connections in Pseudorandomness
RandomnessExtractors
Expander Graphs
List-DecodableError-Correcting
Codes
PseudorandomGenerators
Samplers
[Tre99,RRV99,ISW99,SU01,U02]
[Tre99,TZ01,TZS01,SU01]
[CW89,Z96]
This Work
[PV05,GR06]
This Work
Outline
• Expander Construction
• Application to Extractors
• Connections
• Conclusions
(Bipartite) Expander Graphs
Goals:• Minimize D• Maximize A• Minimize M
|(S)| A¢|S|D
NM
S, |S| K
Nonconstructive:• D = O(log(N/M)/)
• A = (1-)¢D
• M = (KD/
“(K,A) expander”
O(1) if M=N
log N)
if M·pN
=
Applications of Expanders• Fault-tolerant networks (e.g., [Pin73,Chu78,GG81])• Sorting in parallel [AKS83]• Complexity theory [Val77,Urq87]• Derandomization [AKS87,INW94,Rei05,…]• Randomness extractors [CW89,GW94,TUZ01,RVW00]• Ramsey theory [Alo86]• Error-correcting codes [Gal63,Tan81,SS94,Spi95,LMSS01]
• Distributed routing in networks [PU89,ALM96,BFU99].
• Data structures [BMRS00].
• Distributed storage schemes [UW87].
• Hard tautologies in proof complexity [BW99,ABRW00,AR01].• Other areas of Math [KR83,Lub94,Gro00,LP01]
Need explicit constructions (deterministic, time poly(log N)).
Advantage of Expansion (1-)¢D
• At least (1-2) D |S| elements of (S) areunique neighbors: touch exactly one edge from S
|(S)| (1-) D |S|D
NM
S, |S| K
x
• Fault tolerance: Even if an adversary removes most (say ¾) edges from each vertex, lossless expansion maintained (with =4)
Application to Data Structures [BMRS00]
• Goal: store small S½[N] s.t. can test membership by (probabilistically) reading 1 bit.
• Expansion (1-)¢D ) 9 0,1 assignment to [M] s.t. for every x2[N], a 1-O() fraction of neighbors have correct answer!
D
NM
S, |S| K |(S)| (1-)¢ D¢|S|
/2
000110110
Application to Data Structures [BMRS00]
Size: M=O(K¢ log N) with optimal expander• (K¢log N) necessary to represent set.• Perfect hashing: same size, but read O(log N)-bit word
D
NM
S, |S| K /2
000110110
Explicit Constructions
Nonconstructive O(log(N/M)) (1-)¢D O(KD
Ramanujan graphs[…LPS86,M88]
O(1) ¼ D/2[Kah94]
N
Zig-zag CRVW02] O(1) (1-)¢D N
Ta-Shma, Umans, Zuckerman [TUZ01]
polylog(N)quasipoly(log N)
(1-)¢D(1-)¢D
quasipoly(KD) poly(KD)
Our Result polylog(N) (1-)¢D poly(KD)
degree D expansion A |right-side| M
arbitrary constant. quasipoly(t)=exp(polylog t)
Our Result
Thm: For every N, K, >0, 9 explicit (K,A) expander with• degree D = poly(log N, 1/)
• expansion A = (1-)¢D
• #right vertices M = D2¢ K1.01.
|(S)| A¢|S|D
NM
S, |S| K
Our Construction
Left vertices = Fqn = polys of degree · n-1 over Fq
Degree = q
Right vertices = Fqm+1
(f,y) = y’th neighbor of f = (y, f(y), (fh mod E)(y), (fh2 mod E)(y), …, (fhm-1 mod E)(y))
where E(Y) = irreducible poly of degree n h = a parameter
Thm: This is a (K,A) expander with K=hm, A = q-hnm.
Setting Parameters
(f,y) = (y, f(y), (fh mod E)(y), …, (fhm-1 mod E)(y))
f = poly of degree · n-1, E = irreducible of degree n
N = Fqn , D = q, M = Fq
m+1
Thm: This is a (K,A) expander with K=hm, A = q-hnm.
Set h = poly(nm/)q = h1.01Then:
• D = q = poly(log N, 1/) • A = q-hnm ¸ (1-)¢ D
• M = qm+1 = q¢ (h1.01)m = D¢ K1.01
Rel’n to Parvaresh-Vardy Codes [PV05]
(f,y) = (y, f(y), (fh mod E)(y), …, (fhm-1 mod E)(y))
f = poly of degree · n-1, E = irreducible of degree n
Thm: This is a (K,A) expander with K=hm, A = q-hnm.
• (f,y) = (y, y’th symbol of PV encoding f)
• Proof of expansion inspired by list-decoding algorithm for PV codes.
List-Decoding View of Expanders
• For Tµ [M], define LIST(T) = {x2 [N] : (x)µT}
• Lemma: G is a (=K,A) expander iff for all Tµ [M] of size AK-1, we have |LIST(T)| · K-1
|(S)| A¢ KD
N
S, |S|=K
M
“(=K,A) expander”
Comparing List-Decoding Views
: [N] £ [D] ! [D] £ [M]
T µ [D] £ [M]
Object Interpretation x2 LIST(T) iff Decoding Problem
expanders x,y) = y’th nbr of x
8 y (x,y)2 T |T| < AK ) |LIST(T)| < K
list-decodable codes
x,y) = (y,ECC(x)y)
Pry [(x,y) 2 T]
¸ 1/M + T = {(y,ry)}) |LIST(T)| < K
Proof of Expansion
(f,y) = (y, f(y), (fh mod E)(y), …, (fhm-1 mod E)(y))
f = poly of degree · n-1, E = irreducible of degree n
Thm: For A=q-nmh and any K· hm, we have
Tµ Fqm+1 of size AK-1) |LIST(T)|· K-1
Proof Outline (following [S97,GS99,PV05]):• Find a low-degree poly Q vanishing on T.• Show that every f 2LIST(T) is a “root” of a related
polynomial Q’.
• Show that deg(Q’) · K-1
=
Proof of Expansion: Step 1
(f,y) = (y, f(y), (fh mod E)(y), …, (fhm-1 mod E)(y))
f = poly of degree · n-1, E = irreducible of degree n
Thm: For A=q-nmh, K= hm, |T|·AK-1) |LIST(T)|· K-1.
Step 1: Find a low-degree poly Q vanishing on T.
• Take Q(Y,Z1,…,Zm) to be of degree · A-1 in Y,degree · h-1 in each Zi.
• # coefficients = A K > |T| = # constraints ) nonzero solution
• WLOG E(Y) doesn’t divide Q(Y,Z1,…,Zm).
Proof of Expansion: Step 2
(f,y) = (y, f(y), (fh mod E)(y), …, (fhm-1 mod E)(y))
f = poly of degree · n-1, E = irreducible of degree n
Thm: For A=q-nmh, K= hm, |T|·AK-1) |LIST(T)|· K-1.
Step 1: 9 Q vanishing on T, deg · A-1 in Y, h-1 in Zi, E-Q
Step 2: Every f 2LIST(T) is a “root” of a related Q’.
f(Y) 2LIST(T)
) 8 y2 Fq Q(y, f(y), (fh mod E)(y), …, (fhm-1 mod E)(y)) = 0
) Q(Y, f(Y), (fh mod E)(Y), …, (fhm-1 mod E)(Y)) 0
) Q(Y, f(Y), f(Y)h, …, f(Y)hm-1) 0 (mod E(Y))
) Q’(f) = 0 in Fq[Y]/E(Y), where
Q’(Z) = Q(Y,Z,Zh,…,Zhm-1) mod E(Y)
Proof of Expansion: Step 3
(f,y) = (y, f(y), (fh mod E)(y), …, (fhm-1 mod E)(y))
f = poly of degree · n-1, E = irreducible of degree n
Thm: For A=q-nmh, K= hm, |T|·AK-1) |LIST(T)|· K-1.
Step 1: 9 Q vanishing on T, deg · A-1 in Y, h-1 in Zi, E-Q
Step 2: 8 f2LIST(T) Q’(f) = 0 where Q’(Z) = Q(Y,Z,Zh,…,Zhm-1) mod E(Y)
Step 3: Show that deg(Q’) · K-1
• Q’(Z) nonzero because Q(Y,Z1,….,Zm) not divisible by E(Y) & is of deg · h-1 in Zi
• deg(Q’(Z)) · h-1+(h-1)¢ h++(h-1)¢ hm-1 = hm-1 = K-1
Proof of Expansion: Wrap-Up
(f,y) = (y, f(y), (fh mod E)(y), …, (fhm-1 mod E)(y))
f = poly of degree · n-1, E = irreducible of degree n
Thm: For A=q-nmh, K= hm, |T|·AK-1) |LIST(T)|· K-1.
Step 1: 9 Q vanishing on T, deg · A-1 in Y, h-1 in Zi, E-Q
Step 2: 8 f2LIST(T) Q’(f) = 0 where Q’(Z) = Q(Y,Z,Zh,…,Zhm-1) mod E(Y)
Step 3: Show that deg(Q’) · K-1
Proof of Thm:
|LIST(T)| · deg(Q’) · K-1. ¥
Our Result
Thm: For every N, K, >0, 9 explicit (K,A) expander with• degree D = poly(log N, 1/)
• expansion A = (1-)¢D
• #right vertices M = D2¢ K1.01.
|(S)| A¢|S|D
NM
S, |S| K
Outline
Expander Construction
• Application to Extractors
• Connections
• Conclusions
Extractors: Original Motivation[SV84,Vaz85,VV85,CG85,Vaz87,CW89,Zuc90,Zuc91]
• Randomization is pervasive in CS– Algorithm design, cryptography, distributed computing, …
• Typically assume perfect random source.– Unbiased, independent random bits– Unrealistic?
• Can we use a “weak” random source?– Source of biased & correlated bits.– More realistic model of physical sources.
• (Randomness) Extractors: convert a weak random source into an almost-perfect random source.
Applications of Extractors
• Derandomization of (poly-time/log-space) algorithms [Sip88,NZ93,INW94, GZ97,RR99, MV99,STV99,GW02]
• Distributed & Network Algorithms[WZ95,Zuc97,RZ98,Ind02].
• Hardness of Approximation [Zuc93,Uma99,MU01,Zuc06]
• Data Structures [Ta02]
• Cryptography [BBR85,HILL89,CDHKS00,Lu02,DRS04,NV04]
• Metric Embeddings [Ind06]
• Def: A (k,)-extractor is Ext : {0,1}n £{0,1}d ! {0,1}m s.t.
8 k-source X, Ext(X,Ud) is -close to Um.
8x Pr[X=x] · 2-k
Extractors [NZ93]
d random bits
“seed”
• Optimal (nonconstructive): d = log(n-k)+2log(1/)+O(1)m = k+d-2log(1/)-O(1)
EXT
k-source of length n
m almost-uniform bits
in variationdistance
Our Result
d random bits
“seed”
EXT
k-source of length n
m almost-uniform bits
Thm: For every n, k, >0, 9 explicit (k,) extractor with seed length d=O(log(n/)) and output length m=.99k.
• Previously achieved by [LRVW03]
– Only worked for ¸ 1/no(1)
– Complicated recursive construction
Approach: Condensers [RR99,RSW00]
d random bits
“seed”
CON
k-source of length n
¼ k’-source of length m
Def: A k!k’ condenser is Con : {0,1}n £{0,1}d ! {0,1}m s.t.
8 k-source X, Con(X,Ud) -close to some k’-source.
• Can extract from output: easier if k’/m > k/n.
• Called lossless if k’=k+d.
2k
Lossless Condensers Expanders
Lemma [TUZ01]: Con : {0,1}n £{0,1}d ! {0,1}m is a k!k+d condenser iff it defines a (2k,(1-)¢2d) expander.
Proof ((): • Suffices to condense sources uniform on 2k strings.• Expansion ) can make 1-1 by moving fraction of edges
{0,1}n
{0,1}m
2d
¸ (1-) 2d¢ 2k
n-bit k-source
¼ m-bit (k+d)-source
d-bit seed CON
x
Con(x,y)
y
Our Condenser
Thm: For every N, K, >0, 9 explicit (K,A) expander with• degree D = poly(log N, 1/)
• expansion A = (1-)¢D
• #right vertices M = D2¢ K1.01. (f,y) = (y, f(y), (fh mod E)(y), …, (fhm-1 mod E)(y))
Thm: For every n, k, >0, 9 explicit k!k+d condenser w/
• seed length d = O(log n+log(1/))
• output length m=2d+1.01¢k
• Con(f,y) = (y, f(y), (fh mod E)(y), …, (fhm-1 mod E)(y))
Our Extractor
Condense: 9 explicit k!k+d condenser w/
• seed length d = O(log n+log(1/))
• output length m ¼ 1.01k
• Con(f,y) = (y, f(y), (fh mod E)(y), …, (fhm-1 mod E)(y))
Then Extract: apply extractor for min-entropy rate .99:
• Constant – Ext(x,y) = y’th vertex on expander walk specified by x.
– Extraction follows from Chernoff bound for expander walks [G98], via equivalence of extractors and samplers [Z96].
Our Extractor
Condense: 9 explicit k!k+d condenser w/
• seed length d = O(log n+log(1/))
• output length m ¼ 1.01k
• Con(f,y) = (y, f(y), (fh mod E)(y), …, (fhm-1 mod E)(y))
Then Extract: apply extractor for min-entropy rate .99:
• Arbitrary – Zuckerman’s extractor for constant min-entropy rate [Z96].
Variations on the Condenser
Thm: 9 explicit k!k+d condenser w/
• seed length d = O(log n+log(1/))
• output length m ¼ 1.01k
• Con(f,y) = (y, f(y), (fh mod E)(y), …, (fhm-1 mod E)(y))
Variations (lose constant fraction of min-entropy):
• “Repeated roots” [GS99] in analysis
– seed length d = log n+log(1/)+O(1)
– output length m = O(k ¢ log(n/))
Variations on the Condenser
Thm: 9 explicit k!k+d condenser w/
• seed length d = O(log n+log(1/))
• output length m ¼ 1.01k
• Con(f,y) = (y, f(y), (fh mod E)(y), …, (fhm-1 mod E)(y))
Variations (lose constant fraction of min-entropy):
• E(Y) = Yq-1 - , for primitive root [GR06]
) (fhi mod E)(y) = f (i y)
) univariate analogue of Shaltiel-Umans extractor [SU01].
Outline
Expander Construction
Application to Extractors
• Connections
• Conclusions
Comparing List-Decoding Views
: {0,1}n £{0,1}d ! {0,1}d £ {0,1}m
T µ {0,1}d£ {0,1}m N=2n,D=2d,…
Object Interpretation x2 LIST(T) iff Decoding Problem
expanders x,y) = y’th nbr of x
8 y (x,y)2 T |T| < AK ) |LIST(T)| < K
list-decodable codes
x,y) = (y,ECC(x)y)
Pry [(x,y) 2 T]
¸ 1/2m + T = {(y,ry)}) |LIST(T)| < K
extractors Pry [(x,y) 2 T]
¸ |T|/2m+d + 8 T |LIST(T)| < K
lossy condensers
Pry [(x,y) 2 T]
¸ |T|/2m+d + |T|· K’-1 ) |LIST(T)| · K
Outline
Expander Construction
Application to Extractors
Connections
• Conclusions
Conclusions
• List-decoding view ) best known constructions of– Highly unbalanced expanders– Lossless condensers– Randomness extractors
• Push it further?– Nonbipartite expanders– Direct construction of extractor– Extractors optimal up to additive constants– Better list-decodable codes