Extractors: applications and

constructionsAvi Wigderson

IAS, Princeton

Randomness

Extractors: original motivation

Unbiased,

independent

Probabilistic

algorithms

Cryptography

Game

Theory

Applications

:

Analyzed on

perfect

randomness biased,

dependentReality:

Sources of

imperfect

randomnes

sStock market

fluctuationsSun spots

Radioactive

decay

Extractor Theory

Applications of Extractors• Using weak random sources in prob algorithms [B84,SV84,V85,VV85,CG85,V87,CW89,Z90-91]• Randomness-efficient error reduction of prob

algorithms [Sip88, GZ97, MV99,STV99]• Derandomization of space-bounded algorithms

[NZ93, INW94, RR99, GW02]• Distributed Algorithms [WZ95, Zuc97, RZ98, Ind02].• Hardness of Approximation [Zuc93, Uma99, MU01]• Cryptography [CDHKS00, MW00, Lu02 Vad03]• Data Structures [Ta02]

Unifying Role of ExtractorsExtractors are intimately related to:

• Hash Functions [ILL89,SZ94,GW94]• Expander Graphs [NZ93, WZ93, GW94,

RVW00, TUZ01, CRVW02]• Samplers [G97, Z97]• Pseudorandom Generators [Trevisan 99, …]• Error-Correcting Codes [T99, TZ01, TZS01,

SU01, U02]

Unify the theory of pseudorandomness.

Definitions

Weak random sourcesDistributions X on {0,1}n with some entropy:

• [vN] sources: n coins of unknown fixed bias• [SV] sources: Pr[Xi+1 =1|X1=b1,…,Xi=bi] (δ, 1-δ)• Bit fixing: n coins, some good, some “sticky”• ….. • [Z] k-sources: H∞(X) ≥ k x Pr[X = x] 2-k

e.g X uniform with support 2k

{0,1}n

X

Randomness Extractors(1st attempt)

EXT

X k-source of length n

m almost-uniform bits

Impossible even if k=n-1 and m=1

“weak” random source Xk can be e.gn/2, √n, log n,…

Ext=0

Ext=1

{0,1}n

X

Extractors [Nisan & Zuckerman `93]

d random bits(short) “seed”

EXT

X k-source of length n

m almost-uniform bits

• Ext : {0,1}n x {0,1}d {0,1}m

• X has min-entropy k ( X is a k-source)• m ≤ k+d

Extractors [Nisan & Zuckerman `93]

EXT

k-source of length n

m bits-close to uniform

k-source X, | Ext(X,Ud) – Um|1 < but -fraction of y’s, | Ext(X, y) – Um|1 <

d random bits(short) “seed” {0,1}n

X

{0,1}m

Ext(X,y)

y {0,1}d

Extractors as graphsExtractors as graphs

k-source X |X|=2k

(k,)-extractor Ext: {0,1}n {0,1}d {0,1}m

{0,1}n {0,1}

m

xExt(x,y)

y

B

(X)

Discrepancy: For all but 2k of the x {0,1}n,

| |(X) B|/2d - |B|/2m |<

Sampling

Hashing

Amplification

Coding

Expanders

…

Probabilistic algorithms with weak random bits

k-source of length n

m random bits

EXTd random bits

Probabilistic algorithmInput

(upto )

Output

Error prob <δ+

Where from?

Try all possible2d strings. TakeMajority vote

Efficient?

Want: efficient Ext, small d, , large m

Extractors - Parameters

EXT

k-source of length n

m bits-close to uniform

• Goals: minimize d, , maximize m.• Non-constructive & optimal [Sip88,NZ93,RT97]:

– Seed length d = log(n-k) + 2 log 1/ + O(1).– Output length m = k + d - 2 log 1/ - O(1).

d random bits(short) “seed”

Extractors - Parameters

EXT

k-source of length n

m bits-close to uniform

• Goals: minimize d, maximize m.• Non-constructive & optimal [Sip88,NZ93,RT97]:

– Seed length d = log n + O(1).– Output length m = k + d - O(1).

d random bits(short) “seed”

• = 0.01• k n/2

Explicit ConstructionsNon-constructive & optimal [Sip88,NZ93,RT97]:

– Seed length d = log n + O(1).– Output length m = k + d - O(1).

[...B86,SV86,CG87, NZ93, WZ93, GW94, SZ94, SSZ95, Zuc96, Ta96, Ta98, Tre99, RRV99a, RRV99b, ISW00, RSW00, RVW00, TUZ01, TZS01, SU01, LRVW03,…]

New explicit constructions [GUV07, DW08] - Seed length d = O(log n) [even for =1/n] – Output length m = .99k + d

Applications

Probabilistic algorithms with weak random bits

k-source of length n X

m random bits

EXTd random bits

Probabilistic algorithmInput

(upto )

Output

Error prob <δ+

Try all 2d = poly(n)strings. TakeMajority vote

Efficient!

The error set B {0,1}m of alg is sampled accurately whp

Extractors as samplersn-bit string x

Ext(X,1)

EXT Efficient!k=2m

Ext(X,2) Ext(X,nc)m m m

S(x)={ }For every B {0,1}m, all but 2k of x {0,1}n :

| |S(x) B|/nc - |B|/2m |<

Note: x bad with prob < 2k/2n, n arbitrary

Extractors as list-decodable error-correcting codes [TZ]

Polynomial rate!Efficient encoding!!Efficient decoding?

n-bit string x

Ext(X,1)

EXT

Ext(X,2) Ext(X,D)1 bit 1 bit 1 bit

C(x)= ………

For z {0,1}D let Bz {0,1}d+1 be the set {(i,zi) : i [D] }List decoding: For every z, at most D2 of x have C(x) fall in (1/2 -)D hamming ball around z

c2c1

c3

{0,1}D

c8

c7

c6 c5

c4

c9

z

d = c log nD =2d = nc

C: {0,1}n {0,1}D

Beating e-value expansionTask: Construct an graph on [N] of minimal degree DEG s.t. every two sets of size K are connected by an edge. Any such graph: DEG > N/K Ramanujan graphs: DEG < (N/K)2

Random graphs: DEG < (N/K)1+o(1)

Extractors: DEG < (N/K)1+o(1)

K linear in N and constant DEG [RVW]We’ll see it for “moderate” K [WZ]

N

K

K

Extractors as graphs (again)Extractors as graphs (again)(k,.01)-extractor Ext: {0,1}n {0,1}d

{0,1}m

2k = K = M1+o(1) Ext: [N] x [D] [M]

2d = D < Mo(1)

[N] [M]

|(X)|> .99M

|X|=K

|X’|=K

Take G = Ext2 on [N]

DEG < (N/K)1+o(1)

Many edges betweenany two K-sets X,X’

Constructions

Expanders as extractors

Algx

r

{0,1}m

randomstrings

Thm [Chernoff] r1 r2….

rt independent (tm random bits) Thm [AKS] r1 r2

…. rt random G-path (m+ O(t) random bits)

Algx

rt

Algx

r1

Majority

G explicit expanderof const degree Bx

Pr[error] < 1/3

then Pr[error] = Pr[|{r1 r2…. rt }Bx}| > t/2] < exp(-t)

Expanders as extractors (k large)G expander graph of const degree on {0,1}m

B any subset, δ=|B|/2m

S = {r1 r2….

rt} a random G-path (n = m+ O(t) bits)

Thm [G] Pr[| δ - |SB|/t | > ] < exp(-2t)

Thm [Z] t=cm=2d, Ext : {0,1}n x {0,1}d {0,1}m Ext(r1 r2

…. rt ; i) = ri

is an (k=.99n, )–extractor of d=O(log n) seed

Condensers [RR99,RSW00,TUZ01]

d random bits seed

Con

X k-source of length n

.99k-source of length k

Sufficient to construct such condensers: from here we can use [Z] extractor

Mergers [T96]

d random bits seed

Mer

X1 X2 … XS

.9k-source

Some block Xi is random. The other Xj are correlated arbitrarily with it.Mer outputs a high entropy distribution.

X= n=ks k k … k

k

Mergers [T96]

d random bits seed

Mer

X1 X2 … XS

.9k-source

X= n=ks k k … k

k

XiFqk q ~ n100

Some Xi is random

[LRVW] Mer = a1X1+a2X2+…+asXs aiFq ( d=slog q ) Mer is a random element in the subspace spanned by Xi’s[D] It works! (proof of the Wolf conjecture). [DW] Mer = a1(y)X1+a2(y)X2+…+as(y)Xs yFq ( d=log q )

Mer is a random element in the curve through the Xi’s

The proof

Assume: E [|C(X) B|] > 2ε & B small

x1 x2 xi

xs

x1 x2 xi

xs

C(x)(Fq)k

B

Mer(x)

B

Prx[ |C(x) B|>ε ] >ε Prx[ Q(C(x)) 0 ] >ε

Deg(C) = s-1

Pr [ Q(xi) 0 ] >ε

Q 0 #

low deg Q:(Fq)k Fq Q(B) 0

Open Problems Find explicit extractors with

– Seed length d = log n + O(1).– Output length m = k + d - O(1).

Find explicit bipartitegraph, of constant deg

[N3] [N2]

|X|=N|Γ(X)|≥ N

Extractors as samplers

X k-source of length n

m random bits

EXTd random bits

Any set B {0,1}m

(upto )

WHP estimation error <

Try all 2d = poly(n)strings. Count the fraction falls in B

Efficient!

Given B {0,1}m Estimate |B|/2m