hardness of reconstructing multivariate polynomials. parikshit gopalan u. washington parikshit...

Hardness of Reconstructing Hardness of Reconstructing Multivariate Polynomials.Multivariate Polynomials.

Parikshit GopalanParikshit Gopalan U. Washington U. Washington

Subhash Khot Subhash Khot NYU/Gatech NYU/Gatech

Rishi Saket Rishi Saket Gatech/NYUGatech/NYU

Curve FittingCurve Fitting

Problem: Given data points, find a low degree polynomial that fits best.

Easy if there is a perfect fit.

Well studied problem …

Curve Fitting through the Curve Fitting through the agesages

Statistics: Least SquaresStatistics: Least Squares

Polynomial Reconstruction

Coding Theory

Computational Learning

Cryptography

PCPs

Pseudorandom-ness

The Reconstruction The Reconstruction ProblemProblem

Input:Input: Degree d. Degree d. PointsPoints ValuesValues

xx11 f(xf(x11))

xxii f(xf(xii))

xxmm f(xf(xmm))

Output:Output: A degree d polynomial that A degree d polynomial that best best fitsfits the data. the data.

In this talk:In this talk: Finite fields, Hamming Finite fields, Hamming distance.distance.

The Reconstruction The Reconstruction ProblemProblem

Input:Input: Degree Degree dd, set , set SS, values , values f(x)f(x) for for x x 22 S S..

Output:Output: A degree d polynomial that A degree d polynomial that best fitsbest fits the data.the data.

Parameters that matter:Parameters that matter:

1.1. Degree Degree dd, Field , Field FF..

2.2. Set Set SS..

3.3. How good is the best fit? (error-rate How good is the best fit? (error-rate ))

Algorithms for Algorithms for ReconstructionReconstruction

Univariate Case Univariate Case [Sudan, Guruswami-Sudan]:[Sudan, Guruswami-Sudan]:

Multivariate Case Multivariate Case [Goldreich-Levin, Goldreich-[Goldreich-Levin, Goldreich-Rubinfeld-Sudan, Arora-Sudan, Sudan-Trevisan-Rubinfeld-Sudan, Arora-Sudan, Sudan-Trevisan-Vadhan]:Vadhan]:

Can tolerate very high error rate Can tolerate very high error rate ..Are these algorithms optimal?Are these algorithms optimal?

Hardness Results: Univariate Hardness Results: Univariate CaseCase

Degree Degree dd polynomials, polynomials, nn points in points in FF..

[Guruswami-Vardy]:[Guruswami-Vardy]: NPNP-hard to tell if some -hard to tell if some degree degree dd poly. has poly. has d +2d +2 agreements. agreements.

[Guruswami-Sudan]:[Guruswami-Sudan]: Can tell if some degree Can tell if some degree dd poly. has poly. has (nd)(nd)0.50.5 agreement. agreement.

Hardness Results: Multivariate Hardness Results: Multivariate CaseCase

Linear polynomialsLinear polynomials overover FF22

[Hastad][Hastad]:: NPNP-hard to tell if-hard to tell if Some linear poly. satisfies Some linear poly. satisfies 1- 1- fraction of fraction of points.points. Every linear poly. satisfies less than Every linear poly. satisfies less than 0.5 + 0.5 + fraction of points.fraction of points.

Extends to any Extends to any FF and and d =1d =1. . Implies something for Implies something for d < Fd < F..

d d ¸̧ 2 2 overover F F22:: Nothing known. Nothing known.

Our ResultsOur Results

Over Over FF22 for any for any dd, NP-hard to tell whether, NP-hard to tell whether Some Some linearlinear polynomial satisfies polynomial satisfies 1- 1- fraction of points.fraction of points. Every Every degree ddegree d polynomial satisfies at polynomial satisfies at most most 1 -2 1 -2-d -d + + fraction of points. fraction of points.

SZ Lemma:SZ Lemma: For a degree d poly P For a degree d poly P 0 over 0 over FF22, ,

PrPrxx[ P(x) [ P(x) 0] 0] ¸̧ 2 2-d-d..

d=1d=1 ½ + ½ +

d=2d=2 ¾ + ¾ +

Our ResultsOur Results

Over Over FFqq for any for any dd, NP-hard to tell whether, NP-hard to tell whether Some Some linearlinear polynomial satisfies polynomial satisfies 1- 1- fraction of points.fraction of points. Every Every degree ddegree d polynomial satisfies at polynomial satisfies at most most c(d,q)c(d,q) + + fraction of points. fraction of points.

c(d,q)c(d,q): Schwartz-Zippel for polynomials of : Schwartz-Zippel for polynomials of total degree total degree dd over over FFqq..

Overview of ReductionOverview of Reduction

Reducing from Label-Cover.Reducing from Label-Cover. Dictatorship Testing.Dictatorship Testing. Consistency Testing.Consistency Testing. Putting it all together.Putting it all together.

Label CoverLabel Cover

Graph: G(V,E), |V| =n.

Labels: [k]

Edges: e ½ [k] £ [k]

Goal: Find a labeling satisfying all edges.Thm [PCP + Raz]: It is NP-hard to tell if

• Some labeling satisfies all edges.

• Every labeling satisfies · frac. of edges.

1

2n

3

X11 X1

2 … X1k

The ReductionThe Reduction

X21 X2

2 … X2k

X31 X3

2 … X3k

Xn1 Xn

2 … Xnk

Henceforth d =2, field = F2.

Constraints: Points in {0,1}nk + values.

Yes Case: Some L satisfies most constraints.

No Case: No Q satisfies many constraints.

The ReductionThe ReductionX1

1 X12 … X1

k

X21 X2

2 … X2k

X31 X3

2 … X3k

Xn1 Xn

2 … Xn

k

• If l(v) is a good labelling, then L = v Xv

l(v) will satisfy most points.

The ReductionThe ReductionX1

1 X12 … X1

k

X21 X2

2 … X2k

X31 X3

2 … X3k

Xn1 Xn

2 … Xn

k

• If l(v) is a good labelling, then L = v Xv

l(v) will satisfy most points.

• Any Q that does ¾ + gives a labelling satisfying ’fraction of edges.

Dictatorship:Dictatorship:

QQ11 = Q(X = Q(X1111,…,X,…,X11

kk,0,..,0).,0,..,0).

QQ11 looks like a Dictatorlooks like a Dictator X X11jj..

Will settle for small list.Will settle for small list.

Consistency:Consistency:

Some pair of labels in the list satisfy Some pair of labels in the list satisfy ..

3, 71, 99

17, 45


Constant independe

nt of k.

Dictatorship:Dictatorship:

QQ11 = Q(X = Q(X1111,…,X,…,X11

kk,0,..,0).,0,..,0).

QQ11 looks like a Dictatorlooks like a Dictator X X11jj..

Will settle for small list.Will settle for small list.Can enforce this for Can enforce this for frac. of vertices. frac. of vertices.



Can enforce this for all edges.Can enforce this for all edges.

3, 71, 99

17, 45


3, 71, 99

17, 45


If Q does ¾ +

•Small list for frac. of vertices.

•Consistency for all edges.

Assign random labels from list.

Satisfies constant fraction of edges.


Dictatorship Testing.Dictatorship Testing. Consistency Testing.Consistency Testing. Putting it all together.Putting it all together.

Dictatorship Testing for low-Dictatorship Testing for low-degree Polynomials.degree Polynomials.

Input:Input: Q(XQ(X11,…,X,…,Xkk)) of degree 2. of degree 2.

Goal:Goal: Design a test s.t Design a test s.t Every dictatorship Every dictatorship XXii passes w.p close to 1. passes w.p close to 1. If If QQ does better than ¾, it is close to a does better than ¾, it is close to a

dictatorship.dictatorship.

Test:Test: Pick a random point Pick a random point x x 22 {0,1} {0,1}kk..

Check if Check if Q(x) = yQ(x) = y..

Mini reconstruction problem!Mini reconstruction problem!

Small List


Dictatorships

Quadratic polys.

All polys.

Dictatorship Testing Dictatorship Testing [Hastad, Bourgain, MOO][Hastad, Bourgain, MOO]

Dictatorships

All polys.

Hard to do with just 2 queries.


Dictatorships

Quadratic polys.

Poly. is of low degree.

Allowed one query (!)

Dictatorship TestDictatorship Test

(0,…,0)

(1,…,1)

Dictatorship Test: Pick 2 {0,1}k from the -biased distribution. Check if Q() = 0.

• Uniform dist: Quadratic polys. are 3:1 balanced.

• -biased: Dictatorships are highly skewed.

• Is there a converse?

Each i =1 independentl

y. w.p


Dictatorship Test: Pick 2 {0,1}k from the -biased distribution. Check if Q() = 0. Xi passes w.p 1- .

XiXj passes w.p 1- 2.

X1(X1 + … + X) + X2(X + …) passes w.p 1 - 2


Dictatorship Test: Pick 2 {0,1}k from the -biased distribution. Check if Q() = 0.Define G(Q) to be the graph of Q.

Q = X1X2 + X2X3, G(Q) =

Thm: If Q passes w.p ¾ + , then G(Q) has no large matchings.

1

2

3

Dictatorship TestDictatorship TestDictatorship Test: Pick 2 {0,1}k from the -biased distribution. Check if Q() = 0.


1. Large matching:

Independent monomials.

2. Only small matchings:

Small vertex cover.X1L1 + X2L2


Thm: If Q does better than ¾, then G(Q) has no large matchings.

Q Q’Xi = 0 w.p 1- 2 Xi 2R {0,1}

c =? 0

• If G(Q) has a large matching, then Q’ 0 w.h.p.

• If Q’ 0, then c =1 w.p ¸ ¼ (SZ lemma).

• If Q does well, G(Q) has no large matchings.


Thm: If Q does better than ¾, then G(Q) has no large matchings.If G(Q) has a large matching, then Q’ 0 w.h.p. • Each edge survives w.p 42.

• Events for each matching edge are independent.


Dictatorship Test: Pick 2 {0,1}k from the -biased distribution. Check if Q() = 0.Define G(Q) to be the graph of Q.

Q = X1X2 + X2X3, G(Q) =


1

2

3

Small List: Vertex set of a maximal matching.

Dictatorship:Dictatorship: Assign a small list to a Assign a small list to a vertex.vertex.



3, 71, 99

17, 45


Consistency TestingConsistency Testing

l(x) = l(y)

Consistency TestingConsistency Testing

X1 X2 … Xk Y1 Y2 … Yk

l(x) = l(y)

Given Q(X1,…,Xk,Y1,…,Yk) s.t Q(Xi) and Q(Yj) both pass the dict. Test.

Want Q(X1,..,Xk,0,…,0) = Q(0,…,0,Y1,…,Yk).

Test: Q(r,0) = Q(0,r) for r 2R {0,1}k.

Two queries!

Consistency via FoldingConsistency via Folding

X1 X2 … Xk Y1 Y2 … Yk

l(x) = l(y)

•Yes case: Q = Xi + Yi for some i.

• All of them vanish over H = (r,r).

• Constant on each coset of H.

• Enforce this on Q even in the No case.

H


Def: Q is folded over subspace H µ {0,1}k if Q is constant on every coset of H.

Examples: Linear polys., juntas.

H

Thm: Q is folded over H iff for some nice basis (1,…,t,1,...,k-t), Q = R(1,…,t) is a t-junta for t = k – dim(H) In the nice basis (1,…,t,1,...,k-t)

is: coset of H, js: position in coset.

Template for FoldingTemplate for Folding

Want Q folded over a subspace H.

Compute nice basis (i, j).

Ask for R(1,…,t).

To test if Q(x) = y

o Let x = ( ); test R() = y.

For analysis: Rewrite R() as Q(x).

Now Q is folded.

H

{0,1}n/H


l(x) = l(y)

Fold over H = (r,r) for r 2 {0,1}k.

Polys. folded over H can be written as:

Q(X1,…,Xk,Y1,…,Yk) = R(X1 +Y1, …, Xk + Yk)

Gives Q(X1,…,Xk) = Q(Y1,…,Yk).List of Xis: Vertex set of maximal matching.Every two maximal matchings intersect.

Summary of ReductionSummary of Reduction

Each constraint gives H ½ {0,1}nk.

Fold over the span of all H.

Run Dict. test on every vertex.

No explicit consistency tests.

If Q passes w.p ¾ + ,fraction of vertices do well on Dict. test. Consistency for all edges by folding.

Projections …Projections …X1

1 X12 … X1

k

X21 X2

2 … X2k

X31 X3

2 … X3k

Xn1 Xn

2 … Xn

k

Can handle equality, permutations. Need perfect completeness: no UGC. Have to deal with $#@%! projections.

Projections …Projections …

Decoding is a vertex cover for G(Qi).

Need to show that every two vertex covers intersect.

Projections …Projections …Do every two vertex covers of G intersect?

No:

Projections …Projections …Do every two vertex covers of G intersect?

… but in any three VCs, some pair intersects.

No:

Main TheoremMain Theorem

Over Over FF22 for any for any dd, NP-hard to tell whether, NP-hard to tell whether Some Some linearlinear polynomial satisfies polynomial satisfies 1- 1- fraction of points.fraction of points. Every Every degree ddegree d polynomial satisfies at polynomial satisfies at most most 1 -2 1 -2-d -d + + fraction of points. fraction of points.

Better Hardness?Better Hardness?Problem: Problem: Can we improve soundness to Can we improve soundness to 0.5 0.5

+ + ??

Bottleneck:Bottleneck: Dictatorship test. Dictatorship test.

Present analysis is optimal in general:Present analysis is optimal in general:

Q = (XQ = (X11 + ..+ X + ..+ Xkk)(X)(Xk+1k+1 + … +X + … +X2k2k) ) passes passes w.p ¾.w.p ¾.

Can assume that Q is balanced.Can assume that Q is balanced.

Thank You!Thank You!

hardness of reconstructing multivariate polynomials. parikshit gopalan u. washington parikshit...

Documents

fraction of points

degree d poly p

polynomials of total

n points

low degree polynomial

data points

studied problem slide

reduction x