kolmogorov width of discrete linear spaces: an approach to matrix rigidity joint work with: alex...

Kolmogorov width of discrete linear spaces:

an approach to matrix rigidity

Joint work with: Alex Samorodnitsky (Hebrew University)

and Ilya Shkredov (Steklov Mathematical Inst.)

Sergey Yekhanin

Microsoft

Matrix rigidity

Def: An matrix is -rigid if for any matrix , where

for all we have

Theorem (Valiant’1977): If is -rigid; the linear transformation that maps a vector to

does not have an -size -depth linear circuit.

Lead to a long line of work trying to find explicit rigid matrices. [F’93,R’98,KR’98,L’01,APY’09,D’11,AC’13,…]

Matrix rigidity

Valiant’s reduction needs -rigid matrices.

• With high probability a random matrix is -rigid.

• The best explicit matrices are ?-rigid.

Matrix rigidity

Valiant’s reduction needs -rigid matrices.

• With high probability a random matrix is -rigid.

• The best explicit matrices are 0-rigid.

Matrix rigidity

Valiant’s reduction needs -rigid matrices.

• With high probability a random matrix is -rigid.

• The best explicit matrices are -rigid for . [Friedman’93, SSS’97].

• Untouched minor barrier.

Proof sketch of [SSS’97]:

1. Take a matrix where every minor is of full rank.

2. Observe that after changes per row there is a somewhat large untouched minor.

3. The size of this minor is a lower bound for the rank of the perturbed matrix.

Design matrices: candidates for rigidity over

Design matrix is a binary symmetric matrix, where for

• Every row has ones.

• Supports of every two distinct rows intersect by .

• Matrices can be obtained from hyper-plane vs. point incidence relations in projective geometries over finite fields.

• Rich combinatorial structure

¿𝑛1−2𝜖

𝑛1−𝜖

Goal

Establish -rigidity of matrices , where .

• Stronger rigidity does not follow from combinatorial properties alone.

• Such a result would already be far beyond “untouched minor barrier”.

• Would have some applications in complexity [R’89,SV’12].

The approach

Any rigidity proof needs to exhibit a property that is:

• Satisfied by all low rank matrices.

• Not satisfied by even after perturbations.

𝑉𝑚𝝅

Low rank matrices

Our property is “approximability”:

A matrix is “approximable” if after a certain particular embedding into its rows admit a non-trivial approximation by a low dimensional Euclidian space.

Approximability

Consider the embedding , where

For and integer :

.

The space maximizes the smallest projection of a vector from the set The measure is equivalent to Kolmogorov width of a set

𝑾

𝑣1𝑣2

𝑣3

Proof strategy

• Show that is small.

• Show that is robust under perturbations of the rows of

• Show that for low-dimensional -linear spaces , is large.

We write to denote both the matrix and the set of its rows.

Steps above imply that matrices have high rank even after perturbations.

Our strategy:

Proof strategy

• Show that is small.

• Show that is robust under perturbations of the rows of

• Show that for low-dimensional -linear spaces , is large.

We write to denote both the matrix and the set of its rows.

Steps above imply that matrices have high rank even after perturbations.

Our strategy:

Inapproximability of designs

Lemma: For , we have .

Lemma: For , we have

Lemma: For and matrices , where every row ofdifferes from the

corresponding row of in at most coordinates, we have

Proofs use combinatorial structure of and basic spectral arguments.

𝜖=1𝑚

The conjecture

Conjecture: There exist such that for every linear space for some , we have .

Theorem: The conjecture implies -rigidity of matrices , where

Proof:

• Assume for some we have

• Consider the -linear space

• By the Conjecture we have .

• However by our inapproximability results we have .

The Conjecture holds for linear spaces

Approximability of -linear spaces

Theorem: Let , we have .

Theorem: Let , we have .

Theorem: Let be a cut space; then the Conjecture holds for

One dimensional appoximations

Theorem: Let , we have .

Proof: 𝑛𝑖1

𝑤𝑖= min𝑒∈ 𝐿: 𝑖∈𝑠𝑢𝑝𝑝(𝑒)

𝑤𝑡 (𝑒)𝜇 (𝐿 )= ∑

𝑖 ∈[𝑛 ]𝑤 𝑖

−1

One dimensional space , where

• Thus for all

• Let

• Note that for all

• We have:

Thus it suffices to show that

One dimensional approximations

Lemma:

Proof:

• Consider the hyper-graph

Nodes are coordinates

Edges are supports of vectors

Color all nodes white

• Build a sequence , where

Repeat:

Pick white with the smallest value of

Consider a hyper-edge and

Add to the sequence

Color all nodes in from black.

Each step above increases by at most

Thus the process generates linearly independent

elements in

Summary

Need better approximability results for -linear spaces:

• Use more property of -linear spaces than just the triangular rank.

• Approximability for -subsets of -linear spaces rather than complete spaces.

A property that separates design matrices from subsets of low dimensional spaces:

• Design matrices are extremal with respect to .

• The property is robust to perturbations.

• Strong separations for low-dimensional approximations.

• Weaker results for high dimensional approximations.