meta-algorithms vs. circuit lower boundskabanets/talks/cie-2012-short.pdf · meta-algorithms vs....
TRANSCRIPT
![Page 1: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/1.jpg)
Meta-Algorithms vs.
Circuit Lower Bounds
Valentine Kabanets
Simon Fraser University
Vancouver, Canada
![Page 2: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/2.jpg)
Algorithms vs Lower Bounds
ALGOMANIA SLOWLAND
SAT
PIT
![Page 3: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/3.jpg)
Algorithms Lower Bounds
Algorithms
Lower Bounds
Algorithms yield lower bounds. Lower bounds yield algorithms.
![Page 4: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/4.jpg)
Meta-Algorithms
Algorithms operating on algorithms
Meta-Algorithm Algorithm
![Page 5: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/5.jpg)
Examples of Meta-Algorithms
• Computability Theory :
Virus checker ,
Infinite-loop detector (aka Halting problem)
• Complexity Theory :
SAT ,
Polynomial Identity Testing (PIT)
![Page 6: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/6.jpg)
Circuit - SAT
x1 x2 x3 … xn
Given: poly(n)-size circuit C on n inputs. Decide: Is C satisfiable ?
Canonical NP -complete problem. [ Cook; Levin ]
![Page 7: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/7.jpg)
Polynomial Identity Testing
+
* 3
x1 x2 x3 … xn
Given: poly(n)-size arithmetic circuit C on n inputs (over integers). Decide: Is C 0 ?
Extra structure ( C is a polynomial ) makes PIT easier than UNSAT : PIT in BPP [ Schwartz, Zippel, DeMillo-Lipton, … ]
![Page 8: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/8.jpg)
Randomized PIT-Algorithm
+ * 3
x1 x2 x3 … xn
C(1,17, 9,…,26) = 0 ? 1, 17, 9, …., 26
C
Yes
If C 0, then always correct. Else, correct with high probability. [Schwartz-Zippel]
![Page 9: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/9.jpg)
Meta-Algorithms from
Circuit Lower Bounds :
“ Black-Box ” use of lower bounds
![Page 10: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/10.jpg)
Is randomness useful ?
Can we remove the need for random coins in algorithms, without much slowdown ?
NP
P
NP
BPP= P
BPP P
PIT
OR ? PIT
![Page 11: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/11.jpg)
Derandomization
Computational Hardness
Computational Randomness (pseudorandomness) [ Blum, Micali, Yao; Nisan & Wigderson, Babai et al., … ]
Hard Boolean function (truth table)
PRG
Hardness – into – Randomness Converter
![Page 12: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/12.jpg)
Pseudo-Random Generator (PRG)
PRG
random seed
pseudorandom string random string
look “indistinguishable” to any small circuit
![Page 13: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/13.jpg)
Incompressibility Argument
Each x Bad has “small” description relative to C : log |Bad| bits specifying the rank of x in Bad, plus the description of C
C
n-bit string
Bad ={ x |C(x) =0}
all n-bit strings
Any string incompressible relative to C is accepted by C.
![Page 14: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/14.jpg)
Incompressibility Argument
Each x Bad has “small” description relative to C
C
n-bit string
Bad
all n-bit strings
Any string incompressible relative to C is accepted by C.
High circuit complexity =
incompressibility relative to small circuits
![Page 15: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/15.jpg)
Hardness into Randomness
EXP
P BPP= P
Size(2o(n)) P
Hard language
PIT
EXP
ArithmSize(2o(n))
Hard polynomial
Quasi
P
PIT
EXP requires circuit size 2(n) BPP = P [Impagliazzo & Wigderson ]
EXP requires arithmetic circuit size 2(n) PIT in Time(npolylog n ) [ K. & Impagliazzo ]
![Page 16: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/16.jpg)
Non - “ Black-Box ” Use of
Circuit Lower Bounds
![Page 17: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/17.jpg)
Elusive Circuit Lower Bounds
NEXP
P
EXP
PSPACE
PH
NP
P
PolySize
…
TC0
ACC0
AC0
AC0: Constant depth, unbounded fan-in, poly-size ACC0: AC0 with MOD m gates TC0: AC0 with MAJ gates
NEXP in PolySize ?
![Page 18: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/18.jpg)
Natural Proofs [ Razborov, Rudich ]
A combinatorial property T of n-variable Boolean functions is natural against a class C if it is
– Constructive: “f in T” is decidable in poly(2n) time
– Large: |T| > 1/poly(2n) of all n-variable fns
– Useful against C: f in T f not in C
T
C
![Page 19: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/19.jpg)
Natural Proofs No Crypto
A natural proof of a circuit lower bound =
a proof using a natural property .
Theorem [Razborov, Rudich]:
A natural proof of a circuit lower bound against a class C
algorithm breaking every candidate PRG implemented in C
(i.e., class C cannot compute a strong PRG )
![Page 20: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/20.jpg)
Parity is not in AC0
20
NEXP
P
EXP
PSPACE
PH
NP
P
SIZE(poly)
…
TC0
ACC0
AC0
[FSS+84, Yao85, Hastad86]
Parity
AC0: Constant depth, unbounded fan-in, poly-size
![Page 21: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/21.jpg)
Switching Lemma
Given an AC0 circuit C(x1,…,xn),
• Choose a random subset of variables,
• Assign them to 0 or 1 randomly.
Very likely, the circuit becomes shallow.
[ Hastad ]
C not too large can make it a constant function, with some variables still free.
So, C can’t compute PARITY.
![Page 22: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/22.jpg)
AC0-functions are sparse
small AC0-circuit
sparse Fourier representation
[ Linial, Mansour, Nisan ]
Can approximately learn AC0 –computable functions.
![Page 23: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/23.jpg)
AC0-SAT faster than “brute-force”
DNF : 2n (1 - ) ANDs, with = 1/(log c + d log d)d-1
[ Impagliazzo, Mathews, Paturi ]
x1 x2 x3 … xn
AC0 circuit: size cn, depth d
ANDs have disjoint sets of satisfying assignments
SAT for such DNFs is easy !
![Page 24: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/24.jpg)
Circuit Lower Bounds from
Meta-Algorithms
![Page 25: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/25.jpg)
Circuit - SAT
x1 x2 x3 … xn
Given: poly(n)-size circuit C on n inputs. Decide: Is C satisfiable ?
![Page 26: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/26.jpg)
If pigs can fly ...
If SAT in P, then EXP requires circuit size > 2n/n
EXP SIZE(2n/n) P NP
[ Kannan ]
Facts: • Almost all Boolean functions f(x1,…,xn) require circuit size > 2n /n . • But, open if NEXP PolySize .
EXP
Size(2n /n) NP=P
SAT
![Page 27: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/27.jpg)
Can we use this approach to get any actual
circuit lower bounds ???
![Page 28: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/28.jpg)
Elusive Circuit Lower Bounds
NEXP
P
EXP
PSPACE
PH
NP
P
PolySize
…
TC0
ACC0
AC0
AC0: Constant depth, unbounded fan-in, poly-size ACC0: AC0 with MOD m gates TC0: AC0 with MAJ gates
![Page 29: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/29.jpg)
Elusive Circuit Lower Bounds
NEXP
P
EXP
PSPACE
PH
NP
P
PolySize
…
TC0
ACC0
AC0
AC0: Constant depth, unbounded fan-in, poly-size ACC0: AC0 with MOD m gates TC0: AC0 with MAJ gates
![Page 30: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/30.jpg)
Elusive Circuit Lower Bounds
NEXP
P
EXP
PSPACE
PH
NP
P
PolySize
…
TC0
ACC0
AC0
AC0: Constant depth, unbounded fan-in, poly-size ACC0: AC0 with MOD m gates TC0: AC0 with MAJ gates
NEXP in ACC0 ?
![Page 31: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/31.jpg)
Williams’ Circuit Lower Bound
Theorem 1: If can solve C – SAT slightly better than “brute-force”, then NEXP not in C – PolySize.
Theorem 2: ACC0 – SAT can be solved faster than “brute-force”.
Corollary: NEXP not in ACC0.
![Page 32: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/32.jpg)
C - Circuit Satisfiability
Theorem 1. There is k>0 such that :
If C – SAT for nc-size n-input circuits is in time O( 2n /nk ) for every c,
then NTime( 2n ) is not in C - PolySize.
Contrast: “Brute-force” C-SAT algorithm is in time 2n poly(nc).
![Page 33: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/33.jpg)
Circuit Lower Bounds from
PIT-Algorithms
![Page 34: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/34.jpg)
Polynomial Identity Testing (PIT)
+
* 3
x1 x2 x3 … xn
Given: poly(n)-size arithmetic circuit C on n inputs (over integers). Decide: Is C 0 ?
NP
P BPP P
PIT
![Page 35: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/35.jpg)
PIT easy NEXP hard
PIT in P NEXP requires superpoly-size arithmetic circuits
(i.e., NEXP not in PolySize, OR Permanent not in Arithmetic PolySize ) [ K., Impagliazzo ]
NEXP
ArithmSize(poly) P
PIT
![Page 36: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/36.jpg)
Important Polynomial Identities Permanent: For X = ( xi,j )n x n Permn (X) = xi,(i)
Defining Identities (expansion by minors): Permn (X) x1,j Permn-1 (X1,j ) . . . Perm1(x) x PIT easy efficient program-checker for Permanent
![Page 37: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/37.jpg)
Summary
• Good meta-algorithms (SAT, PIT, Learning)
strong circuit lower bounds
• Can get unconditional results on each side.
![Page 38: Meta-Algorithms vs. Circuit Lower Boundskabanets/talks/CiE-2012-short.pdf · Meta-Algorithms vs. Circuit Lower Bounds Valentine Kabanets Simon Fraser University Vancouver, Canada](https://reader035.vdocument.in/reader035/viewer/2022081407/60563db2e495627e04798098/html5/thumbnails/38.jpg)
Some Challenges
• Non-black-box “SAT-Algorithms Circuit Lower Bounds” conversions ?
[ BPP = P circuit l.b. for NEXP,
but PRG circuit l.b. for EXP ]
• Better circuit lower bounds for AC0 better SAT-algorithms ?
[ beyond the Switching Lemma ? ]