How are we computing? Invalidation Certificates

Consider invalidating the constraints (prior info, and N dataset units)

The invalidation certificate is a binary tree, with L leaves. At the i’th leaf – coordinate-aligned cube

– Polynomial/rational functions (“surrogate models”) & error bounds, which satisfy

– sum of squares certificate proving the emptiness of

Moreover Caveat: with each Mj relatively complex, these error bounds are generally heuristic,

implicitly assuming regularity in Mj

Building Quadratic, rational surrogates on

Given L samples of M, namely

Relaxing the denominator condition number bound leads to a quasi-convex problem, solved with bisection and semidefinite programming.

– Richer approximation than quadratics only

– Parametrization is only twice as large (as quadratics)

– Non-explicit control of behavior off sample points (with κ)

peak fitting error

using rational, quadratic function

on samples

with condition number bound on denominator over the domain (not just sample points)

minimize

Quadratic, rational surrogates on

Given L samples of M, namely

Enforce using the S-procedure:

Find nonnegative and such that

Quadratic forms… nonnegative everywhere

Two (1+n)x(1+n) matrices that depend affinely on coefficients of

D and λ and τ must be positive semidefinite

Quadratic, rational surrogates on

Given L samples of M, namely

But ensures that , so

Quadratic, rational surrogates on

Given L samples of M, namely

For fixed t– linear constraints on (coefficients of) D and N

– Semidefinite constraints on (coefficients of) D and λ and τ

– linear constraints on λ and τ

– Check feasibility with, e.g., SeDuMi

Bisect on t to optimal

Prediction with Rational surrogates

Given 1+P models and data

“Bound”: Inner and Outer bounds to minimum and maximum value that can take on

Local search

S-procedure, weak duality

Prediction with rational surrogates

The prediction problem is

But so

Minimize such that the S-procedure proves

Fixed , SDP. Bisect to optimal outer bound

So, in this context, many computations involve proving empty intersections of systems of quadratic inequalities.