On the Vorobyev method of moments

Zdenek StrakošCharles University in Prague and Czech Academy of Sciences

http://www.karlin.mff.cuni.cz/˜strakos

Conference in honor of Volker MehrmannBerlin, May 2015

Z. Strakoš 2

Thanks, bounds for 1955

Gene Golub, for pushing me to momentsBernd Fischer, for the beautiful book and much moreGérard Meurant, for many moment related joint interestsClaude Brezinski, for pointing out the work of VorobyevJorg Liesen, for sharing interests and many years of collaboration

Volker Mehrmann, for lasting inspiration and support in many ways.

1954: Operator orthogonal polynomials and approximation methodsfor determination of the spectrum of linear operators.

1958 (1965): Method of moments in applied mathematics.

Z. Strakoš 3

Broader context of 1955

● Euclid (300BC), Hippassus from Metapontum (before 400BC), ...... ,

● Bhascara II (around 1150), Brouncker and Wallis (1655-56): Three termrecurences (for numbers)

● Euler (1737, 1748), ...... , Brezinski (1991), Khrushchev (2008)

● Gauss (1814), Jacobi (1826), Christoffel (1858, 1857), ....... ,Chebyshev (1855, 1859), Markov (1884), Stieltjes (1884, 1893-94):Orthogonal polynomials, quadrature, analytic theory of continuedfractions, problem of moments, minimal partial realization,Riemann-Stieltjes integralGautschi (1981, 2004), Brezinski (1991), Van Assche (1993),Kjeldsen (1993),

● Hilbert (1906, 1912), ...... , Von Neumann (1927, 1932), Wintner (1929)resolution of unity, integral representation of operator functions inquantum mechanics

Z. Strakoš 4

Broader context of 1955

● Krylov (1931), Lanczos (1950, 1952, 1952c), Hestenes and Stiefel(1952), Rutishauser (1953), Henrici (1958), Stiefel (1958), Rutishauser(1959), ...... , Vorobyev (1954, 1958, 1965), Golub and Welsh (1968),..... , Laurie (1991 - 2001), ......

● Gordon (1968), Schlesinger and Schwartz (1966), Steen (1973),Reinhard (1979), ... , Horácek (1983-...), Simon (2007)

● Paige (1971), Reid (1971), Greenbaum (1989), .......

● Magnus (1962a,b), Gragg (1974), Kalman (1979), Gragg, Lindquist(1983), Gallivan, Grimme, Van Dooren (1994), ....

Who is Yu. V. Vorobyev?

All what we know can be found in Liesen, S, Krylov subspace methods,OUP, 2013, Section 3.7.

Z. Strakoš 5

Book (1958, 1965)

Z. Strakoš 6

The problem of moments in Hilbert space

Let z0, z1, . . . , zn be n+1 linearly independent elementsof Hilbert space V . Consider the subspace Vn generatedby all possible linear combinations of z0, z1, . . . , zn−1 and constructa linear operator Bn defined on Vn such that

z1 = Bnz0,

z2 = Bnz1,

...

zn−1 = Bnzn−2,

Enzn = Bnzn−1,

where Enzn is the projection of zn onto Vn .

Z. Strakoš 7

Approximation of bounded linear operators

Let B be a bounded linear operator in Hilbert space V . Choosing anelement z0 , we first form a sequence of elements z1, . . . , zn, . . .

z0, z1 = Bz0, z2 = Bz1 = B2z0, . . . , zn = Bzn−1 = Bnzn−1, . . .

For the present z1, . . . , zn are assumed to be linearly independent. Bysolving the moment problem we determine a sequence of operators Bn

defined on the sequence of nested subspaces Vn such that

z1 = Bz0 = Bnz0,

z2 = B2z0 = (Bn)2z0,

...

zn−1 = Bn−1z0 = (Bn)n−1z0,

Enzn = EnBnz0 = (Bn)nz0.

Z. Strakoš 8

Approximation of bounded linear operators

Using the projection En onto Vn we can write for the operatorsconstructed above (here we need the linearity of B )

Bn = En BEn .

The finite dimensional operators Bn can be used to obtain approximatesolutions to various linear problems. The choice of the elementsz0, . . . , zn, . . . as above gives Krylov subspaces that are closelyconnected with the application (described, e.g. by partial differentialequations).

Challenges: 1. convergence, 2. computational efficiency.

The most important classes of operators to study:

● completely continuous (compact),● self-adjoint.

Z. Strakoš 9

Inner product and Riesz map

Let V be a real (infinite dimensional) Hilbert space with the innerproduct

(·, ·)V : V × V → R, the associated norm ‖ · ‖V ,

V # be the dual space of bounded (continuous) linear functionals on Vwith the duality pairing

〈·, ·〉 : V # × V → R .

For each f ∈ V # there exists a unique τf ∈ V such that

〈f, v〉 = (τf, v)V for all v ∈ V .

In this way the inner product (·, ·)V determines the Riesz map

τ : V # → V .

Z. Strakoš 10

Operator formulation of the PDE BVP

Consider a PDE problem described in the form of the functional equation

Ax = b, A : V → V #, x ∈ V, b ∈ V #,

where the linear, bounded, and coercive operator A is self-adjoint withrespect to the duality pairing 〈·, ·〉 .

Standard approach to solving boundary-value problems using thepreconditioned conjugate gradient method (PCG) preconditions thealgebraic problem,

A, 〈b, ·〉 → A,b → preconditioning → PCG applied to Ax = b ,

i.e., discretization and preconditioning are often considered separately.

Z. Strakoš 11

2 Krylov subspaces in Hilbert spaces

Using the Riesz map τA : V → V , one can form for g ∈ V the Krylovsequence

g, τAg , (τA)2g , . . . in V

and define Krylov subspace methods in the Hilbert space operator setting(here CG) such that with r0 = b−Ax0 ∈ V # the approximations xn tothe solution x , n = 1, 2, . . . belong to the Krylov subspaces in V

xn ∈x0 + Kn(τA, τr0) ≡

x0 + span{τr0, τA(τr0), (τA)2(τr0), . . . , (τA)n−1(τr0)} .

Approximating the solution x = (τA)−1τb using Krylov subspaces isnot the same as approximating the operator inverse (τA)−1 by theoperators I, τA, (τA)2, . . . Vorobyev moment problem depends on τb !

Z. Strakoš 12

Vorobyev moment problem

Using the orthogonal projection En onto Kn with respect to the innerproduct (·, ·)V , consider the orthogonally restricted operator

τAn : Kn → Kn , τAn ≡ En (τA) En ,

by formulating the following equalities

τAn (τr0) = τA (τr0) ,

(τAn)2 τr0 = τAn (τA (τr0)) = (τA)2 τr0 ,

...

(τAn)n−1 τr0 = τAn ((τA)n−2 τr0) = (τA)n−1 τr0 ,

(τAn)n τr0 = τAn ((τA)n−1 τr0) = En (τA)n τr0 .

Z. Strakoš 13

Lanczos process and Jacobi matrices

The n-dimensional approximation τAn of τA matches the first2n moments

((τAn)ℓτr0, τr0)V = ((τA)ℓτr0, τr0)V , ℓ = 0, 1, . . . , 2n− 1 .

Denote symbolically Qn = (q1, . . . , qn) a matrix composed of thecolumns q1, . . . , qn forming an orthonormal basis of Kn determinedby the Lanczos process

τAQn = Qn Tn + δn+1 qn+1 eTn

with q1 = τr0/‖τr0‖V . We get (τAn)ℓ = Qn Tℓn Q∗

n, ℓ = 0, 1, . . .and the matching moments condition

e∗1 Tℓn e1 = q∗1(τA)ℓq1, ℓ = 0, 1, . . . , 2n− 1 ,

Z. Strakoš 14

Conjugate gradient method - first n steps

Tn =

γ1 δ2

δ2. . .

. . .. . .

. . .. . .

. . .. . . δn

δn γn

is the Jacobi matrix of the orthogonalization coefficients and the CGmethod is formulated by

Tnyn = ‖τr0‖V e1, xn = x0 + Qnyn , xn ∈ V .

Z. Strakoš 15

Spectral representation

Since τA is bounded and self-adjoint, its spectral representation is

τA =

∫ λU

λL

λ dEλ .

The spectral function Eλ of τA represents a family of orthogonalprojections which is

● non-decreasing, i.e., if µ > ν , then the subspace onto which Eµprojects contains the subspace into which Eν projects;

● EλL= 0, EλU

= I ;● Eλ is right continuous, i.e. lim

λ′→λ+

Eλ′ = Eλ .

The values of λ where Eλ increases by jumps represent theeigenvalues of τA , τAz = λz, z ∈ V .

Z. Strakoš 16

Representation of the moment problem

For the (finite) Jacobi matrix Tn we can analogously write

Tn =n

∑

j=1

θ(n)j s

(n)j (s

(n)j )∗ , λL < θ

(n)1 < θ

(n)2 < · · · < θ(n)

n < λU ,

and the operator moment problem turns into the 2n equationsfor the 2n unknowns θ

(n)j , ω

(n)j

n∑

j=1

ω(n)j {θ

(n)j }

ℓ = mℓ ≡

∫ λU

λL

λℓ dω(λ) , ℓ = 0, 1, . . . , 2n− 1 ,

where dω(λ) = q∗1dEλq1 represents the Riemann-Stieltjes distributionfunction associated with τA and q1 . The distribution function ω(n)(λ)approximates ω(λ) in the sense of the nth Gauss-Christoffel quadrature;Gauss (1814), Jacobi (1826), Christoffel (1858).

Z. Strakoš 17

Gauss-Christoffel quadrature

τA, q1 = τr0/‖τr0‖V ←→ ω(λ),

∫

f(λ) dω(λ)

↑ ↑

Tn, e1 ←→ ω(n)(λ),n

∑

i=1

ω(n)i f

(

θ(n)i

)

Using f(λ) = λ−1 gives

∫ λU

λL

λ−1 dω(λ) =n

∑

i=1

ω(n)i

(

θ(n)i

)

−1

+‖x− xn‖

2a

‖τr0‖2V

Continued fraction representation, minimal partial realization etc.

Z. Strakoš 18

References

● J. Málek and Z.S., Preconditioning and the Conjugate Gradient Methodin the Context of Solving PDEs. SIAM Spolight Series, SIAM (2015)

● J. Liesen and Z.S., Krylov Subspace Methods, Principles and Analysis.Oxford University Press (2013)

● Z.S. and P. Tichý, On efficient numerical approximation of the bilinearform c∗A−1b, SIAM J. Sci. Comput., 33 (2011), pp. 565-587

Non self-adjoint compact operators?

Z. Strakoš 19

Gauss quadrature in complex plane?

Vorobyev moment problem can be based on generalization of the Lanczosprocess to non self-adjoint operators with starting elements z0, w0 .Then, however, the tridiagonal matrix of the recurrence coefficients for theproperly normalized formal orthogonal polynomials (assuming, for thepresent, their existence) is complex symmetric but not (in general)Hermitian.

Generalization of the n-weight Gauss quadrature representation of theVorobyev moment problem that eliminates restrictive assumptions ondiagonalizability can be based on quasi-definite functionals; see theposter of Stefano Pozza and

S. Pozza, M. Pranic and Z.S., Gauss quadrature for quasi-definite linearfunctionals, submitted (2015).

Z. Strakoš 20

Conclusions

● Vorobyev work was built on the deep knowledge of the previous results.

● It is amazingly thorough and as to the coverage and references.

● Published in 1958 (1965), it was much ahead of time. It stimulates newdevelopments for the future.

Volker, Many Thanks and Congratulations!

Z. Strakoš 21

Whatever we try, does not work