the poisson matrix and kronecker products · poisson problem the discrete poisson problem can be...

27
The Poisson matrix and Kronecker Products Tom Lyche University of Oslo Norway The Poisson matrix and Kronecker Products – p. 1/2

Upload: others

Post on 30-Jul-2020

42 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

The Poisson matrix and KroneckerProducts

Tom Lyche

University of Oslo

Norway

The Poisson matrix and Kronecker Products – p. 1/27

Page 2: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Plan for the dayNumerical solution of the Poisson problem

The finite difference scheme

Formulation as a matrix equation

Formulation in standard form Ax = b.

The Poisson matrix A is block tridiagonal

LU factorization of a block tridiagonal matrix

Solving the Poisson problem, complexity

Kronecker product

Relation to Poisson problem

Basic properties

Eigenvalues and eigenvectors

Eigenpairs for the Poisson matrix

The Poisson matrix and Kronecker Products – p. 2/27

Page 3: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Numerical solution of the Poisson Problem

−(u +u ) =f u=0

u=0

u=0

u=0 yyxx

m ∈ N, h = 1/(m + 1), vj,k ≈ u(jh, kh)

jh

(jh,kh)kh j+1,k

j,k+1

j-1,k

j,k-1

0 (m+1)h

0

(m+1)h

j,k

The Poisson matrix and Kronecker Products – p. 3/27

Page 4: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Discrete Equation

g′′(t) ≈ g(t − h) − 2g(t) + g(t + h)

h2

j,k j+1,k

j,k+1

j-1,k

j,k-1v

v v v

v

From differential equation for j, k = 1, 2, . . . , m

(−vj−1,k + 2vj,k − vj+1,k) + (−vj,k−1 + 2vj,k − vj,k+1) = h2fj,k (1)

From boundary conditions

vj,k = 0 if j = 0, m + 1 or k = 0, m + 1 (2)

The Poisson matrix and Kronecker Products – p. 4/27

Page 5: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Matrix Equation TV + V T = h2F

2 −1

−1 2

v11 v12

v21 v22

+

v11 v12

v21 v22

2 −1

−1 2

=1

9

f11 f12

f21 f22

T := T m = tridiagm(−1, 2,−1) ∈ Rm,m

V :=

v11 · · · v1m

......

vm1 · · · vmm

∈ Rm,m F :=

f11 · · · f1m

......

fm1 · · · fmm

∈ Rm,m

(1), (2) ⇔ TV + V T = h2F

(

TV + V T)

jk=

m∑

i=1

T jivik +m

i=1

vjiT ik = l.h.s in (1) = h2(F )jk

The Poisson matrix and Kronecker Products – p. 5/27

Page 6: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Convert TV + V T = h2F to Ax = b

>

v in gridj,k

1,1

1,2

1,3

2,1

2,2

2,3

3,1

3,2

3,3

1

4

7

2

5

8

3

6

9

x in gridi

j,k j+1,k

j,k+1

j-1,k

j,k-1

i i+1

i+m

i-1

i-m

4x-x

-x

-x

-x-v

v v -v

-v

- 4

-->

4xi − xi−1 − xi+1 − xi−m − xi+m = bi

The Poisson matrix and Kronecker Products – p. 6/27

Page 7: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Linear system Ax = b

For m = 3 or n = 9 we have the following linear system:

4x1 −x2 −x4 = h2f11

−x1 4x2 −x3 −x5 = h2f21

−x2 4x3 −x6 = h2f31

−x1 4x4 −x5 −x7 = h2f12

−x2 −x4 4x5 −x6 −x8 = h2f22

−x3 −x5 4x6 −x9 = h2f32

−x4 4x7 −x8 = h2f13

−x5 −x7 4x8 −x9 = h2f23

−x6 −x8 4x9 = h2f33

The Poisson matrix and Kronecker Products – p. 7/27

Page 8: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Blockstructure of A

A =

4 −1 0 −1 0 0 0 0 0

−1 4 −1 0 −1 0 0 0 0

0 −1 4 0 0 −1 0 0 0

−1 0 0 4 −1 0 −1 0 0

0 −1 0 −1 4 −1 0 −1 0

0 0 −1 0 −1 4 0 0 −1

0 0 0 −1 0 0 4 −1 0

0 0 0 0 −1 0 −1 4 −1

0 0 0 0 0 −1 0 −1 4

A =

T + 2I −I 0

−I T + 2I −I

0 −I T + 2I

Block tridiagonalThe Poisson matrix and Kronecker Products – p. 8/27

Page 9: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

LU -factorization of a tridiagonal matrix

d1 c1

a2 d2 c2

. . .. . .

. . .

an−1 dn−1 cn−1

an dn

=

1

l2 1

. . .. . .

ln 1

u1 c1

. . .. . .

un−1 cn−1

un

.

(3)

Note that L has ones on the diagonal, and that A and U have the samesuper-diagonal.By equating entries in (3) we find

d1 = u1, ak = lkuk−1, dk = lkck−1 + uk, k = 2, 3, . . . , n.

The Poisson matrix and Kronecker Products – p. 9/27

Page 10: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

The AlgorithmA = LU (LU -factorization)

Ly = b (forward substitution)

Ux = y (backward substituion)

u1 = d1, lk = ak

uk−1

, uk = dk − lkck−1, k = 2, 3, . . . , n.

y1 = b1, yk = bk − lkyk−1, k = 2, 3, . . . , n,

xn = yn/dn, xk = (yk − ckxk+1)/uk, k = n − 1, . . . , 2, 1.

This process is well defined if A is strictly diagonallydominant.

The number of arithmetic operations (flops) is O(n).

The Poisson matrix and Kronecker Products – p. 10/27

Page 11: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

LU of Block Tridiagonal Matrix

D1 C1

A2 D2 C2

. . .. . .

. . .Am−1 Dm−1 Cm−1

Am Dm

=

1L2 1

. . .. . .Lm 1

U1 C1

. . .. . .

Um−1 Cm−1

Um

.

U1 = D1, Lk = AkU−1

k−1, Uk = Dk − LkCk−1, k = 2, 3, . . . , m. (4)

To solve the system Ax = b we partition b conformally with A in the formbT = [bT

1 , . . . , bTm]. The formulas for solving Ly = b and Ux = y are as

follows:

y1 = b1, yk = bk − Lkyk−1, k = 2, 3, . . . , m,

xm = U−1m ym, xk = U−1

k (yk − Ckxk+1), k = m − 1, . . . , 2, 1.(5)

The solution is then xT = [xT1 , . . . , xT

m].

The Poisson matrix and Kronecker Products – p. 11/27

Page 12: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Poisson ProblemThe discrete Poisson problem can be solved writing it as a blocktridiagonal matrix.

We can also use Cholesky factorization of a band matrix.

The matrix has bandwidth d =√

n.

Cholesky requires O(2nd2) = O(2n2) flops.

The work using block tridiagonal solver is also O(n2).

Need better methods (next time).

The Poisson matrix and Kronecker Products – p. 12/27

Page 13: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Kronecker ProductDefinition 1. For any positive integers p, q, r, s we define the Kronecker product of two

matrices A ∈ Rp,q and B ∈ R

r,s as a matrix C ∈ Rpr,qs given in block form as

C =

Ab1,1 Ab1,2 · · · Ab1,s

Ab2,1 Ab2,2 · · · Ab2,s

......

...

Abr,1 Abr,2 · · · Abr,s

.

We denote the Kronecker product of A and B by C = A ⊗ B.

Defined for rectangular matrices of any dimension

# of rows(columns) = product of # of rows(columns) in A and B

The Poisson matrix and Kronecker Products – p. 13/27

Page 14: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Kronecker Product Example

T =

2 −1

−1 2

, I =

1 0

0 1

T ⊗ I =

T 0

0 T

=

2 −1 0 0

−1 2 0 0

0 0 2 −1

0 0 −1 2

I ⊗ T =

2I −I

−I 2I

=

2 0 −1 0

0 2 0 −1

−1 0 2 0

0 −1 0 2

The Poisson matrix and Kronecker Products – p. 14/27

Page 15: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Poisson Matrix = T ⊗ I + I ⊗ T

A =

T

T

T

+

2I −I 0

−I 2I −I

0 −I 2I

= T ⊗ I + I ⊗ T

Definition 2. Let for positive integers r, s, k, A ∈ Rr,r , B ∈ R

s,s and Ik be the

identity matrix of order k. The sum A ⊗ Is + Ir ⊗ B is known as the Kronecker sum of

A and B.

The Poisson matrix is the Kronecker sum of T with itself.

The Poisson matrix and Kronecker Products – p. 15/27

Page 16: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Matrix equation ↔ KroneckerGiven A ∈ R

r,r, B ∈ Rs,s, F ∈ R

r,s. Find V ∈ Rr,s such that

AV BT = F

For B ∈ Rm,n define

vec(B) :=

b1

b2

...

bn

∈ Rmn, bj =

b1j

b2j

...

bmj

jth column

Lemma 1.

AV BT = F ⇔ (A ⊗ B) vec(V ) = vec(F ) (6)

AV + V BT = F ⇔ (A ⊗ Is + Ir ⊗ B) vec(V ) = vec(F ). (7)

The Poisson matrix and Kronecker Products – p. 16/27

Page 17: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

ProofWe partition V , F , and BT by columns as V = (v1, . . . , vs),F = (f1, . . . , fs) and BT = (b1, . . . , bs). Then we have

(A ⊗ B) vec(V ) = vec(F )

Ab11 · · · Ab1s

......

Abs1 · · · Abss

v1

...

vs

=

f1

...

fs

⇔ A∑

j

bijvj = fi, i = 1, . . . , m

⇔ [AV b1, . . . , AV bs] = F ⇔ AV BT = F .

This proves (6)

The Poisson matrix and Kronecker Products – p. 17/27

Page 18: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Proof Continued

(A ⊗ Is + Ir ⊗ B) vec(V ) = vec(F )

⇔ (AV ITs + IrV BT ) = F ⇔ AV + V BT = F .

This gives a slick way to derive the Poisson matrix. Recall

AV + V BT = F ⇔ (A ⊗ Is + Ir ⊗ B) vec(V ) = vec(F )

Therefore,

TV + V T = h2F ⇔ (T ⊗ I + I ⊗ T ) vec(V ) = h2 vec(F )

The Poisson matrix and Kronecker Products – p. 18/27

Page 19: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Mixed Product Rule

(A ⊗ B)(C ⊗ D) = (AC) ⊗ (BD)

Proof If B ∈ Rr,t and D ∈ R

t,s for some integers r, t then

Ab1,1 · · · Ab1,t

......

Abr,1 · · · Abr,t

Cd1,1 · · · Cd1,s

......

Cdt,1 · · · Cdt,s

=

E1,1 · · · E1,s

......

Er,1 · · · Er,s

,

where for all i, j

Ei,j =

t∑

k=1

bi,kdk,jAC = (AC)(BD)i,j = ((AC) ⊗ (BD))i,j .

where in the last formula i, j refers to the ij-block in the Kroneckerproduct. �

The Poisson matrix and Kronecker Products – p. 19/27

Page 20: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Properties of Kronecker ProductsThe usual arithmetic rules hold. Note however that

(A ⊗ B)T = AT ⊗ BT

(A ⊗ B)−1 = A−1 ⊗ B−1 if A and B are nonsingular

A ⊗ B 6= B ⊗ A

(A ⊗ B)(C ⊗ D) = (AC) ⊗ (BD) (mixed product rule)

The mixed product rule is valid for any matrices as long as the productsAC and BD are defined.

The Poisson matrix and Kronecker Products – p. 20/27

Page 21: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Eigenvalues and EigenvectorsThe Kronecker product of two vectors u ∈ R

p and v ∈ Rr is a vector

u ⊗ v ∈ Rpr given by u ⊗ v =

[

uT v1, . . . , uT vr

]TSuppose now A ∈ R

r,r

and B ∈ Rs,s and

Aui = λiui, i = 1, . . . , r, Bvj = µjvj , j = 1, . . . , s,

then for i = 1, . . . , r, j = 1, . . . , s

(A ⊗ B)(ui ⊗ vj) = λiµj(ui ⊗ vj), (8)

(A ⊗ Is + Ir ⊗ B)(ui ⊗ vj) = (λi + µj)(ui ⊗ vj). (9)

Thus the eigenvalues of a Kronecker product(sum) are the products(sums) of the eigenvalues of the factors. The eigenvectors of a Kroneckerproduct(sum) are the products of the eigenvectors of the factors.

The Poisson matrix and Kronecker Products – p. 21/27

Page 22: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Proof of Eigen-formulaeThis follows directly from the mixed product rule. For (8)

(A ⊗ B)(ui ⊗ vj) = (Aui) ⊗ (Bvj) = (λiui) ⊗ (µjvj) = (λiµj)(ui ⊗ vj).

From (8)

(A ⊗ Is)(ui ⊗ vj) = λi(ui ⊗ vj), and (Ir ⊗ B)(ui ⊗ vj) = µj(ui ⊗ vj)

The result now follows by summing these relations. �

The Poisson matrix and Kronecker Products – p. 22/27

Page 23: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

The Poisson Matrix A = T ⊗ I + I ⊗ T

Recall eigenvalues and eigenvectors of the T matrix. Let h = 1/(m + 1).

1. We know that Tsj = λjsj for j = 1, . . . , m, where

sj = (sin (jπh), sin (2jπh), . . . , sin (mjπh))T , (10)

λj = 2 − 2 cos (jπh) =4 sin2 (jπh

2). (11)

2. The eigenvalues are distinct and the eigenvectors are orthogonal

sTj sk =

1

2hδj,k, j, k = 1, . . . , m. (12)

The Poisson matrix and Kronecker Products – p. 23/27

Page 24: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Eigenvalues/-vectors A = T ⊗ I + I ⊗ T

1. We have Axj,k = λj,kxj,k for j, k = 1, . . . , m, where

xj,k = sj ⊗ sk, (13)

sj = (sin (jπh), sin (2jπh), . . . , sin (mjπh))T , (14)

λj,k = 4 sin2 (jπh

2) + 4 sin2 (

kπh

2). (15)

2. The eigenvectors are orthogonal

xTj,kxp,q =

1

4h2δj,pδk,q, j, k, p, q = 1, . . . , m. (16)

3. A is symmetric AT = (T ⊗ I + I ⊗ T )T = T T ⊗ IT + IT ⊗ T T = A

4. A is positive definite (positive eigenvalues)

The Poisson matrix and Kronecker Products – p. 24/27

Page 25: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

A = T ⊗ I + I ⊗ T for m = 2

s1 =

sin(π3)

sin( 2π3

)

=

√3

2

1

1

, s2 =

sin( 2π3

)

sin( 4π3

)

=

√3

2

1

−1

λ1 = 4 sin2 (π

6) = 1, λ2 = 4 sin2 (

6) = 3

Axij = µijxij , µij = λi + λj , xij = si ⊗ sj i, j = 1, 2

µ11 = 2, µ12 = µ21 = 4, µ22 = 6.

s1 ⊗ s1 =3

4[1, 1, 1, 1]T , s1 ⊗ s2 =

3

4[1, 1,−1,−1]T ,

s2 ⊗ s1 =3

4[1,−1, 1,−1]T , s2 ⊗ s2 =

3

4[1,−1,−1, 1]T

xTj,kxp,q =

9

4δj,pδk,q, j, k, p, q = 1, 2.

The Poisson matrix and Kronecker Products – p. 25/27

Page 26: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

Check

A =

2 −1

−1 2

1 0

0 1

+

1 0

0 1

2 −1

−1 2

4 −1 −1 0

−1 4 0 −1

−1 0 4 −1

0 −1 −1 4

1

1

1

1

= 2

1

1

1

1

,

4 −1 −1 0

−1 4 0 −1

−1 0 4 −1

0 −1 −1 4

1

1

−1

−1

= 4

1

1

−1

−1

etc.

The Poisson matrix and Kronecker Products – p. 26/27

Page 27: The Poisson matrix and Kronecker Products · Poisson Problem The discrete Poisson problem can be solved writing it as a block tridiagonal matrix. We can also use Cholesky factorization

SummaryStudied the Poisson matrix A

Shown that A can be written as a Kronecker sum

studied properties of general Kronecker products and Kroneckersums

used this to derive properties of A

The Poisson matrix and Kronecker Products – p. 27/27