householder matrices

7/21/2019 Householder Matrices

1/10

Householder Matrices

DEFINITION. A linear transformation P of nis said to be a projection if

P2= P* = P.

PROPOSITION. If P is a projection, then 1 - P is a projection.

PROOF. We have that(1 - P)* = 1 - P

and

(1 - P)(1 - P) = 1 - (1)P - P(1) + PP = 1 - P - P + P = 1 - P.

Thus, the linear transformation 1 - P is a projection. Q.E.D.

EXAMPLE. Let a be a unit vector in n . Let a!a denote the linear transformation

a!a(x) = a.

Then a!a is a projection. This was demonstrated in Homework 8. The matrix of this linear transformation is A =

aa* since

Ax = aa*x = (a*x) a = a.

EXAMPLE. More generally, let a and b be vectors in n. Let a!b denote the linear transformation

a!b(x) = a.

The matrix of a!b is given by ab*.

DEFINITION. Let a be a unit vector in n. The linear transformations of the type Ua given by

Ua(x) = x - 2a

is called a Householder transformations.

PROPOSITION. Let a be a unit vector in n. Then the Householder transformation Uais a selfadjoint unitary

transformation.

PROOF. The Householder transformation Uacan be expressed as

Ua = (1 - Pa) - Pa.

where Pa= P is the projection

P(x) = a.

So we have that

Ua* = ((1 - P) - P)*

= (1 - P)* - P*

= (1 - P) - P = Ua

and

Ua*Ua = UaUa

= ((1 - P) - P)((1 - P) - P)

= (1 - P)2-P(1 - P) - (1 - P)P + P 2

= (1 - P) - (P - P2 ) - (P - P2) + P

= (1 - P) - (P - P) - (P - P) + P

= 1 - P + P = 1.

So we have that Uais a selfadjoint unitary. Q.E.D.

Householder Matrices page 1


2/10

PROPOSITION. Let w = (0, , 0, cos ", 0, , 0, sin ", 0, , 0). Then

A Householder transformation has a simple geometric description.

PROPOSITION. Let a be a unit vector. Then Ua(a) = - a and Ua(x) = x for every x perpendicular to a. Here x is

perpendicular to a if = 0.

PROOF. We have that

Ua(a) = a - 2a = a - 2a = -aand

Ua(x) = x - a = x

if x is perpendicular to a. Q.E.D.

The diagram for the Householder transformation is the following:

The geometry of the Householder transformation is useful in deriving formulae for moving vectors aroud via a

Householder transformation.

PROPOSITION. Let x and y be vectors of the same length in nwith x #y. Then Householder transformation Uw

with

w =x - y

||x - y||

takes x onto y.

REMARK. If x = y in the preceding formula, then the Householder transformation U0= 1 maps x onto y = x.

PROOF. We have that v = x + y is perpendicular to w since

=x - y

||x - y||, x + y =

1

||x - y|| + - - = 0.

Here we used the fact that

=

from the hypothesis on the length and the fact that

=

since x and y are in $n. So we have that

Uw(x + y) = x + y.

Thus, we get that



3/10

Uw(x) = Uwx - y

2+

x + y

2=

1

2Uw(x - y) +

1

2Uw(x + y) = -

1

2(x - y) +

1

2(x + y) = y.

Q.E.D.

A Householder matrix can be used to drive a matrix to so-called Hessenberg form.

DEFINITION. An n" n matrix A = (ai j) is said to be in (upper) Hessenberg form if ai j= 0 if i > j + 1.

The definition means that A is in Hessenberg from if everything below the subdiagonal of a matrix is 0. The

subdiagonal is the diagonal from the upper left hand corner to the lower right hand corner that lies immediately

below the main diagonal.

We now show that there is a Householder transformation that will "zero-out" the lower entries of a vector.

LEMMA (Zeroing Out Lemma). Let a = (a1, , an)Tbe a vector in !n. Then there is a Householder matrix

Uwsuch that

U

w

a = ||a|| e

1(respectively, Uwa = - ||a||e1).

PROOF. First we see that the vector w%= a - ||a|| e1is perpendicular to a + ||a||e1since

= ||a||2- ||a|| + ||a|| < e1, a> - ||a||

2 = 0

Setting

w = w%/||w%||,

if w%#0, we get that

Uw

(a) = 1

2(U

w(a - ||a||e1) +

1

2U

w(a + ||a||e1)) = -

1

2(a - ||a||e1) +

1

2(a + ||a||e1) = ||a||e1.

If w%= 0, then we take U equal to the identity.

A similar proof holds for the second case. Q.E.D.

REMARK. A same Lemma holds for complex vectors. We show that there is a complex number &of modulus 1

and a unit vector w with the propery

Uwa = &||a||e1.

Here we assume that there is some need for the zeroing out, i.e., that

|a2| + + |a

n| > 0.

For every complex number &of modulus 1, the vector

a - &||a||e1= (a1- &||a||, a2, , an)

is nonzero. We shall give the proper choice of&presently. We form the Householder transform Uw

where w is the

vector

w = a - &||a||e1

|| a - &||a||e1||.

By the geometry of the Householder matrix, we get

Uw

(a - &||a||e1) = - (a - &||a||e1)

and

Uw

(a + &||a||e1) = a + &||a||e1

since



4/10

= ||a||2- & +& - |&|

2 ||a||2.

Now we see that the proper choice of &is the complex number of modulus 1 with the property

& '$.

This is the number

&=

if &is nonzero or 1 if & = 0. Then we have that

& = &

so that

- & +& = 0.

Now we can compute Uw

a as

Uw

(a) = 1

2(U

w(a - &||a||e1) +

1

2U

w(a + &||a||e1)) = -

1

2(a - &||a||e1) +

1

2(a + &||a||e1) = &||a||e1.

To prove the second part we use the vector a + &||a||e1

THEOREM. Let A be an n "n complex matrix. There are Householder matrices U1, , U

n -1such that

Un -1

U2U

1A

is upper triangular.

PROOF. We do this in the ususal way by a deflation process. We examine the first column a1of A and find aHouseholder matrix U

1such that

U1a

1=

(

0

0

.

Here U1is found as in the previous part so that

U1a

1= ||a||e

1.

Then we have that

U1A = [U

1a

1, U

1a

2, , U

1a

n]

where aiis the ith column of i. So we have that

U1a =

b11 b12 b1n

0 b22 b2n

0 bnw b2nnn

= B.

We find an (n - 1) "(n - 1) dimensional Householder matrix U%2with

U%2B =

c22 c23 c2n

0 c33 c3n

0 cn3 cnn

.

We then have that

U%2= 1

n -1- 2a!a

for some a in subspace spanned by e2, , e

n. So we have that

U2=

1 0

0 U%2

= 1 - 2&a!a

is a Householder matrix. But now we have that



5/10

U2B =

1 0

0 U%2

b11 b12 b1n

0 b22 b2n

0 bn2 b2n

=

b11 b12 b1n

0 c22 c23 c2n

0 c33 c3n

0

0 0 cn3 cnn

.

So we have set up a recurrence and we can get to upper triangular form in n - 1 steps. Q.E.D.

Now we show that we can reach Hessenberg form with n - 1 successive transforms

ad Ui

where Uiare Householder matrices. We need a little modification of the preceding zeroing-out Lemma to preserve

the Hessenberg form under the right multiplication of Ui.

LEMMA. Let a = (a1, , an)Tbe a vector in n. Then there is a unit vector w (respectively, w%) of the form such

(0, w2, , w

n) such that the Householder matrix Uwsatisfies

Uwa = (a1, ||a%||, 0, , 0)T

(respectively, Uw%a = (a1, - ||a%||, 0, , 0)T) where a%= (a2, , an)).

PROOF. This a modification of the previous zeroing-out lemma. By the previous zeroing out lemma, there is a

unit vector

w%= ( w1, , wn - 1)

in n - 1such that

Uw%a%= (||a%||, 0, , 0)T.

However, the matrix

U =1 0

0 Uw%

is a Householder matrix for the unit vector

w = (0, w1, , wn - 1)

in n. To verify this we only need to compute as follows:

1 - 2(w!w) =

1 0 0

0 1 0

0 0 1

- 2

0

w1

wn - 1

0, w1, , wn - 1



6/10

=

1 0 0

0 1- 2w1w1 - 2w1wn - 1

0 - 2wn -1w1 1- 2wn -11wn - 1

=1 0

0 Uw%

.

Finally we have that

Ua =a1

Uw%a%= (a1, ||a%||, 0, , 0)

T.

Q.E.D.

Now we have the reduction to upper Hessenberg form.

THEOREM. Let A be an n " n matrix. Then there are at most n - 1 Householder matrices U1, , Un - 1

such that Un - 1Un - 2U1AU1Un - 2Un - 1is a Hessenberg matrix. If A is selfadjoint, then

Un - 1Un - 2U1AU1Un - 2Un - 1

is a tridiagonal matrix.

PROOF. We start a recurrence using the previous zeroing-out lemma. We can find a Householder matrix

U1= Uw1with w1 a unit vector of the form

w1= (0, w12, , w1n)

such that U1a1has the form

U1a1= (($ ($0, , 0)T.

Here a1is the first column of A and in general aiis column i of A. Since U1has the form

U1=

1 0

0 Uw%

we get that

U1A = [U1a1, , U1an] =

* * * *

* * * *

0 * * *

0 * * *

.

Consequently, we have that



7/10

U1AU

1= (U

1A)U

1=

* * * * ** * * * *0 * * * *

0 * * * *

0 * * * *

1 0

0 Uw%

=

b11 b12 b13 b14 b1n

b11 b22 b33 b34 b2n

0 b32 b33 b34 b3n

0 b42 b43 b44 b4n

0 bn2 bn3 bn4 bnn

.

Here (indicates that the place is filled with an entry which may not be 0.

Now we do the recurrence step l Using the matrix

B =

b 22 b 23 b 24 % b 2n

b 32 b 33 b 34 % b 3n

b 42 b 43 b 44 % b 4n

% % % % %

b n2 b n3 b n4 % b nn

,

there is a unit vector w%%' n -1with 0 in the first coordinate such that

Uw%%

b 11 b 12 b 13 b 14 % b 1n

b 21 b 22 b 23 b 24 %

b 2n

0 b 32 b 33 b 34 % b 3n

0 b 42 b 43 b 44 % b 4n

% % % % % %

0 b n2 b n3 b n4 % b nn

Uw%%=

c 11 c 12 c 13 % c 1n

c 21 c 22 c 23 % c 2n

0 c 32 c 33 % c 3n

% % % % %

0 c n2 c n3 % c nn

Since the first coordinate of w %%is 0, we have as before that

Uw%%=1 0

0 Uw%%%

and that the n "n matrix

U2=

1 0 0

0 1 0

0 0 Uw%%

is also a Householder matrix. Here w%%%are the last n - 2 coordinates of w%%. Pre and post multiplying U1AU

1by U

2

does not disturb the first column. So we have that

U2U

1AU

1U

2= U

2

b 11 b 12 b 13 b 14 % b 1n

b 21 b 22 b 23 b 24 % b 2n

0 b 32 b 33 b 34 % b 3n

0 b 42 b 43 b 44 %

b 4n

% % % % % %

0 b n2 b n3 b n4 % b nn

U2

=

b 11 c 12 c 13 c 14 % c 1n

c 21 c 22 c 23 c 24 % c 2n

0 c 32 c 33 c 34 % c 3n

0 0 c

43

c

44

% c

4n% % % % % %

0 0 c n3 c n4 % c nn

.

So now we get the proof of the theorem from recurrence.

If A s selfadjoint, then U1A has first column equal to (b

11$ b

21, 0 , 0)Tand has first row equal to the original

first row of A. So we have ththe first row of



8/10

U1AU

1= (U

1A)U

1

is equal to

(b11$ b

21, 0 , 0).

So we are heading to a tridiagonal matrix and the further iterations confirm this Q.E.D.

Homework

Find Householder matrices U1and U2such that U2U1AU1U2is tridiagonal where

A =

120 80 40 - 16

80 120 16 -40

40 16 120 -80

-16 -40 -80 120

.



9/10

Homework 9

Find Householder matrices U1and U2such that U2U1AU1U2is tridiagonal where

A =

120 80 40 - 16

80 120 16 -40

40 16 120 -80

-16 -40 -80 120

.

SOLUTION. We let

w%1=1

|| a%1- ||a%1||e1||a%1- ||a%1||e1

where a1is the first column of A and a%1= (80, 40, -16)T. We calculate

||a%1|| =

80

40

16

&

80

40

16

= 90.863

w%1=80

40

16

- 90.863

0

0

=10.863

40

16

and

w1=1

02

+ (10.863)2

+ 402+ (- 16)

2

0

10.863

40

16

=

0

0.24449

0.9003

0.36012

.

This gives the first Householder matrix as

U1=

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

2w1 w1T

=

1 0 0 0

0 0.88045 0.44023 0.17609

0 0.44023 0.62108 0.64843

0 0.17609 0.64843 0.74063

.

We then have

U1 A U1=

120 90.863 0 0

90.863 157.21 56.97 36.378

0 56.97 152.85 1.0047

0 36.378 1.0047 49.94

.

Now we find U2. We have that

56.97

36.378&

56.97

36.378= 67.59

and



10/10

0

0

56.97

36.378

0

0

67.594

0

=

0

0

124.56

36.378

so that

1

0

0

124.56

36.378

&

0

0

124.56

36.378

00

124.56

36.378

00

124.56

36.378

=

0 0 0 00 0 0 0

0 0 0.92141 0.26909

0 0 0.26909 0.078588

with

U

2

=

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

2

0 0 0 0

0 0 0 0

0 0 0.92141 0.26909

0 0 0.26909 0.078588

=

1 0 0 0

0 1 0 0

0 0 0.84282 0.53819

0 0 0.53819 0.84282

.

This produces

U2U1AU1U2=

120 90.863 0 0

90.863 157.21 67.594 0

0 67.594 123.95 46.257

0 0 46.257 78.836

which is tridiagonal.


householder matrices

Documents