linear algebra - shandong university

LINEAR ALGEBRA

Lecture 8: Inner-Product Spaces

April 21, 2014

Jianfei Shen

School of Economics, Shandong University

Motivation

» We study vector spaces in which it makes sense to speak of˚

the length of a vector, and

the angle between two vectors.

,Jianfei Shen (Shandong University) 2/31

1 Euclidean Structure

2 Inner-Product Spaces

3 Orthonormal Bases


Norms

» The length of a vector x in R2,

denoted as kxk, is defined as its

distance to the origin.

» In Rn, the norm of x D .x1; : : : ; xn/is defined by

kxk Dqx21 C � � � C x2n:

» The norm is not linear on Rn.

0x1

x2

kxkDq

x21C x22

x

xy


Inner Product on Rn

» For x;y 2 Rn, the inner product of x and y , denoted hx;yi, is

defined by

hx;yi D x1y1 C � � � C xnyn:

EXAMPLE On R, the inner prod-

uct of x; y 2 R is defined by

hx; yi D xy:

» Fix Nx 2 R. Then h Nx; yi D Nxy is

linear in y.

» Fix Ny 2 R. Then hx; Nyi D x Ny is

linear in x. xy


Properties of Inner Product on Rn

» hx;xi D x21 C � � � C x2n D kxk2.» hx;xi > 0 for all x 2 Rn, with equality iff x D 0.

» If y 2 Rn is fixed, then the map x 7! hx;yi is linear.

» hx;yi D hy;xi for all x;y 2 Rn.

» Bilinearity:

hx C u;yi D hx;yi C hu;yi ;hx;y C vi D hx;yi C hx; vi :


Interpretation of Inner Product on Rn

» On Rn we have

ku � vk2 D hu � v;u � viD hu;ui � 2 hu; vi C hv; viD kuk2 � 2 hu; vi C kvk2:

(1)

» kuk, kvk and ku � vk have the same value in any Cartesian

coordinate system.

» It follows from (1) that hu; vi has the same value in any Cartesian

coordinate system.


» Chose a coordinate axes, the first one

through u, the second so that v is

contained in the plane spanned by the

first two axes.

» The coordinates of u and v in this new

coordinate system are

u D �kuk; 0; : : : ; 0�v D �kvk cos �; : : :

�:

x

y

z

u

v

kuk

v

x0

y0

kvk cos��

kvk

» Therefore,

hu; vi D kuk � kvk cos �; (2)

� the angle between u and v.


Law of Cosine

» The vectors x and y form a triangle.

» Relations (1) and (2) can be written as

kx�yk2 D kxk2Ckyk2�2kxkkyk cos �:0

x

y

x�

y

�

» If � D �2

, we get the Pythagorean Theorem.




3 Orthonormal Bases


Inner Product on Vector Spaces

DEFINITION An inner product on V is a function that takes each

ordered pair .u; v/ of elements of V to a number hu; vi 2 R and has

the following properties:

» hv; vi > 0 for all v 2 V ;

» hv; vi D 0 iff v D 0;

» huC v;wi D hu;wi C hv;wi for all u; v;w 2 V ;

» hav;wi D a hv;wi for all a 2 R and v;w 2 V ;

» hv;wi D hw; vi.

DEFINITION An inner-product space is a vector space V along

with an inner product on V .,

Jianfei Shen (Shandong University) 11/31

Norms

» For v 2 V , we define the norm of v, denoted kvk, by

kvk Dphv; vi:

» Then for every u; v 2 V ,

kuC vk2 D huC v;uC vi D kuk2 C 2 hu; vi C kvk2: (3)

» Eq. (3) is called the polar identity.


Orthogonal Vectors

» The greatest advantage of an inner product space is its

underlying concept of orthogonality.

DEFINITION Let V be an inner product space. We call u; v 2 Vorthogonal and write u ? v if hu; vi D 0.

» 0 ? v for every v 2 V .

» 0 is the only vector that is orthogonal to itself:

v ? v ” hv; vi D 0 ” v D 0:


Pythagorean Theorem

THEOREM If u and v are orthogonal vectors in V , then

kuC vk2 D kuk2 C kvk2 :

Proof. If u ? v, then

kuC vk2 D huC v;uC viD kuk2 C kvk2 C 2 hu; viD kuk2 C kvk2 : ut


Orthogonal Decomposition

» u; v 2 V with v ¤ 0.

» We seek w 2 V and a 2 R so that˚

u D avCw

w ? v

» Let u D avC .u � av/˜w

. So w ? v implies

u

U

v

av

w

hu � av; vi D 0 H) a D hu; viıkvk2:» Therefore,

u D hu; vikvk2 vC

u � hu; vikvk2 v

!: (6.5)


Example

» Let V D R2 and u D .1; 1/.» If v D .x; y/ 2 R2, then a D xCy

x2Cy2.

�20

2 �2

0

2�2

0

2

xy

a

�20

2 �2

0

2�2

0

2

xy

a


Cauchy-Schwarz Inequality

THEOREM If u; v 2 V , thenˇ̌hu; viˇ̌ 6 kuk kvk. This inequality is an

equality if and only if one of u, v is a scalar multiple of the other.

Proof. We suppose that v ¤ 0.

» The orthogonal decomposition: u D hu;vi

kvk2 vCw, where hv;wi D 0.

» By the Pythagorean theorem:

kuk2 D hu; vikvk2 v

2

C kwk2 D hu; vi2

kvk2 C kwk2 >

hu; vi2kvk2 :

» The Cauchy-Schwarz inequality holds with equality iff w D 0 iff u

is a multiple of v. ut


u

U

v

av

w

ˇ̌hu; viˇ̌ < kuk � kvk

v

ˇ̌hu; viˇ̌ D kuk � kvk


Example

» Let V D R2 and u D .1; 2/.» Then

ˇ̌hu; viˇ̌ Djx C 2yj and

kukkyk Dq5.x2 C y2/. �2

02 �2

0

20

5

xy


Triangle Inequality

THEOREM If u; v 2 V , then kuC vk 6 kuk C kvk. This inequality

is an equality if and only if one of u, v is a nonnegative multiple of

the other.

Proof. We have

kuC vk2 D huC v;uC viD kuk2 C 2 hu; vi C kvk26 kuk2 C 2 kuk kvk C kvk2

D �kuk C kvk�2 : u

vuC

v

» The triangle inequality holds with equality iff hu; vi D kuk kvk iff

one of u, v is a nonnegative multiple of the other.


Example

» Let V D R2 and u D .1; 1/.» Then

kuC vk Dq.1C x/2 C .1C y/2

and

kuk C kyk Dp2C

qx2 C y2.

�20

2 �2

0

20

2

4

xy


Parallelogram Equality

THEOREM If u; v 2 V , then

kuC vk2 C ku � vk2 D 2�kuk2 C kvk2

�:

Proof. The polar identity (3) implies that

kuC vk2 D kuk2 C 2 hu; vi C kvk2ku � vk2 D kuk2 � 2 hu; vi C kvk2:

Now add. ut

u

v

uC vu �v




3 Orthonormal Bases


Orthonormal Bases

DEFINITION A list of vectors is called orthonormal if the vectors

in it are pairwise orthogonal and each vector has norm 1.

» A list .e1; : : : ; em/ of vectors in V

is orthonormal if

˝ej ; ek

˛ D˚0 if j ¤ k1 if j D k:

» The standard basis in Rn is

orthonormal.

x

y

z

e1

e2

e3


Properties of Orthonormal Vectors

PROPOSITION If .e1; : : : ; em/ is an orthonormal list of vectors in

V , then for all a1; : : : ; am 2 R,

ka1e1 C � � � C amemk2 D a21 C � � � C a2m:

Proof. We apply the Pythagorean Theorem repeatedly:

» ha1e1; a2e2 C � � � C amemi D a1a2 he1; e2i C � � � C a1am he1; emi D 0» Hence, .a1e1/ ? .a2e2 C � � � C amem/

» We now have

ka1e1 C .a2e2 C � � � C amem/k2 D a21 C ka2e2 C � � � C amemk2D a21 C a22 C ka3e3 C � � � C amemk2D a21 C � � � C a2m: ut


Orthonormal Vectors and Linear Independence

PROPOSITION Every orthonormal list of vectors is linearly inde-

pendent.

Proof. Suppose .e1; : : : ; em/ is orthonormal and a1; : : : ; am 2 R

are such that

a1e1 C � � � C amem D 0:

Then a21 C � � � C a2m D 0 H) a1 D � � � D am D 0. ut


Orthonormal Bases

DEFINITION An orthonormal basis (ONB) of V is an orthonor-

mal list of vectors in V that is also a basis of V .

» The standard basis .e1; : : : ; en/ is an ONB of Rn.

» Every orthonormal list of vectors in V with length dim.V / is an

ONB of V .

» For instance, the following list �1

2;1

2;1

2;1

2

�;

�1

2;1

2;�12;�12

�;

�1

2;�12;�12;1

2

�;

��12;1

2;�12;1

2

�!

is an ONB of R4.


The Coordinate Matrix w.r.t. ONB

THEOREM Suppose O D .e1; : : : ; en/ is an ONB of V . Then

Œv�O D

2664hv; e1i:::

hv; eni

3775 ; (6.18)

and

kvk2 D hv; e1i2 C � � � C hv; eni2 ; (6.19)

for every v 2 V .


Proof

» O is a basis H) v D a1e1 C � � � C anen

»˝v; ej

˛ D aj» kvk2 D a21 C � � � C a2n D hv; e1i2 C � � � C hv; eni2


The Gram-Schmidt Orthogonalization Process

THEOREM If .v1; : : : ; vm/ is a linearly independent list of vectors

in V , then there exists an orthonormal list .e1; : : : ; em/ of vectors in

V such that

span.v1; : : : ; vj / D span.e1; : : : ; ej /

for j D 1; : : : ; m.

Proof. I postpone the proof until the next lecture. ut


Corollaries

COROLLARY Every finite-dimensional inner-product space has an

ONB.

COROLLARY Every orthonormal list of vectors in V can be ex-

tended to an ONB of V .


linear algebra - shandong university

Documents