vector and matrix norm

7/27/2019 Vector and Matrix Norm

1/5

Numerical Analysis Massoud Malek

Vector and Matrix Norms

Vector Norm. Given an n-dimensional vector x = x1x2...

xn

, a vector norm||x||, is a non-negative real number, defined such that

1. x >0 when x = 0and||x|| = 0if and only if x= , the zero vector,2. ||kx|| = |k| | |x|| for any scalar k,3. ||x + y| | | |x|| + ||y||,

For p= 1, 2, . . . , , the p-vector norm||xp is defined as

xp

= i |

xi|p

1/p

.

The special casex is defined as

x = maxi

|xi| .

The most commonly encountered vector norm (often simply called the norm of a vector)

is the L2-norm, given by

x2 = x =

x21

+ x22

+ ... + x2n.

This and other types of vector norms are summarized in the following table, togetherwith the value of the norm for the example vector v= (1, 2, 3)t.

norm symbol value approx.

L1 ||x||1 6 6.000L2 ||x||2

14 3.742

L3 ||x||3 62/3 3.302L4 ||x||4 21/4

7 3.146

L ||x|| 3 3.000The concept of unit circle (the set of all vectors of norm 1) is different in different norms:

for the 1-norm the unit circle in IR2 is a square with vertices at (1, 0), (0, 1), (

1, 0), and

(0, 1); for the 2-norm (Euclidean norm) it is the well-known unit circle, while for theinfinity norm it is a square with vertices at (1, 1), (1, 1), (1, 1), and (1, 1). Note thatdue to the definition of the norm, the unit circle is always convex and centrally symmetric

(therefore, for example, the unit ball may be a rectangle but cannot be a triangle).

Two norms|| || and|| || on a vector space V are called equivalent if there exist positivereal numbersR andSsuch that

Rx x Sx,

California State University, East Bay


2/5

Numerical Analysis Vector and Matrix Norms Page 2

for all x V. On a finite-dimensional vector space all norms are equivalent. For instance,the L1, L2, and L norms are all equivalent on IRn:

x2 x1

nx2

x x2 nxx x1 nx .

Equivalent norms define the same notions of continuity and convergence and for many

purposes do not need to be distinguished. To be more precise the uniform structure

defined by equivalent norms on the vector space is uniformly isomorphic.

Given any norm|| || and any non-singular matrix A, we can define a vector norm|| ||such that||x|| = ||Ax|| .

Matrix Norm. Given a complex or real matrix A= (aij), a matrix norm||A|| is a non-negative number associated with Ahaving the properties:

1. ||A|| >0 when A = 0 and||A|| = 0 if and only if A= Z, the zero matrix,2. ||kA|| = |k| | |A|| for any scalar k,3. ||A + B| | | |A|| + ||B||,

For square matrices, a sub-multiplicative matrix norm also satisfies:

4. ||AB| | | |A| || |B||.

If a matrix norm is not sub-multiplicative, then it is called a generalized matrix norm.

Let 1, 2,...,n be the eigenvalues of A, then

1

||A1|| || | |A||. (1)

The spectrum of a square matrix A, denoted by (A) is the set of all eigenvalues of A.

The spectral radius of A, denoted by (A) is defined as:

(A) = max{|| : (A)}

The spectral norm of m n matrix A, denoted by||A||2, which is the square root of the

maximum eigenvalue of the positive semi-definite matrix A

Aor AA

(we choose the onein smaller size),

||A||2 =

(AA) =

(AA) (2)

is often referred to as the matrix norm.

The row norm is defined by

||A|| = maxi

nj=1

|aij |. (3)



3/5


||A||1, ||A||2,and||A|| satisfy the inequality

||A||22 ||A||1 ||A||.

The Frobenius norm of an m n matrix A defined as the square root of the sum of theabsolute squares of its elements; it is also equal to the square root of the trace of the

positive semi-definite matrix AA

||A||F = m

i=1

nj=1

|aij |2 =

trace(AA) . (4)

The Frobenius norm can also be considered as a vector norm.

Induced Norm.If vector norms on IKn are given (IKn is field of real or complex numbers),

then one defines the corresponding induced norm, natural norm, or operator norm on

the space of matrices as the follows:

A = max{Ax :x IKn with x 1}= max{Ax :x IKn withx = 1}

= max

Axx :x IK

n with x =

.

For example, the operator norm corresponding to the p-norm for vectors is:

Ap = maxx=0

Axpxp.

In the case of p= 1and p= , the norms can be computed as:

A1

= max1jn

mi=1

|aij |,

as the column norm of the matrix Aand

A = max1im

n

j=1

|aij |,

as the row norm of the matrix A. In the case ofp= 2(the Euclidean norm), the induced

matrix norm is the spectral norm.

Theorem 1. All induced norms are sub-multiplicative.

Proof. Let Aand B be two matrices where AB is well defined, then

||AB|| = maxx=

ABxx = maxx=

ABxBx

Bxx

maxx=

Axx

maxx=

bxx

= A B.



4/5


Theorem 2. IfA is an n n, then (A) ||A|| for any sub-multiplicative matrix norm|| ||.

Proof. Let v be a eigenvector of A corresponding to the dominant eigenvalue

(i.e., Av = v with|| = (A) ). Define a matrix V, where the first column of V is vand the next n 1 columns are zero vectors. Define the matrix

U= 1

VV with the property A U =U;

then we have

(A) = || . 1 = || ||U|| = ||U|| = ||AU| | | |A|| U = A.

Note. The max normAmax = max{|aij |} is not a sub-multiplicative matrix norm. Alsothe sub-multiplicative Frobenius norm is not an induced norm since the induced norm of

any n n identity matrix In is one, but|| In||F = n.

A matrix norm abon IKmn is called consistent with a vector norm a on IKn anda vector norm b on IKm if:

Axb Aabxa

for all A IKmn, x IKn. All induced norms are consistent by definition.

Equivalence of Norms.For any two vector norms|| || and|| ||, we have

RA A S A

for some positive numbersRandS, for all matrices A IKmn.Moreover, when A IRnn,then for any vector norm , there exists a unique positive numberK such thatLAis a sub-multiplicative matrix norm for everyL K. A matrix norm|| ||a is said to beminimal if there exists no other matrix norm|| ||b satisfying|| ||a || ||b.

Examples of norm equivalence. For a matrix A IRmn, the following inequalities hold:

A2 AF

nA2Amax A2 m nAmax

1nA A2

mA

1m

A1 A2

nA1 .

ExampleLet

A=

1 2 34 5 67 8 9

, B =

1 1 02 3 15 0 2

, and C=

1 1 22 1 11 2 1

.



5/5


Then for A, we obtain:

(A) |16.1168| = 16.1168,A = max{ |1| + | 2| + |3|, | 4| + |5| + | 6|, |7| + | 8| + |9| } = 24,

A1 = max{ |1| + | 4| + |7|, | 2| + |5| + | 8|, |3| + | 6| + |9| } = 18,A2 =

(AA)

283.8585 16.8481,

AF =

|1|2 + | 2|2 + |3|2 + | 4|2 + |5|2 + | 6|2 + |7|2 + | 8|2 + |9|2 =

285 16.8819,Amax= max{ |1|, | 2|, |3|, | 4|, |5|, | 6|, |7|, | 8|, |9|} = |9| ,

with

Amax < (A)< A2 < AF < A1 < A .

Now for B, we obtain:

(B) |2.3772| = 2.3772,B = max{ |1| + | 1| + |0|, |2| + |3| + | 1|, |5| + |0| + | 2| } = 7,B1 = max{ |1| + |2| + |5|, | 1| + |3| + |0|, |0| + | 1| + | 2| } = 8,B2 =

(BB))

36.0994 6.0083,

BF =

|1|2 + | 1|2 + |0|2 + |2|2 + |3|2 + | 1|2 + |5|2 + |0|2 + | 2|2 =

45 6.7082,Bmax= max{ |1|, | 1|, |0|, |2|, |3|, | 1|, |5|, |0|, | 2| } = |5| ,

(B)

vector and matrix norm

Documents