7.3: systems of linear equations, linear independence ...park633/ma266/boyce_de10_ch...7.3: systems...
TRANSCRIPT
-
7.3: Systems of Linear Equations, Linear
Independence, Eigenvalues
• A system of n linear equations in n variables:
can be expressed as a matrix equation Ax = b:
• If b = 0, then system is homogeneous; otherwise it is nonhomogeneous.
nnnnnn
n
n
b
b
b
x
x
x
aaa
aaa
aaa
2
1
2
1
,2,1,
,22,21,2
,12,11,1
,,22,11,
2,222,211,2
1,122,111,1
nnnnnn
nn
nn
bxaxaxa
bxaxaxa
bxaxaxa
-
Nonsingular Case
• If the coefficient matrix A is nonsingular, then it is invertible and
we can solve Ax = b as follows:
• This solution is therefore unique. Also, if b = 0, it follows that
the unique solution to Ax = 0 is x = A-10 = 0.
• Thus if A is nonsingular, then the only solution to Ax = 0 is the
trivial solution x = 0.
1 1 1 1 Ax b A xAx A b I A Ax b b
-
Example 1: Nonsingular Case (1 of 3)
• From a previous example, we know that the matrix A below is nonsingular
with inverse as given.
• Using the definition of matrix multiplication, it follows that the only solution
of Ax = 0 is x = 0:
4/14/34/1
4/14/74/5
4/14/54/3
,
112
211
3211
AA
0
0
0
0
0
0
4/14/34/1
4/14/74/5
4/14/54/310Ax
-
Example 1: Nonsingular Case (2 of 3)
• Now let’s solve the nonhomogeneous linear system Ax = b below using A-1:
• This system of equations can be written as Ax = b, where
• Then
0834
2301
220
321
321
321
xxx
xxx
xxx
1
3 / 4 5 / 4 1/ 4 7 2
5 / 4 7 / 4 1/ 4 5 1
1/ 4 3 / 4 1/ 4 4 1
x A b
4
5
7
,,
112
211
321
3
2
1
bxA
x
x
x
-
Example 1: Nonsingular Case (3 of 3)
• Alternatively, we could solve the nonhomogeneous linear system Ax = b
below using row reduction.
• To do so, form the augmented matrix (A|b) and reduce, using elementary row
operations.
1
1
2
1
2
732
1100
2110
7321
4400
2110
7321
10730
2110
7321
10730
2110
7321
4112
5211
7321
3
32
321
x
bA
x
xx
xxx
42
52
732
321
321
321
xxx
xxx
xxx
-
Singular Case
• If the coefficient matrix A is singular, then A-1 does not exist, and either a
solution to Ax = b does not exist, or there is more than one solution (not
unique).
• Further, the homogeneous system Ax = 0 has more than one solution. That is,
in addition to the trivial solution x = 0, there are infinitely many nontrivial
solutions.
• The nonhomogeneous case Ax = b has no solution unless (b, y) = 0, for all
vectors y satisfying A*y = 0, where A* is the adjoint of A.
• In this case, Ax = b has solutions (infinitely many), each of the form x = x(0)
+ , where x(0) is a particular solution of Ax = b, and is any solution of
Ax = 0.
*( , ) ( , ) ( , )b y Ax y x A y
1 2 2:
2 4 1A X c
-
Linear Dependence and Independence
• A set of vectors x(1), x(2),…, x(n) is linearly dependent if
there exists scalars c1, c2,…, cn, not all zero, such that
• If the only solution of
is c1= c2 = …= cn = 0, then x(1), x(2),…, x(n) is linearly
independent.
0xxx )()2(2)1(
1
n
nccc
0xxx )()2(2)1(
1
n
nccc
-
Example 3: Linear Dependence (1 of 2)
• Determine whether the following vectors are linear dependent
or linearly independent.
• We need to solve or
0
0
0
1131
112
421
0
0
0
11
1
4
3
1
2
1
2
1
3
2
1
21
c
c
c
ccc
0xxx )3(3)2(
2
)1(
1 ccc
11
1
4
,
3
1
2
,
1
2
1)3()2()1(
xxx
-
Example 3: Linear Dependence (2 of 2)
• We can reduce the augmented matrix (A|b), as before.
• So, the vectors are linearly dependent:
• Alternatively, we could show that the following determinant is zero.
(Question) The columns (or rows) of A are linearly independent
if and only if A is nonsingular ?
1 2 3
2 3 33
1 2 4 0 1 2 4 0 1 2 4 0
2 1 1 0 0 3 9 0 0 1 3 0
1 3 11 0 0 5 15 0 0 0 0 0
2 4 0
3 0 where can be any
2
3 number
10 0
c c c
c c cc
A
c
b
11
1
4
,
3
1
2
,
1
2
1)3()2()1(
xxx
0
1131
112
421
)det(
ijx(1) (2) 3)
3
(2if 1 3,c x x x 0
-
Linear Independence and Invertibility
• Consider the previous two examples:
– The first matrix was known to be nonsingular, and its column vectors were
linearly independent.
– The second matrix was known to be singular, and its column vectors were
linearly dependent.
• This is true in general: the columns (or rows) of A are linearly independent iff
A is nonsingular iff A-1 exists.
• Also, A is nonsingular iff detA 0, hence columns (or rows) of A are
linearly independent iff detA 0.
• Further, if A = BC, then det(C) = det(A)det(B). Thus if the columns (or
rows) of A and B are linearly independent, then the columns (or rows) of C are
also.
-
Linear Dependence & Vector Functions
• Now consider vector functions x(1)(t), x(2)(t),…, x(n)(t), where
• As before, x(1)(t), x(2)(t),…, x(n)(t) is linearly dependent on I if there exists
scalars c1, c2,…, cn, not all zero, such that
• Otherwise x(1)(t), x(2)(t),…, x(n)(t) is linearly independent on I
See text for more discussion on this.
,,,,2,1,
)(
)(
)(
)(
)(
)(
2
)(
1
Itnk
tx
tx
tx
t
k
m
k
k
k
x
Ittctctc nn allfor ,)()()()()2(
2
)1(
1 0xxx
(1) (2) (n) 1 2( ) ( ) ( ) 0, C=T
nX t X t X t C c c c
-
Eigenvalues and Eigenvectors
• The equation Ax = y can be viewed as a linear transformation that maps (or
transforms) x into a new vector y.
• Nonzero vectors x that transform into multiples of themselves are important in
many applications.
• Thus we solve Ax = x or equivalently, (A-I)x = 0.
• This equation has a nonzero solution if we choose such that det(A-I) = 0.
• Such values of are called eigenvalues of A, and the nonzero solutions x are called eigenvectors.
(Example) Find eigenvalues of
24
13A
-
Example 4: Eigenvalues (1 of 3)
• Find the eigenvalues and eigenvectors of the matrix A.
• Solution: Choose such that det(A-I) = 0, as follows.
24
13A
1,2
122
4123
24
13det
10
01
24
13detdet
2
IA
-
Example 4: First Eigenvector (2 of 3)
• To find the eigenvectors of the matrix A, we need to solve (A-I)x = 0 for = 2 and = -1.
• Eigenvector for = 2: Solve
and this implies that . So
0
0
44
11
0
0
224
123
2
1
2
1
x
x
x
x0xIA
(2(1)
2
1)1
, arbitrary choose 1
1
1
xc c
x
x x
21 xx
-
Example 4: Second Eigenvector (3 of 3)
• Eigenvector for = -1: Solve
and this implies that . So
0
0
14
14
0
0
124
113
2
1
2
1
x
x
x
x0xIA
(2)
1
2)1(1
, arbitrary choos4
e 4
1
4
xc c
x
xx
12 4xx
-
Normalized Eigenvectors
• From the previous example, we see that eigenvectors are
determined up to a nonzero multiplicative constant.
• If this constant is specified in some particular way, then the
eigenvector is said to be normalized.
• For example, eigenvectors are sometimes normalized by
choosing the constant so that ||x|| = (x, x)½ = 1.
-
Algebraic and Geometric Multiplicity
• In finding the eigenvalues of an n x n matrix A, we solve det(A-I) = 0.
• Since this involves finding the determinant of an n x n matrix, the problem
reduces to finding roots of an nth degree polynomial.
• Denote these roots, or eigenvalues, by 1, 2, …, n.
• If an eigenvalue is repeated m times, then its algebraic multiplicity is m.
• Each eigenvalue has at least one eigenvector, and an eigenvalue of algebraic
multiplicity m may have q linearly independent eigenvectors, 1 q m,
and q is called the geometric multiplicity of the eigenvalue.
-
Eigenvectors and Linear Independence
• If an eigenvalue has algebraic multiplicity 1, then it is said to be simple, and the geometric multiplicity is 1 also.
• If each eigenvalue of an n x n matrix A is simple, then A has n distinct
eigenvalues. It can be shown that the n eigenvectors corresponding to these
eigenvalues are linearly independent.
• If an eigenvalue has one or more repeated eigenvalues, then there may be
fewer than n linearly independent eigenvectors since for each repeated
eigenvalue, we may have q < m. This may lead to complications in solving
systems of differential equations.
-
Example 5: Eigenvalues (1 of 5)
• Find the eigenvalues and eigenvectors of the matrix A.
• Solution: Choose such that det(A-I) = 0, as follows.
011
101
110
A
1,1,2
)1)(2(
23
11
11
11
detdet
221
2
3
IA
-
Example 5: First Eigenvector (2 of 5)
• Eigenvector for = 2: Solve (A-I)x = 0, as follows.
1
1
1
choosearbitrary,
1
1
1
00
011
011
0000
0110
0101
0000
0110
0211
0330
0330
0211
0112
0121
0211
0211
0121
0112
)1(
3
3
3
)1(
3
32
31
xx cc
x
x
x
x
xx
xx
-
Example 5: 2nd and 3rd Eigenvectors (3 of 5)
• Eigenvector for = -1: Solve (A-I)x = 0, as follows.
1
1
0
,
1
0
1
choose
arbitrary,where,
1
0
1
0
1
1
00
00
0111
0000
0000
0111
0111
0111
0111
)3()2(
3232
3
2
32
)2(
3
2
321
xx
x xxxx
x
x
xx
x
x
xxx
-
Example 5: Eigenvectors of A (4 of 5)
• Thus three eigenvectors of A are
where x(2), x(3) correspond to the double eigenvalue = - 1.
• It can be shown that x(1), x(2), x(3) are linearly independent.
• Hence A is a 3 x 3 symmetric matrix (A = AT ) with 3 real eigenvalues
and 3 linearly independent eigenvectors.
1
1
0
,
1
0
1
,
1
1
1)3()2()1(
xxx
011
101
110
A
-
Example 5: Eigenvectors of A (5 of 5)
• Note that we could have we had chosen
• Then the eigenvectors are orthogonal, since
• Thus A is a 3 x 3 symmetric matrix with 3 real eigenvalues and 3 linearly
independent orthogonal eigenvectors.
1
2
1
,
1
0
1
,
1
1
1)3()2()1(
xxx
0,,0,,0, )3()2()3()1()2()1( xxxxxx
-
Hermitian Matrices
• A self-adjoint, or Hermitian matrix, satisfies A = A*, where we recall that
A* = AT .
• Thus for a Hermitian matrix, aij = aji.
• Note that if A has real entries and is symmetric (see last example), then A is
Hermitian.
• An n x n Hermitian matrix A has the following properties:
– All eigenvalues of A are real.
– There exists a full set of n linearly independent eigenvectors of A.
– If x(1) and x(2) are eigenvectors that correspond to different eigenvalues of A, then
x(1) and x(2) are orthogonal.
– Corresponding to an eigenvalue of algebraic multiplicity m, it is possible to
choose m mutually orthogonal eigenvectors, and hence A has a full set of n
linearly independent orthogonal eigenvectors.
-
Example 2: Singular Case (1 of 2)
• Solve the nonhomogeneous linear system Ax = b below using row reduction.
Observe that the coefficients are nearly the same as in the previous example
• We will form the augmented matrix (A|b) and use some of the steps in
Example 1 to transform the matrix more quickly
03
30
32
3000
110
321
312
211
321
321
321
2132
1321
321
21
1
3
2
1
bbb
bbb
bbxx
bxxx
bbb
bb
b
b
b
b
bA
3321
2321
1321
32
2
32
bxxx
bxxx
bxxx
-
Example 2: Singular Case (2 of 2)
• From the previous slide, if , there is no solution to the system
of equations
• Requiring that , assume, for example, that
• Then the reduced augmented matrix (A|b) becomes:
• It can be shown that the second term in x is a solution of the nonhomogeneous
equation and that the first term is the most general solution of the homogeneous
equation, letting , where α is arbitrary
0
3
4
1
1
1
3
4
00
3
232
3000
110
321
3
3
3
3
32
321
321
21
1
x
x
x
x
xx
xxx
bbb
bb
b
xx
03 321 bbb
5,1,2 321 bbb
03 321 bbb
3321
2321
1321
32
2
32
bxxx
bxxx
bxxx
3x