review of linear algebra - koç...
TRANSCRIPT
REVIEW OF LINEAR ALGEBRA
INDR 262INTRODUCTION TO OPTIMIZATION METHODS
Metin TürkayDepartment of Industrial Engineering
Koç University, Istanbul
MATRICES
Øm and n are positive integersØOrder of matrix: mxnØThe number in the i th row and jth column of A is
called the ij th element of A and is written aij .
) =+,, +,-+-, +-,
⋯ +,/⋯ +-/
⋮ ⋮+1, +1-
⋱ ⋮⋯ +1/
EXAMPLE
! =1 2 34 5 67 8 9
,-- = 1,./ = 6,/- = 7
EQUAL MATRICES
ØTwo matrices A and B are equal if and only if Aand B are of the same order and for all i and j, aij=bij .
( = 1 23 4 - = . /
0 1
ØIf A=B, then x=1, y=2, w=3, z=4
VECTORS
ØA list of n real numbers, say (a1,a2,…,an) is called an n-dimensional vector. An n-dimensional vector also maybe displayed as a 1 by n matrix.
12 , 1 2 3
SCALAR PRODUCT OF TWO VECTORS
ØThe scalar product of vectors
! = !# !$ ⋯ !& and * =*#*$⋮*&
is !#*# + !$*$ + ⋯ +!&*&
EXAMPLE
uv = 1x2+2x1+3x2 = 10
[ ]úúú
û
ù
êêê
ë
é==
212
321 vu
NOTES
ØIf u = [1 2 3] and , then uv is not defined
because the vectors are of different dimensions.
ØTwo vectors are perpendicular to each other if and only if their scalar product is equal to 0.
e.g., u = [1 -1] and
úû
ùêë
é=43
v
úû
ùêë
é=11
v
SCALAR MULTIPLE OF A MATRIX
ØGiven any matrix A and any scalar c, the scalar multiple of matrix A, cA, is obtained from the matrix A by multiplying each element of A by c.
úû
ùêë
é-
=úû
ùêë
é-
=0363
3 0121
AA
ADDITION OF TWO MATRICES
ØLet A=[aij] and B=[bij] be two matrices with the same
order (say mxn). Then, the matrix C=A+B is defined
to be the mxn matrix whose ijth element is aij+bij.
úû
ùêë
é=ú
û
ùêë
é-+-+---
=+=
úû
ùêë
é----
=úû
ùêë
é-
=
002000
111120332211
112321
110321
BAC
BA
THE TRANSPOSE OF A MATRIX
ØGiven any mxn matrix
ØThe transpose of A (written AT) is the nxm matrix
úúúú
û
ù
êêêê
ë
é
=
mnmm
n
n
aaa
aaaaaa
A
..........
..
..
21
22221
11211
úúúú
û
ù
êêêê
ë
é
=
nmnn
n
n
T
aaa
aaaaaa
A
..........
..
..
21
22212
12111
EXAMPLE
ØFor any matrix A, (AT)T=A.
( ) úû
ùêë
é=
úúú
û
ù
êêê
ë
é=ú
û
ùêë
é=
654321
635241
654321 T TTAAA
MATRIX MULTIPLICATION
ØGiven two matrices A and B, the matrix product of A and B is defined if and only if the number of columns in A is equal to the number of rows in B. The matrix product C=AB is determined as follows:
cij = scalar product of (row i of A and column j of B)
PROPERTIES OF MATRIX MULTIPLICATION
1. Matrix multiplication is associative, i.e., A(BC)=(AB)C.
2. Matrix multiplication is distributive, i.e., A(B+C)=AB+AC.
EXAMPLE
[ ]
[ ]
[ ]
[ ] 531
12
421
12
431
11
321
11
22
21
12
11
=úû
ùêë
é=
=úû
ùêë
é=
=úû
ùêë
é=
=úû
ùêë
é=
c
c
c
c
ABC
BA
=
úû
ùêë
é=ú
û
ùêë
é=
3211
1211
úû
ùêë
é==
5443
ABC
MATRICES AND SYSTEMS OF LINEAR EQUATIONS
ØConsider a system of linear equations given by
Øx1, x2, …, xn are referred to as variablesØaij’s and bi’s are constantsØA set of equations like above is called a linear system
of m equations in n variables
11 1 12 2 1 1
21 1 22 2 2 2
1 1 2 2
...
...... ... ... ... ... ... ... ... ...
...
n n
n n
m m mn n m
a x a x a x ba x a x a x b
a x a x a x b
+ + + =+ + + =
+ + + =
SOLUTION
ØA solution to a linear system of m equations in nunknowns is a set of values for the unknowns that satisfy each of the systems m equations.
EXAMPLE
x1+ 2x2 = 52x1 - x2= 0
úû
ùêë
é=21
x
úû
ùêë
é=13
x
00 0 ? 2)1(2
55 5 ? )2(21
=-
=+
05 0 ? 1)3(2
55 5 ? )1(23
¹-
=+
Solution
Not a solution
MATRIX REPRESENTATION OF SYSTEMS OF LINEAR EQUATIONS
Ax = b11 12 1 1 1
21 22 2 2 2
1 2
...
..., ,
... ... ... ... ... ......
n
n
m m mn n m
a a a x ba a a x b
A x b
a a a x b
é ù é ù é ùê ú ê ú ê úê ú ê ú ê ú= = =ê ú ê ú ê úê ú ê ú ê úë û ë û ë û
[ ]úúúúú
û
ù
êêêêê
ë
é
=
mmnmm
n
n
b
bb
aaa
aaaaaa
bA...
...............
...
...
2
1
21
22221
11211
Augmented matrix
THE GAUSS-JORDAN METHOD FOR SOLVING SYSTEMS OF LINEAR EQUATIONS
ØGauss-Jordan method is used to find solution(s) to systems of linear equations. A system of linear equations must satisfy one of the following cases:§ Case 1: The system has no solution§ Case 2: The system has a unique solution§ Case 3: The system has an infinite number of solutions
ELEMENTARY ROW OPERATIONS
ØAn elementary row operation (ero) transforms a given matrix A into a new matrix A’ via one of the following operations.§ Type 1 ero: A’ is obtained by multiplying any row of A by a
nonzero scalar.§ Type 2 ero: Begin by multiplying any row of A (say, row i) by a
nonzero scalar c. For some j≠i, let row j of A’ = c(row i of A) + row j of A, and let the other rows of A’ be the same as the rows of A.
§ Type 3 ero: Interchange any two rows of A.
FACTS
ØIf the matrix A’ is obtained from A via an ero, A’ and Aare equivalent.
ØIf the augmented matrix [A’Ιb’] is obtained from [AΙb]via an ero, the systems Ax=b and A’x=b’ are equivalent.
ØAny sequence of ero’s performed on the augmented matrix [AΙb] corresponding to the system Ax=b will yield an equivalent linear system.
GAUSS-JORDAN METHOD
ØThe Gauss-Jordan method solves a linear system of equations by utilizing ero’s in a systematic fashion.
Step 1 To solve Ax=b, write down the augmented matrix [AΙb].
Step 2 At any stage, define a current row, current column, and a current entry. Begin with row 1 as the current row, column 1 as the current column, and a11 as the current entry.
GAUSS-JORDAN METHOD
a. If a11 (the current entry) is nonzero, use ero’s to transform column 1(the current column) to [1 0 … 0]T. Then, obtain the new current row, column, and entry by moving down one row and one column to the right, and go to Step 3.
b. If a11 (the current entry) equals 0, then do a Type 3 ero involving the current row and any row that contains a nonzero entry in the current column. Use ero’s to transform column 1 (the current column) to [1 0 … 0]T. Then, obtain the new current row, column, and entry by moving down one row and one column to the right, and go to Step 3.
c. If there are no nonzero numbers in the first column, obtain a new current column and entry by moving one column to the right. Then go to Step 3.
GAUSS-JORDAN METHOD
Step 3a. If the new current entry is nonzero, use ero’s to transform it to 1
and the rest of the current column’s entries to 0. When finished, obtain a new current row, column, and entry. If this is impossible, stop. Otherwise, repeat Step 3.
b. If the current entry is 0, do a Type 3 ero with the current row and any row that contains a nonzero entry in the current column. Then, use ero’s to transform column entry to 1 and the rest of the current column’s entries to 0. When finished, obtain the new current row, column, and entry. If this is impossible, stop. Otherwise, repeat Step 3.
c. If the current column has no nonzero numbers below the current row, obtain a new current column and entry and repeat Step 3. If it is impossible, stop.
GAUSS-JORDAN METHOD
Step 4 Write down the system of equations A’x=b’ that corresponds to the matrix [A’Ιb’] obtained when Step 3 is completed. Then, A’x=b’ will have the same set of solutions as Ax=b.§The Gauss–Jordan method converts the augmented matrix [AΙb]into [A’ Ιb’] such that
[ ]úúúúú
û
ù
êêêêê
ë
é
=
'
'2
'1
...1...00............0...100...01
''
mb
bb
bA ''22
'11 ..., , , mn bxbxbx ===
EXAMPLE
§ Solve the following system of linear equations using the Gauss-Jordan method.
2x1 + 2x2 + x3 = 92x1 - x2 + 2x3 = 6x1 - x2 + 2x3 = 5
SOLUTION
[ ]úúú
û
ù
êêê
ë
é
--=
569
211212122
bA [ ]úúúú
û
ù
êêêê
ë
é
--=
5629
2112122111
11 bA
[ ]úúúú
û
ù
êêêê
ë
é
---=
5329
2111302111
22 bA[ ]úúúú
û
ù
êêêê
ë
é
---=
21329
23201302111
33 bA
SOLUTION
[ ]úúúú
û
ù
êêêê
ë
é
-
-=
21129
232031102111
44 bA [ ]úúúú
û
ù
êêêê
ë
é
-
-=
21127
232031106501
55 bA
[ ]úúúú
û
ù
êêêê
ë
é
-=
25127
650031106501
66 bA[ ]
úúúú
û
ù
êêêê
ë
é
-=3127
10031106501
77 bA
SOLUTION
[ ]úúú
û
ù
êêê
ë
é
=321
100010001
99 bA[ ]úúú
û
ù
êêê
ë
é-=
311
1003110001
88 bA
x1 = 1 x2 = 2 x3 = 3
ANALYSIS OF THE SOLUTIONS TO SYSTEMS OF LINEAR EQUATIONS
For any linear system, a variable that appears with a coefficient of 1 in a single equation and a coefficient of 0 in all other equations is called a basic variable. Any variable that is not a basic variable is called a nonbasic variable.
x set of all of the variables in the system Ax=bxB set of all of the basic variables in the system Ax=bxN set of all of the nonbasic variables in the system Ax=b
x = xBUxN
ANALYSIS OF THE SOLUTIONS TO SYSTEMS OF LINEAR EQUATIONS
ØThe solution to A’x=b’ can be categorized in one of the three cases:
Case 1: A’x=b’ has at least one row of the form [0 0 … 0Ιc] and c≥0. Then, Ax=b has no solution.
Case 2: When case 1 does not apply and xN=x, then Ax=bhas a unique solution.
Case 3: When case 1 does not apply and xN≠x, then Ax=bhas infinite number of solutions.
EXAMPLE
Ø Case 1 does not apply since there are no rows of the form [0 0 … 0Ιc] and c≠0.
Ø Case 2 does not apply since,xB={x1, x2, x3}xN={x4, x5}
Ø There are infinite number of solutions.
[ ]úúúúú
û
ù
êêêêê
ë
é
=
0123
00000101000201011001
'' bA
EXAMPLE
ØAssign arbitrary values to the variables in xN; x4=c, x5=k.ØWrite down the equations in [A’ Ιb’],
x1 + c + k = 3 è x1 = 3 - c – kx2 + 2c = 2 è x2 = 2 - 2cx3 + k = 1 è x3 = 1 - k
Ø It is easy to see that there are infinite number of values of c and kthat will satisfy this system of equations.
[ ]úúúúú
û
ù
êêêêê
ë
é
=
0123
00000101000201011001
'' bA
LINEAR COMBINATION
ØA linear combination of the vectors in V is any vector of the form c1v1+c2v2+…+ckvk, where c1, c2, …, ck are arbitrary scalars.
Example: V={[1,2], [2,1]}
2v1-v2 = 2([1 2]) – [2 1] = [0 3]0v1-3v2 = 0([1 2]) – 3([2 1]) = [6 3]
LINEAR INDEPENDENCE &LINEAR DEPENDENCE
ØA set V of m-dimensional vectors is linearly independent if the only linear combination of vectors in V that equals 0 is the trivial linear combination.
ØA set V of m-dimensional vectors is linearly dependent if there is a nontrivial linear combination of the vectors in V that adds up to 0.
EXAMPLE 1
ØV={[1,0], [0,1]} Try to find a linear combination of vectors in V that yields 0.
Øc1([1 0]) + c2([0 1]) = [0 0]ØIn order to satisfy this, [c1 c2] = [0 0] è c1=c2=0ØThe only linear combination of vectors in V that yields
0 is the trivial linear combination. Therefore, V is a linearly independent set of vectors.
EXAMPLE 2
ØV={[1,2], [2,4]} Try to find a linear combination of vectors in V that yields 0.
Øc1([1 2]) + c2([2 4]) = [0 0][c1 2c1] + [2c2 4c2] = [0 0]c1 + 2c2 = 0 è c1 = -2c22c1 + 4c2 = 0 è 2c1 = -4c2
ØSo, c1 = 2 c2 = -1 is one of the possible solutions.ØThere exists a nontrivial linear combination of vectors
in V that yields 0. Therefore, V is a linearly dependent set of vectors.
THE RANK OF A MATRIX
ØLet A be any mxn matrix, and denote the rows of Aby r1, r2, …, rm. Also define R={ r1, r2, …, rm}.
ØThe rank of A is the number of vectors in the largest linearly independent subset of R.
ØIf for a matrix A with m rows, rank A=m; then the matrix is a collection of linearly independent set of vectors. If rank A<m; then the matrix contains a linearly dependent set of vectors.
EXAMPLES
úúú
û
ù
êêê
ë
é
3202110001
úúú
û
ù
êêê
ë
é=
320120001
Aúúú
û
ù
êêê
ë
é
1002110001
úúú
û
ù
êêê
ë
é
100010001
rank(A) = 3
1 0 00 1 01 1 0
Bé ùê ú= ê úê úë û
rank(B) = 2
THE INVERSE OF A MATRIX
ØA single linear equation in a single variable can be solved by multiplying both sides of the equation by multiplicative inverse of the variable coefficient.
ØExample:4x=3 è 4-1(4x) = (4-1)3 è x=3/4
ØWe can generalize this approach to square systems of linear equations (i.e., number of equations = number of unknowns).
SQUARE AND IDENTITY MATRIX
Ø A square matrix is any matrix that has an equal number of rows and columns.
Ø The diagonal elements of a square matrix are those elements aij such that i=j.
Ø A square matrix for which all diagonal elements are equal to 1 and all non-diagonal elements are equal to 0 is called an identity matrix.
úúú
û
ù
êêê
ë
é=ú
û
ùêë
é=
100010001
, 1001
32 II
INVERSE OF A
ØFor a given mxm matrix A, the mxm matrix B is the inverse of A if
BA = AB = Im
úúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é--
úúú
û
ù
êêê
ë
é
-
-
100010001
201715101
101213102
AA-1=I
FINDING THE INVERSE OF A MATRIX WITH THE GAUSS-JORDAN METHOD
Step 1 Write down the mx2m matrix [AΙIm].
Step 2 Use ero’s to transform [AΙIm] into [ImΙB]. This will only be possible if rank(A)=m; in this case, B=A-1. If rank(A)<m, then A has no inverse.
EXAMPLE
úû
ùêë
é=
3152
A
[ ]úúû
ù
êêë
é=
1001
3152
2IA [ ]úúû
ù
êêë
é=
1002
1
31251
12IA
[ ]úú
û
ù
êê
ë
é
-=12
102
1
210251
22IA[ ]úúû
ù
êêë
é
-=
2102
1
10251
32IA
[ ]úúû
ù
êêë
é
--
=2153
1001
42IA úû
ùêë
é-
-=-
21531A
USING MATRIX INVERSE TO SOLVE LINEAR SYSTEMS OF EQUATIONS
ØGiven a linear system of equations, Ax=bØMultiply both sides by A-1
A-1Ax = A-1b è x = A-1b
EXAMPLE
2x1 + 5x2 = 7x1 + 3x2 = 4 ú
û
ùêë
é=ú
û
ùêë
éúû
ùêë
é47
3152
2
1xx
úû
ùêë
é-
-=Þú
û
ùêë
é=
2153
3152 1-AA
úû
ùêë
é=ú
û
ùêë
éÞú
û
ùêë
éúû
ùêë
é-
-=ú
û
ùêë
é11
47
2153
2
1
2
1xx
xx