gaussian-, jordan elimination and matrix inversesb_novak/banki/gauss_ang.pdf · gaussian-, jordan...
TRANSCRIPT
© Bércesné dr. Novák Ágnes 2015. február 13.
Gaussian-, Jordan Elimination and Matrix Inverses
GAUSSIAN ELIMINATION
A system of linear equations consists of several linear equation :
Example: in general: for instance:
Here a11 is a coefficient – a number. The first unknown variable is x1. aik is a coefficient – a number. The kth unknown variable is xk.
Consider the following system of linear equations in three variables.
x1 + x2 + 2x3 = 8
x2 - 5x3 = -9 x3 = 2
It is easy to solve, since x3 = 2 is given in the third equation. We just substitute this value into
the 2nd equation, in order to get x2, and finally, with x2 and x3 being substituted into the first
we get the solution.
How nice would it be, if each system had the same echelon form!
The good news is, that even if they usually do not look like the one above - but using
eqvivalent elementary raw operations, we are able to achieve this form.
Elementary row operations are steps for solving the linear system of equations.
The insight of the method of elementary row operations is that in some sense they do not
change the system of equations. For example, the solution to a set of linear equations does
not depend on what equation is written first; thus swapping rows should not change the
solution.
There are three elementary row operations:
I. Interchange two equations
II. Multiply an equation with non zero real number
III. Add a multiple of one equation to another equation
a11x1+a12x2+ a13x3…a1nxn =b1
a21x1+a22x2+ a23x3…a2nxn =b2
a31x1+a32x2+ a33x3…a3nxn =b3
…
am1x1+am2x2+ am3x3…amnxn= bm
x1+ 2x2+ 3x3+ 4x4 = 5
6x1+ 7x2+ 8x3+ 9x4 = 10
11x1+12x2+ 13x3+14x4 =15
16x1+17x2+ 18x3+19x4= 20
© Bércesné dr. Novák Ágnes 2015. február 13.
Look at this system of equation.
One possible algebraic solution is this. First we divided the 3rd equation by 3, getting x+y+ z=6. This one is moved into the very first row. The other operations are denoted in the
boxes below. → →
Here is the echelon form, and from there, with back substitution we get z=3, y=2, and x=1.
So, this equation has ONE solution.
4x-2y+ 2z=6
4x+ y- z=3
3x+ 3y+3z=18
x+ y+ z=6
4x-2y+ 2z=6 /+(-4)*I.e
4x+ y- z=3 /+(-4)*I.e
x+y+z=6
-6y-2z=-18
-3y-5z=-21/+(-1/2)*2*II.e
e
x+ y+ z=6
-6y-2z=-18
-4z=-12
→ → →
4x-2y+ 2z=6
4x+ y- z=3
3x+ 3y+3z=18
Each equation can be interpreted as an equation of a plane, as you can see
in the figure below. These planes have exactly on common point: (1,2,3)
so, this is the only solution of this system.
Now we will follow how to get an algebraic solution,.
© Bércesné dr. Novák Ágnes 2015. február 13.
Methods for solving System of Linear equations
1. Gaussian Elimination Method
2. Gauss – Jordan Elimination Method
Gaussian Elimination Method
STEP 1. by using elementary row operations we produce an echelon form, being
eqvivalent to the original system of equation.
a11x1+a12x2+ a13x3…a1nxn =b1
a21x1+a22x2+ a23x3…a2nxn =b2
a31x1+a32x2+ a33x3…a3nxn =b3
…
am1x1+am2x2+ am3x3…amnxn= bm
α11x1+ α 12x2+ α13x3+… α 1nxn =1
α 22x2+ α23x3+… α 2nxn =2
α33x3+… α 3nxn =3
…
α m’nxn =m
= =
© Bércesné dr. Novák Ágnes 2015. február 13.
How can we do this?
Suppose we are at the first step, that is, with a11 x1 we want to eliminate x1 from the ith
equation. In a.) we produce a 1 coefficient in equation 1. for x1 . In b.) the new 1st equation is
multiplied by -ai1 so have the opposit coefficient in the 1st and in the ith equatoion. Adding
these two, in c.) we get 0 x1 in the ith equation. . Similarly we eliminate x1 from all the
equations below except for the first equation.
First and ith
equation from
the system,
a.)
c.)
When we finished the elimination of x1, the process continues with the elimination of only
x2 , that are standing below the main diagonal. For this, we use the x2 in the second equation.
In general, xk is eliminated always by the help of the kth element of the kth equation, except
when we have 0 at the position akk. In this case we change the equations in order to have a
nonzero element in akk. We are ready when we have only zeros below the main diagonal.
1
1111
13
11
132
11
121
1b
ax
a
ax
a
ax
a
ax n
n
...
332211 ininiii bxaxaxaxa
ininiii bxaxaxaxa 332211
11313212111 bxaxaxaxa nn
...
332211 ininiii bxaxaxaxa
/ : a11
1
11
1
11
113
11
13132
11
1212
1
1111
13
11
132
11
121
10
1
ba
abxa
aaax
a
aaax
a
aaa
ba
xa
ax
a
ax
a
ax
iinn
iiniiii
nn
/ (-ai1)
+
© Bércesné dr. Novák Ágnes 2015. február 13.
STEP 2. Find solution by back – substitutions.
Since at the end we have an echelon form, the last unknown is calculated by a simple
division by α m’n. . The previous one can be got from the previous equation by simple
substitution of xn and so on…
So, a Gaussian elimination is a method by which we achieve an echelon form, and by back-
substituting we get all the values for the unknowns.
Example
1. x1+ 2x2 + 3x3 + 4x4 = 5
2. 6x1+ 7x2+ 8x3+ 9x4 = 10
3. 11x1+12x2+ 13x3+14x4 = 15
4. 16x1+ 17x2+ 18x3+19x4 = 20
Solution:
:
1. x1+ 2x2+ 3x3+ 4x4 = 5
2. 6x1+ 7x2+ 8x3+ 9x4 = 10 /+ (-6)* 1. eq. 3. 11x1+12x2+ 13x3+14x4 = 15/+(-11) * 1. eq. 4. 16x1+17x2+ 18x3+19x4 = 20/+(-16) * 1. eq.
Result:
1. x1+ 2x2+ 3x3+ 4x4 = 5
2. 0 -5x2 -10x3 -15x4 = -20
3. 0-10x2 -20x3 -30x4 = -40
4. 0-15x2-30x3-45x4= -60
1st ELIMINATION step is done. Now we are working on the elimination of x2
from equations 3 and 4.
1. x1+2x2+ 3x3 +4x4 = 5
2. 0- 5x2-10x3 - 15x4 = -20/:(-5) → x2 +2x3+3x4 = 4 new 2.eq. 3. 0-10x2-20x3 - 30x4 = -40/+10* new 2.eq. → 0 + 0+ 0 = 0
4. 0-15x2-30x3 - 45x4= -60/+15*new 2.eq. → 0 + 0+ 0 = 0
α11x1+ α 12x2+ α13x3+… α 1nxn =1
α 22x2+ α23x3+… α 2nxn =2
α33x3+… α 3nxn =3
…
α m’nxn =m
© Bércesné dr. Novák Ágnes 2015. február 13.
We kept the rules, but because the equations were not independent - one can be expressed by the
combination of the others – we got two rows with full of zeros. So it is not really an echelon form.
But still we can figure out the solution.
Beginning with the new 2nd equation:
x2+2x3+3x4 = 4→ x2= 4-2x3-3x4 substituted into the 1. equation:
x1+ 2x2+ 3x3+4x4 = 5 → x1= 5-2x2-3x3-4x4= 5- 2(4-2x3-3x4)-3x3-4x4=
= 5 -8+4x3+6x4-3x3- 4x4=-3+x3+2x4
So: x1= 5 -8+4x3+6x4-3x3- 4x4=-3+x3+2x4
x2= 4-2x3-3x4
This is the solution explicitely expressed each unknowns:
x1= -3+ p+2q
x2= 4-2p-3q
x3= p, pR
x4=q, qR
This simpler system of linear equation is the solution!
So there are infinitely many solutions.The meaning is, that each quartett x1, x2 x3 x4 satisfying this
new system, is a solution. x1 and x2 can be calculated with the freely chosen x3 and x4,
It is important, which unknown can be chosen freely. Sometimes it is not unique.
Generally if there IS a number in the main diagonal in the echelon (or last) form, then
that unknown SHOULD NOT BE CHOSEN FREELY, it will be expressed by the other
unknowns.
Gaussian elimination is matrix form
We may simplify the writing. As we learnt in high-school algebra, the variables x, y, and z
are just “place holders” in that the real structure of the problem is contained in the numbers.
So, there is no need to write them down.
One use of matrices is to focus on this structure without attending to the variables.
There are two matrices associated with this problem, the matrix of coefficients and the
augmented matrix. We show these two matrices just below.
© Bércesné dr. Novák Ágnes 2015. február 13.
STEP 1. by using elementary row operations
3
2
1
23
1312
3
2
1
333231
232221
131211
100
10
1
B
B
B
A
AA
b
b
b
aaa
aaa
aaa
STEP 2. Write back to a system of equation, and find solution by back – substitutions.
In other words:
System of linear equations:
33133232131
23123222121
13113212111
bxaxaxa
bxaxaxa
bxaxaxa
can be written in the form of matrices product
3
2
1
3
2
1
333231
232221
131211
b
b
b
x
x
x
aaa
aaa
aaa
or we may write it in the form AX=b,
where A=
333231
232221
131211
aaa
aaa
aaa
, X =
3
2
1
x
x
x
, b =
3
2
1
b
b
b
a11x1+a12x2+ a13x3…a1nxn =b1
a21x1+a22x2+ a23x3…a2nxn =b2
a31x1+a32x2+ a33x3…a3nxn =b3
…
am1x1+am2x2+ am3x3…amnxn= bm
System of a linear equations:
mnmmm
n
n
n
…aaaa
…aaaa
…aaaa
…aaaa
321
3333231
2232221
1131211
Matrix of coefficients Augmented matrix:
mmnmmm
n
n
n
b…aaaa
b…aaaa
b…aaaa
b…aaaa
321
3333231
22232221
11131211
3
© Bércesné dr. Novák Ágnes 2015. február 13.
Augmented matrix is
3
2
1
333231
232221
131211
:
b
b
b
aaa
aaa
aaa
bA
Example:
Solution by Gaussian elimination in matrix form:
4x- 2y+ 2z=6
4x+ y -z=3
3x+ 3y+ 3z=18
4x- 2y+ 2z=6
4x+ y -z=3
3x+ 3y+ 3z=18
augment
ed →
matrix
→
/:3 and swap
rows:
/+(-4)* 1.R
/+(-4)* 1.R
↓
/+(-1/2)* 2.R
←
correspond-
ing
←
system
x + y + z=6
-6y -2z=-18
-4z=-12
/: (-6)
→
/: (-4)
x + y + z=6 x=1
y + (1/3)z=3, y=2
z=3
© Bércesné dr. Novák Ágnes 2015. február 13.
Example: Solve the system of linear equations by Gaussion- elimination method
x1 + x2 + 2x3 = 8
- x1 - 2x2 + 3x3 = 1
3x1 - 7 x2 + 4x3 = 10
Solution: Augmented matrix is
10 473
1 321
8 211
STEP 1.
14- 2100
9 510
8 211
R1+R2, -3R1+R3
104- 5200
9- 510
8 211
-R2, 10R2+R3
2100
9510
8211
-R3/52
Equivalent system of equations form is:
x1 + x2 + 2x3 = 8
x2 - 5x3 = -9 x3 = 2
STEP 2. Back Substitution x3 = 2
x2 = 5x3 -9 =10 – 9 =1
x1 = - x2 - 2x3 + 8 = -1 – 4 +8 = 3
Solution is x1 = 3, x2 = 1, x3 = 2.
© Bércesné dr. Novák Ágnes 2015. február 13.
GAUSS – JORDAN ELIMINATION METHOD
3
2
1
3
2
1
333231
232221
131211
100
010
001
B
B
B
b
b
b
aaa
aaa
aaa
Example.4. Solve the system of linear equations by Gauss - Jorden elimination method
x1 + x2 + 2x3 = 8
- x1 - 2x2 + 3x3 = 1
3x1 - 7 x2 + 4x3 = 10
Solution: Augmented matrix is
10 473
1 321
8 211
14- 2100
9 510
8 211
R1+R2, -3R1+R3
104- 5200
9- 510
8 211
-R2, 10R2+R3
2100
9510
8211
-R3/52
2100
1010
4011
-2R3+R1, 5R3+R2
© Bércesné dr. Novák Ágnes 2015. február 13.
2100
1010
3001
-R2+R1
Equivalent system of equations form is:
x1 = 3
x2 = 1
x3 = 2 is the solution of the system.
SYSTEM WITH NO SOLUTION
Example: 6 . Solve the system of linear equations
5161112
22 7 3
14 2
uzyx
uzyx
uzyx
Solution:
Augmented matrix is:
51611121
22731
14121
Reducing it to row echelon form (using Gaussian - elimination method)
41212100
16650
14121
R2- R1, R3-R1
30000
16650
14121
-R3+2R2
Last equation is
-30 but
30000
uzyx
hence there is no solution for the given system of linear equations.
© Bércesné dr. Novák Ágnes 2015. február 13.
An importamt special case - homogenous system: constants are 0.
Example:
There is always one solution, the so called trivial solution: x=y=z=0
Now we prove that in this case there is no other solution (follow the arrows):
However, in other cases there might be non-trivial solutions as well. IN this case, there are
infinitely many solutions.
Example: Full zero rows are neglected belw, since they do not hold information. Follow
the arrows.
.
x=p , y=-2p z=p, pR.
4x- 2y+2z=0
4x+ y- z=0
3x+ 3y+3z=0
→
/:3 and
change the
rows
/+(-1)* 3.R
/+2*3.R
↓
/+(-1/2)* 2.R
← ←
/:(-4)
↓*
→
/:(-6) →
/+(-1)* 2.R
↓
x =0 6y
y=0
z=0
↓
-x + z=0
2x+3y+4z=0
5x+6y+7z=0
/+2* 1.R
/+5*1.R /+(-2)* 2.R
/* (1/3)
-x + z=0
y+2z=0
→
←
210
001
000
630
101
630
101
1260
630
101
765
432
101
© Bércesné dr. Novák Ágnes 2015. február 13.
Conditions on Solutions
Example:For which values of ‘a’ will be following system
2)14(4
2 53
4 32
2
azayx
zyx
zyx
(i) infinitely many solutions?
(ii) No solution?
(iii) Exactly one solution?
Solution:
Augmented matrix is
21414
2513
4321
2 aa
Reducing it to reduced row echelon form
14270
101470
4321
2 aa
R2-3R1, R3-4R1
41600
210
4321
2
710
aa
71 R2, R3-R2
writing in the equation form,
4)4( 4)(
as written becan 3equation or
3 4)16(
2 2
1 4 32
2
710
azaa
aza
zy
zyx
© Bércesné dr. Novák Ágnes 2015. február 13.
CASE I .
00 4 za
2
4 32
710
zy
zyx
as number of equations are less than number of unknowns, hence the system has
infinite many solutions,
let z = t
78
720
710
434
2
tttx
ty
where ‘t’ is any real number.
CASE II
, -80but , -80z 4 a hence, there is no solution.
CASE III
1let ,4,4 aaa
5
1z
-315z-
41)41)(41( .3. Equatins
z
35
47
35
64
5
3
35
64
5
2
7
10
)(24
x
y
the system will have unique solution when a 4 and a -4
and for a=1 the solution is
. and ,51
3564
3547 zyx
NOTE: (i) a=-4, no solution,
(ii) a=4, infinite many solutions and
(iii)a 4, a -4, exactly one solution .
© Bércesné dr. Novák Ágnes 2015. február 13.
Example:8. What conditions must a, b, and c satisfy in order for the system of
equations
czyx
bzx
azyx
32
2
to be consistent.
Solution: The augmented matrix is
c
b
a
312
101
211
reducing it to reduced row echelon form
ac
ab
a
2110
110
211
R2-R1, R3-2R1
bac
ab
a
000
110
211
R3-R1
The system will be consistent if only if c – a - b = 0
or c = a + b
Thus the required condition for system to be consistent is
c = a + b.
© Bércesné dr. Novák Ágnes 2015. február 13.
INVERSE MATRIX
We next consider a method of solutions based on the inverse of the matrix of coefficients.
Matrix multiplication is a well defined operation, and the definition of a matrix inverse
follows from this operation. Suppose that matrix A has an inverse, denoted A-1.
Definition: AA-1 = A-1A = I, where I is the identity matrix.
For a 3-by-3 matrix I =
100
010
001
.
The following theorem is important to know: if you find one inverse – you find all of them!
Theorem: For each associative operation the inverse is unique.
Proof: Indirectly, suppose, that there are 2 inverses, A* and A-1.
A*=A*I=A*(A A-1)=(A*A) A-1 =I A-1= A-1
Calculating inverse matrix
Suppose we have:
531
532
211
A
333231
232221
131211
1
xxx
xxx
xxx
A
Substituting into the definition:
3
333231
232221
131211
1
100
010
001
531
532
211
E
xxx
xxx
xxx
AA
© Bércesné dr. Novák Ágnes 2015. február 13.
So, we need to solve 3 system of linear equations, having the same
coefficient matrix:
0
0
1
531
532
211
31
21
11
31
21
11
x
x
x
x
x
x
A ,
0
1
0
531
532
211
32
22
12
32
22
12
x
x
x
x
x
x
A ,
1
0
0
531
532
211
33
23
13
33
23
13
x
x
x
x
x
x
A
These identical coefficient matrices mean, that the steps during the Gauss-Jordan elimination
must be exactly the same for each solution, no matter, that the positions in the augmented
matrix for the constants are different:
1
0
0
,
0
1
0
,
0
0
1
0
0
1
5
5
2
3
3
1
1
2
1
,
0
1
0
5
5
2
3
3
1
1
2
1
,
1
0
0
5
5
2
3
3
1
1
2
1
.
1
0
1
3
5
2
2
3
1
0
2
1
,
0
1
0
3
5
2
2
3
1
0
2
1
,
1
0
0
3
5
2
2
3
1
0
2
1
.
SAME NEW COEFFICIENTS
© Bércesné dr. Novák Ágnes 2015. február 13.
1
0
0
0
1
0
0
0
1
5
5
2
3
3
1
1
2
1
)1()3()3(.
Matrix A IDENTITY Matrix I
1
0
0
0
1
0
1
0
1
3
5
2
2
3
1
0
2
1
)1(*2)2()2()1()3()3(
1
0
0
0
1
0
1
2
1
3
1
2
2
1
1
0
0
1
)2(*1)2(
1
0
0
0
1
0
1
2
1
3
1
2
2
1
1
0
0
1
)2(*2)3()3()2()1()1(
1
0
0
2
1
1
3
2
3
1
1
1
0
1
0
0
0
1
)3()2()2()3()1()1(
1
1
1
2
3
1
3
5
0
1
0
0
0
1
0
0
0
1
© Bércesné dr. Novák Ágnes 2015. február 13.
The inverse to A is:
123
135
110
Please check by multiplication.
Example (calculated in the lecture, here is a simplier calculation):
Find the inverse of matrix
521
213
132
The augmented matrix:
1
0
0
0
1
0
0
0
1
5-21
213
1-32
Changing 1st and last row:
001-132
010213
100-521
1st elimination step, zeros in the 1st column:
-2019-10
-31017-50
100-521
© Bércesné dr. Novák Ágnes 2015. február 13.
Changing 2nd and 3rd rows:
-31017-50
20-1-910
100-521
2nd elimination step: zeros in 2nd column (up and down):
71-5-2800
20-1-910
-3021301
Dividing last row by -28:
-1/4-1/285/28100
20-1-910
-3021301
3rd elimination step: zeros in 3rd column:
-1/4-1/285/28100
-1/4-9/2817/28010
1/413/28-9/28001
so the inverse is:
715
7917
7139
28
1
Check it by multiplication!
© Bércesné dr. Novák Ágnes 2015. február 13.
MATRIX INVERSE SUMMARY
Finding an inverse means to solve n systems of n linear equations,
respectively.
So. we have the nn 2 augmented matrix:
1. nn 2 kibővített mátrix:
1
0
0
0
1
0
0
0
1
2
1
2
22
12
1
21
11
nn
n
n
nn
n
a
a
a
a
a
a
a
a
a
EA
from this, applying Gauss-Jordan elimination we get:
nn
n
n
nn
nn
a
a
a
a
a
a
a
a
a
AI
2
1
2
22
12
1
21
11
1
1
0
0
0
1
0
0
0
1