chapters 2 & 3: matrices, systems of linear equations, …gressman/overheads_chap2.pdf ·...

Chapters 2 & 3: Matrices, Systems of LinearEquations, and Determinants

Philip Gressman

University of Pennsylvania

Philip Gressman Math 240 002 2014C: Chapter 2 1 / 36

Matrices: Definitions, Notation, and Systems of Equations

§2.1–2.3 Main Topics

• Matrices: notation, row and column vectors, transposition,diagonal matrices, traces, symmetric and antisymmetricmatrices; matrix of coefficients A and the augmented matrixA] of systems of linear equations

• Matrix Algebra: addition and multiplication, scalarmultiplication of matrices; connections to systems ofequations; Kronecker delta symbol; algebraic laws for matrices(addition, multiplication, and transposition)

• Systems of Linear Equations: Unknowns, coefficients,homogeneous versus inhomogeneous, solutions and solutionsets, consistent versus inconsistent


Matrices

• A matrix is any rectangular array of items (numbers,functions, etc.) The items themselves are called entries orelements of the matrix. A matrix with n rows and m columnsis said to have size n ×m (read: n by m)–i.e., rows alwayscome first.

• Entries of a matrix are labelled with a pair of subscripts: for amatrix A, the entry in row i and column j is usually denotedaij or Aij (again, for consistency, the row index always comesfirst).

• An example:

A :=

4 x 0 −2π −3 1 12 π2 3 x2

Here A is a 3× 4 matrix; A23 = 1 and A32 = π2.

• Two matrices are equal when they have the same size and allcorresponding entries agree.


Matrix Operations

• Any matrix can be multiplied by a constant or a functionthrough the operation called scalar multiplication. Just likevectors, you multiply every entry by the same factor:

A :=

4 x 0 −2π −3 1 12 π2 3 x2

, 3A =

12 3x 0 −63π −9 3 36 3π2 9 3x2

In formulas: (kA)ij = k(Aij).

• If the matrices A and B have the same size, they may beadded together. The formula for matrix addition is(A + B)ij := Aij + Bij . Informally, corresponding entries sum:[

1 2 34 5 6

]+

[9 7 58 6 4

]=

[10 9 812 11 10

].

Matrix addition is commutative: A + B = B + A.


Matrix Multiplication

Matrices of compatible sizes may be multiplied: We define thematrix product AB when the number of columns of A matchesthe number of rows of B.

• If A is n ×m and B is m × p, then the product AB will be amatrix of size n × p.

• In formulas: (AB)ik :=∑m

j=1 AijBjk . In other words: theik-entry of AB is computed by taking a dot product of row iof matrix A with column k of matrix B.

• An example:3 x 0−4 x2 12 x3 01 −1 1

2 −2

3 1−3 9

=

3x + 6 x − 6

3x2 − 11 x2 + 173x3 + 4 x3 − 4−4 6

Matrix multiplication is not commutative!


Multiplication Exercise

4 4 −1 −14 −4 −1 1−2 −1 2 1

1 1 1 1 11 −1 2 −2 61 1 4 4 41 −1 8 −8 24

• Is this multiplication well-defined? What will be the

dimensions of the product?

• Compute the entries.


Why this definition of multiplication?

1. Economy of notation for systems of linear equations:

3x + 2y + z = 86x + 5y + 3z = 0

2x + 9y + z = 1⇔

3 2 16 5 32 9 1

xyz

=

801

.2. Compatibility with geometric operations like rotation: rotationby angle θ in the plane is accomplished via multiplication of 2× 1matrices and the matrix[

cos θ − sin θsin θ cos θ

].

Angle addition formulas have a nice structure:[cos(θ + φ) − sin(θ + φ)sin(θ + φ) cos(θ + φ)

]=

[cos θ − sin θsin θ cos θ

] [cosφ − sinφsinφ cosφ

]Philip Gressman Math 240 002 2014C: Chapter 2 7 / 36

Laws of Matrix Algebra

Associative Laws:

A + (B + C ) = (A + B) + C (addition)

k1(k2A) = (k1k2)A (scalar mult.)

A(BC ) = (AB)C (matrix mult.)

Distributive Laws:

k(A + B) = kA + kB

(k1 + k2)A = k1A + k2A

A(B + C ) = AB + AC

(A + B)C = AC + BC


The Transpose

The transpose of the n ×m matrix A is denoted AT and refers tothe m× n matrix where rows and columns have been interchanged:ATij = Aji . For example:

[1 2 34 5 6

]T=

1 42 53 6

.Laws of Transposition:

(AT )T = A

(A + B)T = BT + AT

(AB)T = BTAT

(kA)T = k(AT )

Note: The order of multiplication gets reversed when youtranspose!


Tons of Terminology

There’s lots of terminology to learn: A square matrix is one withequal numbers of rows and columns; a column vector is a matrixwith only one column; a row vector is a matrix with only one row.Upper Triangular matrices have Aij = 0 when i > j . LowerTriangular matrices have Aij = 0 when i < j (note that the bookgets the definitions backwards!). Diagonal means that all entriesAij are zero when i 6= j (so any nonzero entries must reside on thediagonal from top left to bottom right). Identity means that thematrix is square and has ones on the diagonal and zeros elsewhere.Symmetric means that it equals its own transpose. SkewSymmetric means that it is equal to the opposite of its owntranspose.


Systems of Linear Equations

A system of n linear equations in m unknowns has the form:

a11x1 + a12x2 + · · ·+ a1(m−1)xm−1 + a1mxm = b1

a21x1 + a22x2 + · · ·+ a2(m−1)xm−1 + a2mxm = b2...

an1x1 + an2x2 + · · ·+ an(m−1)xm−1 + anmxm = bn

If b1 = b2 = · · · = bn = 0, the system is called homogeneous(otherwise it is called inhomogeneous). A solution is a vector~x := 〈x1, . . . , xm〉 whose coordinates satisfy all the equationssimultaneously.

Fundamental Fact

Any system of linear equations has either no solutions (called aninconsistent system), exactly one solution (called a uniquesolution), or infinitely many solutions.


Augmented Matrices

a11x1 + a12x2 + · · ·+ a1(m−1)xm−1 + a1mxm = b1...

an1x1 + an2x2 + · · ·+ an(m−1)xm−1 + anmxm = bn

is encoded by the augmented matrix of coefficients: a11 · · · a1n b1...

. . ....

...an1 · · · anm bn

.• Given an augmented matrix, you can recover the system.• What happens when you multiply the augmented matrix by

x1...xm−1

?


Solving Systems of Linear Equations

§2.4–2.6 Main Topics

• Elementary row operations: Manipulations of systems ofequations which preserve the solution set

• Gaussian elimination: Using Elementary row operations tosolve systems of linear equations

• Matrix Inversion: For solving “nice” systems.


Elementary Row Operations

• Pij : Permute (i.e., interchange) row i and row j −1 0 12 3 40 1 −1

P13;

0 1 −12 3 4−1 0 1

• Mi (α): Multiply row i by the constant α 6= 0. −1 0 1

2 3 40 1 −1

M2(12)

;

−1 0 11 3

2 20 1 −1

• Aij(α): Add α times row i to row j . −1 0 1

2 3 40 1 −1

M21(3);

5 9 132 3 40 1 −1

• Elementary row operations can also be achieved by

multiplication on the left by a corresponding elementarymatrix.


Elementary Matrices

Identify what elementary row operations these matrices producewhen multiplied on the left of an arbitrary matrix.

0 0 1 00 1 0 01 0 0 00 0 0 1

,

1 0 0 00 1 0 30 0 1 00 0 0 1

,

1 0 0 00 1 0 00 0 −7 00 0 0 1

.• What elementary matrix would permute row 2 and row 4 in a

matrix with 6 rows?

• What elementary matrix would multiply row 3 by −5 in amatrix with 5 rows?

• What elementary matrix would add 3 times row 4 to row 2 ina matrix with 4 rows?


Row Reduction Exercise

When you are working on a problem in which row reduction is asensible thing to do, you might want to give it a try if having morezeros in your matrix would make your problem easer.Apply elementary row operations to the following matrix and seehow many entries you can force to equal zero: 2 1 1

1 −1 23 2 1

.


Row Echelon Form

• The first nonzero entry of each row is a one (or the row hasonly zero entries).

• No two rows begin in the same column, and rows are arrangedso that they begin further to the right as you move down.

• Rows of all zeros (if any) go at the bottom. Three examples:

• Columns with a leading 1 (green boxes) are called pivotalcolumns. The green boxes are also called pivotal positions.

• The matrix is in reduced row echelon form if the entriesabove every pivotal position are also all zero.

• If your matrix of coefficients is in row echelon form, you caneasily solve the system using back substitution.


Row Reduction

You are going to proceed by cooking up the row echelon form onerow at a time, starting at the top and working your way down.Once a row is found, you are going to “freeze it” and effectivelyignore it from that point forward.

1 Find the leftmost column which is nonzero (ignore rows, ifany, which are frozen). This will be a pivotal column.

2 Permute unfrozen rows so that the top unfrozen entry in yourpivotal column is not a zero.

3 Multiply the top unfrozen row by an appropriate constant sothat it has a 1 in the pivotal position. (Can be tricky withcomplex matrices.)

4 Apply elementary row operations Aij(α) to all the subsequentrows so that you systematically obtain all zero entries belowthe pivotal position.

5 Freeze the the row you just worked on.6 Repeat these steps on the unfrozen rows as necessary.

The Rank of a matrix is the # of all zero rows in its REF.Philip Gressman Math 240 002 2014C: Chapter 2 18 / 36

Reduced Row Echelon Form

Reduced Row Echelon Form is when every pivotal column of arow echelon matrix has zeros everywhere except the pivotalposition. You can always put a row echelon matrix in reduced rowechelon form by adding multiples of rows below to rows above(e.g., add a multiple of row 2 to row 1). Example pictures:

The reduced row echelon form of any matrix A is unique, meaningthat any two people who row reduce A until they arrive at a RREFmatrix (no matter what procedure they use) must ultimately getthe same answer. This is not always true for non-reduced rowechelon matrices.


Gaussian and Gauss-Jordan Elimination: §2.5

1 Take the system of equations and form the Augmented Matrix

2 Row reduce until you are in reduced row echelon form.

3 Convert the resulting matrix back into a system of equations.

4 Variables corresponding to pivotal columns are called boundbecause their values are forced.

5 Variables from non-pivotal columns (if any) are called freesince their values could be anything.

6 If you end up with equations of the form 0 = 0 at the end,just ignore. If you end up with 0 = 1, this means theequations are inconsistent.


Example 1: Inconsistent Systems

x1 + x2 + x3 + 3x4 = 22x1 + x2 + 2x3 + 5x4 = 3x1 − x2 + x3 + x4 = 1

This system of equations has no solutions. You might say that thesolution set is the empty set: ∅. Systems of any size can beinconsistent. Formally, a system will be inconsistent exactly whenthe rank of the matrix of coefficients is smaller than the rank ofthe augmented matrix. This happens exactly when the augmentedmatrix can be row reduced to contain a row in which the very lastentry is pivotal.


Example 2: Underdetermined Systems

x1 + x2 + x3 + 3x4 = 22x1 + x2 + 2x3 + 5x4 = 3x1 − x2 + x3 + x4 = 0

Solution Set:

{(1− s − 2t, 1− t, t, s) | t, s ∈ R}

The scalars s and t correspond to free variables x3 and x4. Thenumber of bound variables is always the rank of the matrix ofcoefficients (not the rank of the augmented matrix). The numberof free variables is the total number of variables minus the boundones.


Example 3: Unique Solutions

x1 + x2 + x3 + x4 = 12x1 + 2x3 = 0−x1 + 2x2 + x3 − 2x4 = −4

x2 − x4 = −1

Unique Solution:(1, 0,−1, 1)

Unique Solutions occur exactly when the matrix of coefficients (notthe augmented matrix) has rank equal to the number of variables.If the number of equations equals the number of variables, this isthe same as saying that the matrix of coefficients can berow-reduced to the identity matrix. If the number of equations isless than the number of variables, the solution is never unique.


Exercise

Determine whether the following system is inconsistent, has aunique solution, or has infinitely many solutions. Then give acomplete description of the solution set.

y1 + 2y2 + y3 + y4 = 1y1 + 2y2 − y3 − y4 = 1

3y1 + 6y2 − y3 + y4 = 12y1 + 4y2 + 2y3 + 3y4 = 1


Applications

Balancing Chemical Equations

�C5H8 + �O2 → �CO2 + �H2O

The unknowns are the coefficients; the equations come fromatomic conservation (each element should appear the samenumber of times on each side).

Physical Statics

Vector Forces should sum to zero at every point.


Applications

Resistance Networks

At any point, incoming current equals outgoing current (currentflows from positive to negative by convention). The sum ofpotential differences (voltage drops) around any closed loop equalszero; use V = IR and add a minus sign if you cross the resistor inthe same direction as the assumed current (no minus sign if goingthe opposite direction).


Inverse Matrices: §2.6

• Every square matrix A of full rank has an inverse matrix A−1

with the property that

A−1A = AA−1 = I .

• To find A−1 by the Gauss-Jordan technique: augment A onthe right with an identity matrix. Row reduce until the lefthalf of the augmented matrix is an identity. The right half willbe A−1.

• The inverse matrix can be used to efficiently solve systemsAx = b. There are also several important algebraic propertiesof matrix inversion to keep in mind.


Matrix Inversion Exercises

1 0 0 10 1 0 00 1 1 00 0 0 1

−1

=

1 0 0 −10 1 0 00 −1 1 00 0 0 1

1 0 1 01 1 1 1−1 −1 1 1−1 1 1 −1

−1

=

12 0 −1

4 −14

−12

12 −1

414

12 0 1

414

−12

12

14 −1

4


Conditions equivalent to non-invertibility

Theorem

The following are all mathematically equivalent statements aboutthe square matrix A:

1 A is singular (i.e., it does not have an inverse).

2 There is a column vector X 6= 0 for which AX = 0.

3 The row-echelon form of A has at least one row of all zeros.

4 The reduced row-echelon form of A is not an identity matrix.

5 The rank of A is strictly less than the number of rows.

6 The determinant of A equals zero.

7 The equation AX = 0 has infinitely many solutions when X isan unknown column vector.

8 It is possible to find a column vector B such that the equationAX = B is inconsistent when X is an unknown column vector.


Facts about Matrix Inverses

Theorem

Assuming that A and B are invertible matrices of the same size:

1 (A−1)−1 = A.

2 (AB)−1 = B−1A−1.

3 (AT )−1 = (A−1)T .


Determinants: §3.1

The determinant of a square matrix A is a number which hasimportant algebraic and geometric properties. You’ve probablyseen it for 2× 2 and 3× 3 matrices; the formula gets increasinglymore complicated as the size of the square matrix grows. We’llstart developing the formula now.

A permutation of the integers 1, . . . , n is any ordering of thesenumbers. There are a total of n! such permutations.

To each permutation, we may assign a signature σ which is equalto +1 or −1. Permutations with signature +1 are called even andpermutations with signature −1 are called odd. Rule: Count thenumber of inversions–the number of pairs which appear in thewrong order–and then set the signature +1 if the number ofinversions is even and odd otherwise.


Formula for the Determinant

detA =∑

all permutations p

σp1p2···pna1p1a2p2 · · · anpn

Example: 2× 2

det

[a11 a12a21 a22

]= a11a22 − a12a21

Example: 3× 3

det

a11 a12 a13a21 a22 a23a31 a32 a33

= a11a22a33 − a12a21a33

− a11a23a32 − a13a22a31

+ a12a23a31 + a13a21a32

In general, this is a long way to compute the determinant.Philip Gressman Math 240 002 2014C: Chapter 2 32 / 36

Terms to compute in a 4× 4 determinant

There are 24 permutations of {1, 2, 3, 4}:

a11a22a33a44 a12a21a33a44 a13a22a31a44 a14a22a33a41a11a23a32a44 a11a24a33a42 a11a22a34a43 a12a21a34a43a13a24a31a42 a14a23a32a41 a12a23a31a44 a13a21a32a44a12a24a33a41 a14a21a33a42 a13a22a34a41 a14a22a31a43a11a23a34a42 a11a24a32a43 a12a23a34a41 a14a21a32a43a14a23a31a42 a13a24a32a41 a13a21a43a42 a12a24a31a43

There are 120 permutations of {1, 2, 3, 4, 5}.


Method 2: Cofactors

1 Pick a row or a column to expand in. Say you pick row i . Foreach j , let A(i |j) be the (n − 1)× (n − 1) matrix obtained byremoving row i and column j . It’s called the (i , j)-minor of A.

2 Take the determinant of the minors A(i |j). The numberCij := (−1)i+j detA(i |j) is called the (i , j)-cofactor of A.

3 Compute the determinant of A using the cofactors:

detA =n∑

j=1

aijCij or =n∑

i=1

aijCij .

This is a recursive algorithm–to compute large determinants, youcompute a large number of smaller determinants. It’s a little easierthan the first method (in terms of bookkeeping), but it’s stillsomewhat lengthy.


Method 3: Row Reduction

The elementary row operations do nice predictable things todeterminants:

1 Multiplying a row by a constant multiplies the determinant bythe same constant.

2 Interchanging the order of two specific rows multiplies thedeterminant by −1.

3 Taking one row and adding a multiple of another row to itleaves the determinant alone.

Last key fact: The determinant of a triangular matrix is just theproduct of the diagonal entries. Using these facts, you can oftendeduce what the determinant should be in far fewer steps than thefirst method.


Other Properties of Determinants

• detAT = detA.

• A matrix with a row (or column) of zeros has zerodeterminant.

• A matrix with two identical rows (or two identical columns)has zero determinant.

• The determinant gives the signed volume of parallelepipeds.

Already Mentioned:

• Interchanging two rows (or columns) changes the determinantby a factor of −1.

• Determinant of triangular is the product of diagonal entries.

• Multiplying a row by k multiplies the determinant by k .

• Adding a multiple of a row to some other row leaves thedeterminant unchanged.

Final Major Property:

• det(AB) = (detA)(detB).


chapters 2 & 3: matrices, systems of linear equations, …gressman/overheads_chap2.pdf ·...

Documents