Section 2.1 Matrix Operations

ELOs:

• Be able to perform matrix operations (addition and multiplication).

• Be able to describe ways in which matrix properties are similar and di↵erent to those of real numbers.

• Be able to find the transpose of a matrix.

Matrix Notation:

Recall that we denote an m ⇥ n matrix A where m represents the number of rows and n represents

the number of columns in the following ways:

A =⇥a1 a2 . . . an

⇤=

2

6666664

a11 . . . a1j . . . a1n...

.

.

....

ai1 . . . aij . . . ain...

.

.

....

am1 . . . amj . . . amn

3

7777775= [aij ] .

The ith scalar entry of the jth column is denoted aij . The main diagonal entries are a11, a22, a33, . . .

Definition 1 A diagonal matrix is a square n⇥ n matrix whose nondiagonal entries are zero.

Note: An example of a diagonal matrix is the identity matrix, In.

Example:

Definition 2 A zero matrix is an m⇥ n matrix whose entries are all zero.

Note: A zero matrix is denoted as 0.

Example:

we've been working w matrices and we'll work more2 I

with them in future

This sectiongoes over some

algebraic operations w matrices

Tcolumnvectra diagonalentries

must be square

t i c

Oo f gcan be any size ataee

o

e

Sums and Scalar Multiples of Matrices

• Two matrices, A and B, are said to be equal if they are the same size, m⇥ n, with the same corre-

sponding entries in each column. That is, aij = bij for i = 1, 2, ...,m and j = 1, 2, ..., n.

Example:

• Two matrices, A and B, of the same size may be added together entrywise. That is, aij + bij for

i = 1, 2, ...,m and j = 1, 2, ..., n.

Example:

• A matrix A may be multiplied by a scalar r entrywise. That is, rA = r [aij ].

Example:

Theorem 1 Properties of Matrix Addition and Scalar Multiplication Let A, B, and C bematrices of the same size, m⇥ n, and let r and s be scalars.

(a) A+B =

(b) (A+B) + C =

(c) A+ 0 =

(d) r(A+B) =

(e) (r + s)A =

(f) r(sA) =

2 I

A B if A fi sand B 24 s

a fi's ft 3

QT AtB

A

oneIII as

RA 5A

BTA fondrdimIEatnity rA rB distributivityAHBthlaassdodiiatffity rAtsA

A Cadditre rs A associativityidentity is0 matrixof right size

Matrix Multiplication as Function Composition

Key Idea: Multiplication of matrices corresponds to composition of linear transformations!

Definition 3 If A is an m⇥ n matrix, and if B is an n⇥ p matrix with columns b1,b2, . . . ,bp, thenthe product of AB is the m⇥ p matrix whose columns are Ab1, Ab2, . . . , Abp. That is,

AB = A⇥b1 b2 . . . bp

⇤=

⇥Ab1 Ab2 . . . Abp

⇤

Thus, A(Bx) = (AB)x for all x 2 Rp.

Note: Each column of AB is a linear combination of the columns of A where the weights arethe entries of the corresponding column of B.

Example:

2 I

AtET El FM

Mutt byAB

ex B rotates I by 45

A stretches its input by5

ABI first rotates 5 by 45then stretches

that rotated vector byfactor of 5

composite of functions is like Layering

of transformations

let's practice matrix multiplication

Afly

B

102243,0

3 2

AB will be 3212 4 3 4 matrix

2 IQi

f p

Row-Column Rule for Computing AB

If the product AB is defined, then the entry in row i and column j of AB is the sum of the products

of corresponding entries from row i of A and column j of B. That is,

(AB)ij = ai1b1j + ai2b2j + · · ·+ ainbnj

where A is an m⇥ n matrix. Note: rowi(AB) = rowi(A) ·B

Example:

Example:

Theorem 2 Properties of Matrix Multiplication Let A,B, and C be matrices and r be a scalarsuch that the sums and products below are defined. Then,

(a) A(BC) =

(b) A(B + C) =

(c) (B + C)A =

(d) r(AB) =

(e) ImA =

2 I

2 2 2 3

A fog y B f's 5.4 Issuingmatrix

AB z fo 4

µol c t 4C3 L2

AB z 9 2 EyLst 8 10

twice

Eti

AB L associativity of meltABtAc distributivityBAKA

RA B AGB Ar B associativity commutativityofscalar maltA mutt identity

WARNINGS:

• Matrix multiplication is NOT commutative. That is, in general, AB 6= BA.

• Cancellation laws do NOT hold. That is, AB = AC does NOT imply B = C

• Matrices CAN have zero divisors. That is, AB = 0 does NOT imply A = 0 or B = 0.

• We CAN take powers of matrices as long as they are n⇥ n. That is, Ak= A · · ·A| {z }

k times

and A0= In.

Example:

Example:

2 I

A E E 3 Inotice neither A nor B o

a I's X 3 fiBA FIGHT

Transpose of a Matrix, AT

Definition 4 Given an m⇥ n matrix A, the transpose of A is the n⇥m matrix, denoted AT , whosecolumns are formed from the corresponding rows of A.

Example: Give the transpose of each of the following matrices:

A =

✓a bc d

◆, B =

0

@�5 2

1 �3

0 4

1

A , C =

✓1 1 1 1

�3 5 �2 7

◆

Theorem 3 Transpose Properties Let A and B be matrices whose sizes are appropriate for thefollowing sums and products.

(a) (AT)T =

(b) (A+B)T=

(c) For any scalar r, (rA)T =

(d) (AB)T=

Note: The transpose of a product of matrices equals the product of their transposes in reverse order.

Example:

2 I

QI

A RATAT BT BTAT

f If f sogAB

II Is 2gest f Ig Io

a at f lies I FI Iso ou

Section 2.2 The Inverse of a Matrix

ELOs:

• Be able to explain (at least) three ways to identify whether or not a matrix is invertible.

• Be able to use the inverse of a 2⇥ 2 matrix to solve a system of equations.

• Be able to use algorithm to find matrix inverses.

Key Idea: A matrix transformation A has an inverse, denoted A�1, when A is one-to-one and onto.

Suppose A is an m⇥ n matrix, such that A : Rn ! Rm.

(a) Will the matrix A have an inverse if n > m? Why or why not?

(b) Will the matrix A have an inverse if n < m? Why or why not?

(c) What about n = m?

(d) Based on the previous observations, what must be true about the size of A in order for A�1to exist?

(e) Based on the previous observations, what does RREF(A) look like?

Definition 1 A square, n⇥n, matrix is said to be invertible (or nonsingular) if there exists an n⇥nmatrix, denoted A�1, such that

A�1A = In and In = AA�1

.Note: A�1 is called the inverse of A. A matrix that is not invertible is said to be singular.

Matrices are fins Sore fins have inverses 2.2Some matriceshave inverses

A is mxn matrix

maybeA has n columns to spas 1km we must have

men

but if we have n din indep columns then

span of RM IR

mm must be true for A to exist

notice fin analog fcf 43 f HD Xexists

Example:

Theorem 4 2⇥ 2 Inverses Let A =

a bc d

�. If ad� bc 6= 0, then A is invertible, and

A�1=

1

ad� bc

d �b

�c a

�.

If ad� bc = 0, then A is not invertible (is singular).

The quantity ad� bc is called the determinant of A.

Thus, a 2⇥ 2 matrix is invertible if and only if det A 6= 0.

Example:

A 5gcheck if B fog 25g

22

is A

qq.caFind A if A 2 Iz

find Atif A I 36

Theorem 5 Matrix Equation Solutions and Inverses If A is an invertible n ⇥ n matrix, thenfor each b 2 Rn, the equation Ax = b has the unique solution, x = A�1

b.

Example:

Theorem 6 Properties of Inverses

(a) If A is an invertible matrix, then A�1 is invertible and

(A�1)�1

=

(b) If A and B are n ⇥ n invertible matrices, then AB is also invertible. The inverse of AB is theproduct of the inverses of A and B in reverse order. That is,

(AB)�1

=

(c) If A is an invertible matrix, then AT is also invertible. The inverse of AT is the transpose of A�1.That is,

(AT)�1

=

Extension: The product of n ⇥ n invertible matrices is invertible, and the inverse is the product of the

individual inverses in reverse order.

2 2

Yay This gives another wayUse Them5 to solve to solve a system of linearX t 3 2 7 equs4x 5 2 7

we can define elementary matrix as a matrix

we get from performing one ERO on In

Ei fooEL L

these are all elementary matriceswe can perform row ops by multiplying by several

elementary matrices

Theorem 7 An n⇥n matrix A is invertible if and only if A is row equivalent to In, and in this case,any sequence of elementary row operations that reduces A to In also transforms In to A�1.

Algorithm for Finding A�1

1) Row reduce the augmented matrix⇥A I

⇤.

2) If A is row equivalent to I, then⇥A I

⇤is equivalent to

⇥I A�1

⇤.

3) Otherwise, A does not have an inverse.

Example:

Matrix Inversion as Simultaneously Solving n Linear Systems

Facts 1 Elem matrices are invertible 2 E is now opthat forms E back to I 2.2

Thisgives us the following algorithm µdform

a at a A f e gfaYi

2 o l

i

iI A

Section 2.3 Characterizations of Invertible Matrices

ELOs:

• Be able to relate the equivalent statements we learned in Chapter 1 to the Invertible Matrix Theorem.

• Be able to apply the Invertible Matrix Theorem to determine whether or not a matrix is invertible.

Theorem 8 The Invertible Matrix Theorem Let A be a square n⇥n matrix. Then the followingstatements are equivalent. That is, for a given A, the statements are either all true or all false.

(a) A is an invertible matrix.

(b) A is row equivalent to the n⇥ n matrix.

(c) A has postions.

(d) The equation Ax = 0 has only the solution.

(e) The columns of A form a linearly set.

(f) The linear transformation x 7! Ax is .

(g) The equation Ax = b has solution for each b in Rn.

(h) The columns of A Rn.

(i) The linear transformation x 7! Ax maps Rn Rn.

(j) There is an n⇥ n matrix C such that CA = .

(k) There is an n⇥ n matrix D such that AD =

(l) AT is an matrix.

Proof:

today's Relating everything we've learned so far 2.3

furiportant point

identityn pivot

trivialindepone to one

at least onespan

ontoII

invertible

hFE Is A so I 4 EJinvertible

O O 4 I0 O O 9

Invertible Linear Transformations

Definition 1 A linear transformation T : Rn �! Rn is said to be invertible if there exists a functionS : Rn �! Rn such that

S(T (x)) = x for all x 2 Rn

T (S(x)) = x for all x 2 Rn

where S = T�1 is the inverse of T .

Theorem�9�Let�T�:�Rn��!�Rn�be�a�linear�transformation�and�let�A�be�the�standard�matrix�for�T�.�Then,�T�is�invertible�if�and�only�if�A�is�an�invertible�matrix,�and�the�linear�transformation�S�given�by�S(x)�=�A�1

x�is�the�unique�function�such�that

A�1(Ax)�=�S(T�(x))�=�x�for�all�x�2�Rn

A(A�1x)�=�T�(S(x))�=�x�for�all�x�2�Rn

where�S�=�T��1�is�the�inverse�of�T�.

Proof:

2.3

y AssumeT exists Then

TCT T T D I V X E IR i.e

T is onto 112 because if 8 is in

IR and 5 145 then TCE TEk5So each 5 is in range of T Thus A

exists

Assume A exists Let SCE Atx ThenS is a lin operator and S Text _SHI

A AE _I T I exist

Section 2.4 Partitioned Matrices

ELOs:

• Be able to multiply partitioned (block) matrices.

Example:

We can express the matrix

A =

2

43 0 �1 5 9 �2

�5 2 4 0 �3 1�8 �6 3 1 7 �4

3

5

as the 2⇥ 3 partitioned (or block) matrix

A11 A12 A13

A21 A22 A23

�

whose entries are submatrices (or “blocks”).

• We can multiply partitioned matrices the same way as non-partitioned matrices, provided the productAB makes sense for each block A and B. Find the product AB where

A =

2

42 �1 1 0 41 5 �2 3 �10 �4 �2 7 �1

3

5 =

A11 A12

A21 A22

�, B =

2

66664

6 4�2 1�3 7�1 35 2

3

77775=

B1

B2

�.

Sometimes its useful to treat a matrix in 2.4blocks It might show up naturally in

applications and also

might reduce computingpower necessary

for largematrix

it i

theyrepartitioned

in rightsizes A L I 1 An f I

Az o 4 2 Azz 7 I

B Biff

area Kant f 3

• A matrix of the form

A =

A11 A12

0 A22

�

is called block upper triangular. Assuming A is invertible, A11 is a p ⇥ p matrix, and A22 is a q ⇥ qmatrix. Find a formula for A�1. That is, find a matrix B such that AB = Ip+q.

2 4

qAB Iris off An jpg B's LIEq

A B 1AzBu Ip AzzBz D

AI AzzBa AI O

B

AcBiz 1 AzBzz O AzzBz IqAI AnBa Aid Ig

finish

Bzz A

A

Section 2.5 Matrix Factorizations

ELOs:

• Be able to identify why an LU factorization of a matrix is useful.

• Be able to compute the LU factorization of a matrix A.

LU Factorization

Suppose we want to solve the set of equations

Ax = b1

Ax = b2

.

.

.

Ax = bp

where A is the same matrix in each equation.

• If A is invertible, what is the (unique) solution to each equation?

• What if A is not invertible?

Idea: If we could rewrite A in a clever way such that A = LU where L is a lower triangularmatrix and U is an upper triangular matrix, this would reduce the number of steps needed to solve

the problem. If we assume that an m⇥ n matrix A can be reduced to REF without row exchanges,

then we can perform this factorization A = LU . Here’s an example:

A =

2

664

1 0 0 0

⇤ 1 0 0

⇤ ⇤ 1 0

⇤ ⇤ ⇤ 1

3

775

| {z }L(m⇥m)

2

664

⇤ ⇤ ⇤ ⇤ ⇤0 ⇤ ⇤ ⇤ ⇤0 0 0 ⇤ ⇤0 0 0 0 0

3

775

| {z }U(m⇥n)

Example:

Here we'll focus on one type of matrix 2.5factorization

i.e write matrix A in somefactored form

A LU

each of these isa linear systemof eques

QfO

ufactorizationfrequently

on

A ft't z 6 a wssedgatI 3 8 O 7 matrices0 6 8 9 13

L's lt i'youu0 O O 5 I

L U

Why is this helpful2.5

It is more computationally efficient

A

f E3 6 26 2

l iO O O I

A Lu check this for yourself

Use this factorization to solve A'x B for

IIImensa

80 1124

1 ue.TT 175Mutt byU

Mutt by L

ie AEE EsFEESUI I

ME 1 s pas

I O l O

matrix 3 4 2 I 5

iii I iiisteps 6 maltGadd

f i 3augmentedmatrix forUI I

E'fhowmany o o l O 391steps 6Mutt o o o I 1976add

IFinding I took 28 operations FLOPS

Row reduction of A 53 to fix takes62 FLOPS

Using A _LU is more efficient

LU Factorization Algorithm

1. Reduce A to row echelon form, REF(A)=U , using only row replacement operations.

Note: this is not always possible.

2. Place entries in L such that the same sequence of row operations reduces L to I.

Example: Find an LU factorization of

A =

2

664

2 4 �1 5 �2

�4 �5 3 �8 1

2 �5 �4 1 8

�6 0 7 �3 1

3

775

these are lower2.5

s their productEp E U is

er S

A Ep E U LU

L Ep EEp Ei e A Uep EL I

3 Zt l 0 O O

a E f Iii6 O 7 3 I 0 O O 1

it iiO 12 4 12 5 3 0 I

p i SO O O 4 7 5 4 O l

U I

Find L

t f ii gO 40 I 5001

i O O O I 3 4 0 I

O O o 4 7

L y

f ya

6 0 7 3 I