matrix invertibility - brown university

Matrix invertibility

Rank-Nullity Theorem: For any n-column matrix A,

nullity A+ rank A = n

Corollary: Let A be an R ⇥ C matrix. Then A is invertible if and only if |R | = |C | and thecolumns of A are linearly independent.

Proof: Let F be the field. Define f : FC �! FR by f (x) = Ax.Then A is an invertible matrix if and only if f is an invertible function.

The function f is invertible i↵ dimKer f = 0 and dimFC = dimFR

i↵ nullity A = 0 and |C | = |R |.

nullity A = 0 i↵ dimNull A = 0i↵ Null A = {0}i↵ the only vector x such that Ax = 0 is x = 0i↵ the columns of A are linearly independent. QED

Matrix invertibility examples

1 2 34 5 6

�is not square so cannot be invertible.

1 23 4

�is square and its columns are linearly independent so it is invertible.

2

41 1 22 1 33 1 4

3

5 is square but columns not linearly independent so it is not invertible.

Transpose of invertible matrix is invertible

Theorem: The transpose of an invertible matrix is invertible.

A =

2

4 v1 · · · vn

3

5 =

2

64a1...an

3

75 A

T =

2

4 a1 · · · an

3

5

Proof: Suppose A is invertible. Then A is square and its columns are linearly independent. Letn be the number of columns. Then rank A = n.

Because A is square, it has n rows. By Rank Theorem, rows are linearly independent.

Columns of transpose A

T are rows of A, so columns of AT are linearly independent.

Since A

T is square and columns are linearly independent, AT is invertible. QED

More matrix invertibilityEarlier we proved: If A has an inverse A

�1then AA

�1is identity matrix

Converse: If BA is identity matrix then A and B are inverses? Not always true.

Theorem: Suppose A and B are square matrices such that BA is an identity matrix 1. Then A

and B are inverses of each other.

Proof: To show that A is invertible, need to show its columns are linearly independent.

Let u be any vector such that Au = 0. Then B(Au) = B0 = 0.On the other hand, (BA)u = 1u = u, so u = 0.

This shows A has an inverse A

�1. Now must show B = A

�1. We know AA

�1 = 1.

BA = 1

(BA)A�1 = 1A�1 by multiplying on the right by A

�1

(BA)A�1 = A

�1

B(AA�1) = A

�1 by associativity of matrix-matrix mult

B 1 = A

�1

B = A

�1QED

Representations of vector spaces

Two important ways to represent a vector space:

As the solution set of homogeneous linear systema1 · x = 0, . . . ,am · x = 0

Equivalently, Null

2

64a1...am

3

75

As Span {b1, . . . ,bk}

Equivalently,

Col

2

66664b1 · · · bk

3

77775

Conversions between the two representations

{[x , y , z ] :[4,�1, 1] · [x , y , z ] = 0,[0, 1, 1] · [x , y , z ] = 0}

Span {[1, 2,�2]}

Span {[4,�1, 1], [0, 1, 1]} {[x , y , z ] : [1, 2,�2] · [x , y , z ] = 0}

Conversions for a�ne spaces?

I From representation as solution set of linear system to representation as a�ne hull

I From representation as a�ne hull to representation as solution set of linear system

Conversions for a�ne spaces?From representation as solution set of linear system to representation as a�ne hull

Iinput: linear system Ax = b

Ioutput: vectors whose a�ne hull is the solution set of the linear system.

1 2 1�1 2 2

�2

4x

y

z

3

5 =

12

�

1 Let u be one solution to the linear system. u = [�0.5, 0.75, 0]

2 Consider the corresponding homogeneous system Ax = 0.

1 2 1�1 2 2

�2

4x

y

z

3

5 =

00

�

Its solution set, the null space of A, is a vector space V.3 Let b1, . . . ,bk be generators for V. b1 = [2,�3, 4]4 Then the solution set of the original linear system is

the a�ne hull of u,b1 + u,b2 + u, . . . ,bk + u. [�0.5,�.75, 0] and[�0.5,�.75, 0] + [2,�3, 4]

From representation as solution set to representation as a�ne hull

One solution to equation

1 2 1�1 2 2

�2

4x

y

z

3

5 =

12

�is u = [�0.5, 0.75, 0]

Null space of

1 2 1�1 2 2

�is Span {b1}:

b1

Solution set of equation is u+ Span {b1},i.e. the a�ne hull of u and u+ b1

u+b1

u

Representations of vector spacesTwo important ways to represent a vector space:

As the solution set of homogeneous linear systema1 · x = 0, . . . ,am · x = 0

Equivalently, Null

2

64a1...am

3

75

As Span {b1, . . . ,bk}

Equivalently,

Col

2

66664b1 · · · bk

3

77775

How to transform between these two representations?

Problem 1 (From left to right):I

input: homogeneous linear system a1 · x = 0, . . . ,am · x = 0,I

output: basis b1, . . . ,bk for solution setProblem 2 (From right to left):

Iinput: independent vectors b1, . . . ,bk ,

Ioutput: homogeneous linear system a1 · x = 0, . . . ,am · x = 0 whose solution set equalsSpan {b1, . . . ,bk}

Reformulating Problem 1

Iinput: homogeneous linear system a1 · x = 0, . . . ,am · x = 0,

Ioutput: basis b1, . . . ,bk for solution set

Let’s express this in the language of matrices:

Iinput: matrix A =

2

64a1...am

3

75

with independent rows

Ioutput: matrix B =

2

4 b1 · · · bk

3

5

with independent columns

such that Col B = Null A

Can require the rows of the input matrix A to be linearly independent. (Discarding a superfluousrow does not change the null space of A.)





Iinput: matrix A =

2

64a1...am

3

75

with independent rows

Ioutput: matrix B =

2

4 b1 · · · bk

3

5 with independent columnssuch that Col B = Null A






Iinput: matrix A =

2

64a1...am

3

75 with independent rows

Ioutput: matrix B =

2

4 b1 · · · bk

3

5 with independent columnssuch that Col B = Null A


Reformulating the reformulation of Problem 1

Iinput: matrix A with independent rows

Ioutput: matrix B with independent columns such that Col B = Null A

By Rank-Nullity Theorem, rank A+ nullity A = n

Because rows of A are linearly independent, rank A = m,so m + nullity A = n

Requiring Col B = Null A is the same as requiring

(i) Col B is a subspace of Null A

=) same as requiring AB =

2

640 · · · 0...

...0 · · · 0

3

75

(ii) dimCol B = nullity A

=) same as requiring number of columns of B = nullity A

same as requiring number of columns of B = n �m

Iinput: m ⇥ n matrix A with independent rows

Ioutput: matrix B with n �m independent columns such that AB =

h0

i







(i) Col B is a subspace of Null A =) same as requiring AB =

2

640 · · · 0...

...0 · · · 0

3

75

(ii) dimCol B = nullity A

=) same as requiring number of columns of B = nullity A




h0

i







(i) Col B is a subspace of Null A =) same as requiring AB =

2

640 · · · 0...

...0 · · · 0

3

75

(ii) dimCol B = nullity A =) same as requiring number of columns of B = nullity A




h0

i

Hypothesize a procedure for reformulation of Problem 1

Problem 1:



h0

i

Define procedure null space basis(M) with this spec:

Iinput: r ⇥ n matrix M with independent rows

Ioutput: matrix C with n � r independent columns such that MC =

h0

i


Iinput: independent vectors b1, . . . ,bk ,

Ioutput: homogeneous linear system a1 · x = 0, . . . ,am · x = 0 whose solution setequals Span {b1, . . . ,bk}


Iinput: n ⇥ k matrix B with independent columns

Ioutput: matrix A with independent rows such that Null A = Col B

As before, Rank-Nullity Theorem impliesnumber of rows of A+ nullity A = number of columns of A

As before, requiring Null A = Col B is the same as requiring

(i) AB =h0

i

(ii) number of rows of A = n � k

Iinput: n ⇥ k matrix B with independent rows

Ioutput: matrix A with n � k independent rows such that AB =

h0

i

Solving Problem 2 with the procedure for Problem 1Problem 1:



h0

i

Define procedure null space basis(M)I

input: r ⇥ n matrix M with independent rows

Ioutput: matrix C with n � r independent columns such that MC =

h0

i

Problem 2:

Iinput: n ⇥ k matrix B with independent rows

Ioutput: matrix A with n � k independent rows such that AB =

h0

i

To solve Problem 2, call null space basis(BT ).

Returns matrix A

T with independent columns such that BTA

T =h0

i

Since B

T is k ⇥ n matrix, AT has n � k columns.

Therefore AB =h0

iand A has n � k independent rows. Therefore A is solution to Problem

2.

matrix invertibility - brown university

Documents