Compiled By

Raj G. Tiwari

Vector: n×1 matrix Interpretation:

a point or line in n-dimensional space

Dot Product, Cross Product, and Magnitude defined on vectors only

c

b

a

v

x

y

v

cfbead

f

e

d

cbaABBA T

ccbbaaAAA T 2

Think of the dot product as a matrix multiplication

The magnitude is the dot product of a vector with itself

The cross-product can be computed as a specially constructed determinant

A

B

A×B

xyyx

zxxz

yzzy

zyx

zyx

baba

baba

baba

bbb

aaa

kji

BA

ˆˆˆ

A matrix is a set of elements, organized into rows and columns

dc

barows

columns

Transpose: Swap rows with columns

ihg

fed

cba

M

ifc

heb

gda

M T

z

y

x

V zyxV T

Addition, Subtraction, Multiplication

hdgc

fbea

hg

fe

dc

ba

hdgc

fbea

hg

fe

dc

ba

dhcfdgce

bhafbgae

hg

fe

dc

ba

Just add elements

Just subtract elements

Multiply each row by each

column

Is AB = BA? Maybe, but maybe not!

Heads up: multiplication is NOT commutative!

Exceptions AB=BA iff

B = a scalar,B = identity matrix I, orB = the inverse of A, i.e., A-1

......

...bgae

hg

fe

dc

ba

......

...fcea

dc

ba

hg

fe

Multiplication of matrices require conformability condition

The conformability condition for multiplication is that the column dimensions of the lead matrix A must be equal to the row dimension of the lag matrix B.

An m×n can be multiplied by an n×p matrix to yield an m×p result

9

A symmetric matrix is a square matrix that is equal to its transpose

A=At

The entries of a symmetric matrix are symmetric with respect to the main diagonal (top left to bottom right). So if the entries are written as A = (aij), then

aij=aji

For Example

A skew-symmetric (or antisymmetric or antimetric) matrix is a square matrix A whose transpose is also its negative; that is, it satisfies the equation A = −AT. If the entry in the i th row and j th column is aij, i.e. A = (aij) then the symmetric condition becomes aij = −aji. For example, the following matrix is skew-symmetric:

000

000

000

.

100

010

001

10

01

etc

or Identity Matrix is a square matrix and also it is a diagonal matrix with 1 along the diagonals similar to scalar “1”

Null matrix is one in which all elements are zero

12

6.837 Linear Algebra Review

Used for inversion If det(A) = 0, then A

has no inverse Can be found using

factorials, pivots, and cofactors!

dc

baA

bcadAA )det(

If M is our d × d matrix, we define Mi|j to be the (d − 1) × (d − 1) matrix obtained by deleting the ith row and the jth column of M:

cegbdiafhcdhbfgaei

cegcdhbfgbdiafhaei

hg

edc

ig

fdb

ih

feaA

)(

)det(

ihg

fed

cba

ihg

fed

cba

ihg

fed

cbaFor a 3×3 matrix:Sum from left to rightSubtract from right to left

Note: In the general case, the determinant has n! terms

ihg

fed

cbaFor Matrix A

252

314

231

Let's expand our matrix along the first row. From the sign chart, we see that 1 is in a positive position, 3

is in a negative position and 2 is in a positive position. By putting the + or - in front of the element, it takes care of the sign adjustment when going from the minor to the cofactor.

1 ( 2 - 15 ) - 3 ( 8 - 6 ) + 2 ( 20 - 2 )= 1 ( -13 ) - 3 ( 2 ) + 2 (18)= -13 - 6 + 36= 17

Det(A)=

The term Mij is known as the ”minor matrix” and is the matrix you get if you eliminate row i and column j from matrix A.

Minor matrix calculation Minor matrix

Cofactor matrix

Adjoint matrix

Adjoint can be found by transposing the matrix of cofactors

6.837 Linear Algebra Review

Identity matrix: AI = A

Some matrices have an inverse, such that:AA-1 = I

Inversion is tricky:(ABC)-1 = C-1B-1A-1

Derived from non-commutativity property

100

010

001

I

Let A be a non-singular matrix. If there exists a square matrix B such that AB = I (identity matrix) then B is called inverse of matrix A and is denoted as A-1. i.e AA-1 = I Example:Matrix A Matrix B =Identity(I)1 3 1 x 2 9 -5 = 1 0 01 1 2 0 -2 1 0 1 02 3 4 -1 -3 2 0 0 1

It is not possible to divide one matrix by another. That is, we can not write A/B. This is because for two matrices A and B, the quotient can be written as AB-1

21

The matrix must be square (same number of rows and columns).

The determinant of the matrix must not be zero (determinants are covered in section 6.4). This is instead of the real number not being zero to have an inverse, the determinant must not be zero to have an inverse.

A square matrix that has an inverse is called invertible or non-singular. A matrix that does not have an inverse is called singular.

A matrix does not have to have an inverse, but if it does, the inverse is unique.

The trace of a d × d (square) matrix, denoted tr[M], is the sum of its diagonal elements:

Let A be a square matrix. A non-zero vector X is called an eigenvector of A if and only if there exists a number (real or complex)  such that

AX= λ X

If such a number  exists, it is called an eigenvalue of A. The vector C is called eigenvector associated to the eigenvalue . 

Remark. The eigenvector C must be non-zero since we have 

for any number  . 

Rewriting (A- λI)X=0

In linear algebra, the characteristic equation (or secular equation) of a square matrix A is the equation in one variable λ

where det is the determinant and I is the identity matrix. The solutions of the characteristic equation are precisely the eigenvalues of the matrix A

Set

corresponding to an eigenvalue λ, we simply solve the system of linear equations given by

(A- λI)X=0

Applying characteristic equation 

If we develop this determinant using the third column, we obtain

Using easy algebraic manipulations, we get  which implies that the eigenvalues

of A are 0, -4, and 3. 

Rewritten as

By Solving

where c is an arbitrary number

Let x and y be vectors of orders n and m respectively:

where each component yi may be a function of all the xj , a fact represented by saying that y is a function of x, or

y = y(x).

Derivative of a Scalar with Respect to Vector

Derivative of Vector with Respect to Scalar

If we have an m-dimensional vector-valued function of a n-dimensional vector x, we calculate the derivatives and represent them as the Jacobian Jacobian matrix

This matrix is also denoted by   and  .

The Jacobian determinant (often simply called the Jacobian) is the determinant of the Jacobian matrix.

3

1

230

012

3

2

2

2

1

23

1

2

1

1

1

x

x

x

y

x

y

x

yx

y

x

y

x

y

x

y