sundermeyer mar 550 spring 2013 1 laboratory in oceanography: data and methods mar550, spring 2013...

25
Sundermeye r MAR 550 Spring 2013 1 Laboratory in Oceanography: Data and Methods MAR550, Spring 2013 Miles A. Sundermeyer Linear Algebra & Calculus Review

Upload: caren-hodges

Post on 26-Dec-2015

215 views

Category:

Documents


2 download

TRANSCRIPT

SundermeyerMAR 550

Spring 2013 1

Laboratory in Oceanography Data and Methods

MAR550 Spring 2013

Miles A Sundermeyer

Linear Algebra amp Calculus Review

SundermeyerMAR 550

Spring 2013 2

Nomenclature

scalar A scalar is a variable that only has magnitude eg a speed of 40 kmh 10 a (42 + 7) log10(a)

vector A geometric entity with both length and direction a quantity comprising both magnitude and direction eg a velocity of 40 kmh north velocity u position x = (x y z)

array An indexed set or group of elements also can be used to represent vectors eg

row vectorarray column vectorarray

21 13 8 5 3 2 1 1

21

13

8

5

3

2

1

1

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 3

matrix A rectangular table of elements (or entries) which may be numbers or more generally any abstract quantities that can be added and multiplied effectively a generalized array or vector - a collection of numbers ordered by rows and columns

[2 x 3] matrix

[m x n] matrix

302010

321

nmm

n

aa

aa

aa

aaaa

1

3313

2212

1312111

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 4

Linear Algebra and Calculus Review

Examples (special matrices)

A square matrix has as many rows as it has columns Matrix A is square but matrix B is not

071

5122

543

A

5122

543B

A symmetric matrix is a square matrix in which xij = xji for all i and j

A symmetric matrix is equal to its transpose Matrix A is symmetric matrix B is not

0101

10122

121

A

0101

21210

121

B

SundermeyerMAR 550

Spring 2013 5

Linear Algebra and Calculus Review

A diagonal matrix is a symmetric matrix where all the off diagonal elements are 0 The matrix D is diagonal

An identity matrix is a diagonal matrix with only 1rsquos on the diagonal For any square matrix A the product IA = AI = A The identity matrix is generally denoted as I

IA = A

100

010

001

I

0101

10122

121

0101

10122

121

100

010

001

700

020

004

D

SundermeyerMAR 550

Spring 2013 6

Linear Algebra and Calculus Review

Example (system of equations)

Suppose we have a series of measurements of stream discharge and stage measured at n different times

time (day) = [0 14 28 42 56 70] stage (m) = [0612 0647 0580 0629 0688 0583]discharge (m3s) = [0330 0395 0241 0338 0531 0279]

Suppose we now wish to fit a rating curve to these measurements Let x = stage y = discharge then we can write this series of measurements as

yi = mxi + b with i = 1n

This in turn can be written as y = X b or

]12[ ]2[ ]1[

1

1

1

1

3

2

1

2

2

1

nn

b

m

x

x

x

x

y

y

y

y

nn

SundermeyerMAR 550

Spring 2013 7

Linear Algebra and Calculus Review

yi = mxi + b

y = X b

]12[ ]2[ ]1[

1

1

1

1

3

2

1

2

2

1

nn

b

m

x

x

x

x

y

y

y

y

nn

SundermeyerMAR 550

Spring 2013 8

Linear Algebra and Calculus Review

VectorsAdditionSubtraction Two vectors can be addedsubtracted if and only if they are of the same dimension

nnnn ba

ba

ba

b

b

b

a

a

a

22

11

2

1

2

1

BA

8

7

1

7

71

34

23

52

7

3

2

5

1

4

3

2Example

SundermeyerMAR 550

Spring 2013 9

Linear Algebra and Calculus Review

Scalar Multiplication If k is a scalar and A is a n-dimensional vector then

nn ka

ka

ka

a

a

a

kk

2

1

2

1

A

Example

10

40

50

52

202

252

5

20

25

2

Example

A + B ndash 3C where

2

1

10

2

1

1

6

3

2

CBA

2

1

27

626

313

3012

2

1

10

3

2

1

1

6

3

2

3CBA

SundermeyerMAR 550

Spring 2013 10

Linear Algebra and Calculus Review

Dot Product Let

be two vectors of length n Then the dot product of the two vectors u and v is defined as

nuuu 21 u nvvv 21 v

n

iiinn vuvuvuvu

12211 vu

A dot product is also an inner product

Example

33)23()72()11()34(27133214 vu

Example (divergence of a vector)

z

w

y

v

x

uwvu

zyx

v

SundermeyerMAR 550

Spring 2013 11

Linear Algebra and Calculus Review

Dot Product and Scalar Product Rules

uv is a scalar

uv = vu

u0 = 0 = 0u

uu = ||u||2

(ku)v = (k)uv = u(kv) for k scalar

u(v plusmn w) = uv plusmn uw

SundermeyerMAR 550

Spring 2013 12

Linear Algebra and Calculus Review

Cross Product Let

be two vectors of length 3 Then the cross product of the two vectors u and v is defined as

)(ˆ)(ˆ)(ˆ

detˆdetˆdetˆ

ˆˆˆ

det

122113312332

21

21

31

31

32

32

321

321

vuvukvuvujvuvui

vv

uuk

vv

uuj

vv

uui

vvv

uuu

kji

vu

Example

kjikji

kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ

713

214

ˆˆˆ

det713214

vu

Example (curl of a vector)

y

u

x

vk

z

u

x

wj

z

v

y

wi

wvuzyx

kji

wvuzyx

ˆˆˆ

ˆˆˆ

detv

321 uuuu 321 vvvv

SundermeyerMAR 550

Spring 2013 13

Linear Algebra and Calculus Review

Cross Product Rules

bull u times v is a vector

bull u times v is orthogonal to both u and v

bull u times 0 = 0 = 0 x u

bull u times u = 0

bull u times v = -(v times u)

bull (ku) times v = k(u times v) = u times (kv) for any scalar k

bull u times (v + w) = (u times v) + (u times w)

bull (v + w) times u = (v times u) + (w times u)

SundermeyerMAR 550

Spring 2013 14

Linear Algebra and Calculus Review

kk

a

a

a

ka

ka

ka

ka

ka

ka

a

a

a

kk

nnnn

AA

2

1

2

1

2

1

2

1

kz

Tjy

Tix

T

kz

Tj

y

Tix

T

z

T

y

T

x

TT

zyxT

ˆˆˆ

ˆˆˆ

NOTE In general for a vector A and a scalar k kA = Ak

However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie

SundermeyerMAR 550

Spring 2013 15

Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added

A + B

nmnmmm

nn

nmm

n

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 16

Linear Algebra and Calculus Review

Example

303020201010

303202101

300020001000

300200100

302010

321

Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers

bullA + B = B + A

bullA + (B + C) = (A + B) + C

bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A

bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0

bullk(A + B) = kA + kB

bull(k+p)A = kA + pA

bull(kp)A = k(pA)

SundermeyerMAR 550

Spring 2013 17

Linear Algebra and Calculus Review

nm

m

T

nmm a

a

a

aaa

aa

aa

aa

33

22

11312

1

3313

2212

n1

13

12

11n1131211

a

a

a

aaaaa

Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar

AT (also denoted Arsquo)

[m x n] [n x m]

Example

303

202

101

302010

321T

SundermeyerMAR 550

Spring 2013 18

Linear Algebra and Calculus Review

Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix

bull(AT)T = A

bull(kA)T = kAT

bull(A + B)T = AT + BT

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 2

Nomenclature

scalar A scalar is a variable that only has magnitude eg a speed of 40 kmh 10 a (42 + 7) log10(a)

vector A geometric entity with both length and direction a quantity comprising both magnitude and direction eg a velocity of 40 kmh north velocity u position x = (x y z)

array An indexed set or group of elements also can be used to represent vectors eg

row vectorarray column vectorarray

21 13 8 5 3 2 1 1

21

13

8

5

3

2

1

1

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 3

matrix A rectangular table of elements (or entries) which may be numbers or more generally any abstract quantities that can be added and multiplied effectively a generalized array or vector - a collection of numbers ordered by rows and columns

[2 x 3] matrix

[m x n] matrix

302010

321

nmm

n

aa

aa

aa

aaaa

1

3313

2212

1312111

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 4

Linear Algebra and Calculus Review

Examples (special matrices)

A square matrix has as many rows as it has columns Matrix A is square but matrix B is not

071

5122

543

A

5122

543B

A symmetric matrix is a square matrix in which xij = xji for all i and j

A symmetric matrix is equal to its transpose Matrix A is symmetric matrix B is not

0101

10122

121

A

0101

21210

121

B

SundermeyerMAR 550

Spring 2013 5

Linear Algebra and Calculus Review

A diagonal matrix is a symmetric matrix where all the off diagonal elements are 0 The matrix D is diagonal

An identity matrix is a diagonal matrix with only 1rsquos on the diagonal For any square matrix A the product IA = AI = A The identity matrix is generally denoted as I

IA = A

100

010

001

I

0101

10122

121

0101

10122

121

100

010

001

700

020

004

D

SundermeyerMAR 550

Spring 2013 6

Linear Algebra and Calculus Review

Example (system of equations)

Suppose we have a series of measurements of stream discharge and stage measured at n different times

time (day) = [0 14 28 42 56 70] stage (m) = [0612 0647 0580 0629 0688 0583]discharge (m3s) = [0330 0395 0241 0338 0531 0279]

Suppose we now wish to fit a rating curve to these measurements Let x = stage y = discharge then we can write this series of measurements as

yi = mxi + b with i = 1n

This in turn can be written as y = X b or

]12[ ]2[ ]1[

1

1

1

1

3

2

1

2

2

1

nn

b

m

x

x

x

x

y

y

y

y

nn

SundermeyerMAR 550

Spring 2013 7

Linear Algebra and Calculus Review

yi = mxi + b

y = X b

]12[ ]2[ ]1[

1

1

1

1

3

2

1

2

2

1

nn

b

m

x

x

x

x

y

y

y

y

nn

SundermeyerMAR 550

Spring 2013 8

Linear Algebra and Calculus Review

VectorsAdditionSubtraction Two vectors can be addedsubtracted if and only if they are of the same dimension

nnnn ba

ba

ba

b

b

b

a

a

a

22

11

2

1

2

1

BA

8

7

1

7

71

34

23

52

7

3

2

5

1

4

3

2Example

SundermeyerMAR 550

Spring 2013 9

Linear Algebra and Calculus Review

Scalar Multiplication If k is a scalar and A is a n-dimensional vector then

nn ka

ka

ka

a

a

a

kk

2

1

2

1

A

Example

10

40

50

52

202

252

5

20

25

2

Example

A + B ndash 3C where

2

1

10

2

1

1

6

3

2

CBA

2

1

27

626

313

3012

2

1

10

3

2

1

1

6

3

2

3CBA

SundermeyerMAR 550

Spring 2013 10

Linear Algebra and Calculus Review

Dot Product Let

be two vectors of length n Then the dot product of the two vectors u and v is defined as

nuuu 21 u nvvv 21 v

n

iiinn vuvuvuvu

12211 vu

A dot product is also an inner product

Example

33)23()72()11()34(27133214 vu

Example (divergence of a vector)

z

w

y

v

x

uwvu

zyx

v

SundermeyerMAR 550

Spring 2013 11

Linear Algebra and Calculus Review

Dot Product and Scalar Product Rules

uv is a scalar

uv = vu

u0 = 0 = 0u

uu = ||u||2

(ku)v = (k)uv = u(kv) for k scalar

u(v plusmn w) = uv plusmn uw

SundermeyerMAR 550

Spring 2013 12

Linear Algebra and Calculus Review

Cross Product Let

be two vectors of length 3 Then the cross product of the two vectors u and v is defined as

)(ˆ)(ˆ)(ˆ

detˆdetˆdetˆ

ˆˆˆ

det

122113312332

21

21

31

31

32

32

321

321

vuvukvuvujvuvui

vv

uuk

vv

uuj

vv

uui

vvv

uuu

kji

vu

Example

kjikji

kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ

713

214

ˆˆˆ

det713214

vu

Example (curl of a vector)

y

u

x

vk

z

u

x

wj

z

v

y

wi

wvuzyx

kji

wvuzyx

ˆˆˆ

ˆˆˆ

detv

321 uuuu 321 vvvv

SundermeyerMAR 550

Spring 2013 13

Linear Algebra and Calculus Review

Cross Product Rules

bull u times v is a vector

bull u times v is orthogonal to both u and v

bull u times 0 = 0 = 0 x u

bull u times u = 0

bull u times v = -(v times u)

bull (ku) times v = k(u times v) = u times (kv) for any scalar k

bull u times (v + w) = (u times v) + (u times w)

bull (v + w) times u = (v times u) + (w times u)

SundermeyerMAR 550

Spring 2013 14

Linear Algebra and Calculus Review

kk

a

a

a

ka

ka

ka

ka

ka

ka

a

a

a

kk

nnnn

AA

2

1

2

1

2

1

2

1

kz

Tjy

Tix

T

kz

Tj

y

Tix

T

z

T

y

T

x

TT

zyxT

ˆˆˆ

ˆˆˆ

NOTE In general for a vector A and a scalar k kA = Ak

However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie

SundermeyerMAR 550

Spring 2013 15

Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added

A + B

nmnmmm

nn

nmm

n

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 16

Linear Algebra and Calculus Review

Example

303020201010

303202101

300020001000

300200100

302010

321

Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers

bullA + B = B + A

bullA + (B + C) = (A + B) + C

bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A

bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0

bullk(A + B) = kA + kB

bull(k+p)A = kA + pA

bull(kp)A = k(pA)

SundermeyerMAR 550

Spring 2013 17

Linear Algebra and Calculus Review

nm

m

T

nmm a

a

a

aaa

aa

aa

aa

33

22

11312

1

3313

2212

n1

13

12

11n1131211

a

a

a

aaaaa

Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar

AT (also denoted Arsquo)

[m x n] [n x m]

Example

303

202

101

302010

321T

SundermeyerMAR 550

Spring 2013 18

Linear Algebra and Calculus Review

Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix

bull(AT)T = A

bull(kA)T = kAT

bull(A + B)T = AT + BT

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 3

matrix A rectangular table of elements (or entries) which may be numbers or more generally any abstract quantities that can be added and multiplied effectively a generalized array or vector - a collection of numbers ordered by rows and columns

[2 x 3] matrix

[m x n] matrix

302010

321

nmm

n

aa

aa

aa

aaaa

1

3313

2212

1312111

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 4

Linear Algebra and Calculus Review

Examples (special matrices)

A square matrix has as many rows as it has columns Matrix A is square but matrix B is not

071

5122

543

A

5122

543B

A symmetric matrix is a square matrix in which xij = xji for all i and j

A symmetric matrix is equal to its transpose Matrix A is symmetric matrix B is not

0101

10122

121

A

0101

21210

121

B

SundermeyerMAR 550

Spring 2013 5

Linear Algebra and Calculus Review

A diagonal matrix is a symmetric matrix where all the off diagonal elements are 0 The matrix D is diagonal

An identity matrix is a diagonal matrix with only 1rsquos on the diagonal For any square matrix A the product IA = AI = A The identity matrix is generally denoted as I

IA = A

100

010

001

I

0101

10122

121

0101

10122

121

100

010

001

700

020

004

D

SundermeyerMAR 550

Spring 2013 6

Linear Algebra and Calculus Review

Example (system of equations)

Suppose we have a series of measurements of stream discharge and stage measured at n different times

time (day) = [0 14 28 42 56 70] stage (m) = [0612 0647 0580 0629 0688 0583]discharge (m3s) = [0330 0395 0241 0338 0531 0279]

Suppose we now wish to fit a rating curve to these measurements Let x = stage y = discharge then we can write this series of measurements as

yi = mxi + b with i = 1n

This in turn can be written as y = X b or

]12[ ]2[ ]1[

1

1

1

1

3

2

1

2

2

1

nn

b

m

x

x

x

x

y

y

y

y

nn

SundermeyerMAR 550

Spring 2013 7

Linear Algebra and Calculus Review

yi = mxi + b

y = X b

]12[ ]2[ ]1[

1

1

1

1

3

2

1

2

2

1

nn

b

m

x

x

x

x

y

y

y

y

nn

SundermeyerMAR 550

Spring 2013 8

Linear Algebra and Calculus Review

VectorsAdditionSubtraction Two vectors can be addedsubtracted if and only if they are of the same dimension

nnnn ba

ba

ba

b

b

b

a

a

a

22

11

2

1

2

1

BA

8

7

1

7

71

34

23

52

7

3

2

5

1

4

3

2Example

SundermeyerMAR 550

Spring 2013 9

Linear Algebra and Calculus Review

Scalar Multiplication If k is a scalar and A is a n-dimensional vector then

nn ka

ka

ka

a

a

a

kk

2

1

2

1

A

Example

10

40

50

52

202

252

5

20

25

2

Example

A + B ndash 3C where

2

1

10

2

1

1

6

3

2

CBA

2

1

27

626

313

3012

2

1

10

3

2

1

1

6

3

2

3CBA

SundermeyerMAR 550

Spring 2013 10

Linear Algebra and Calculus Review

Dot Product Let

be two vectors of length n Then the dot product of the two vectors u and v is defined as

nuuu 21 u nvvv 21 v

n

iiinn vuvuvuvu

12211 vu

A dot product is also an inner product

Example

33)23()72()11()34(27133214 vu

Example (divergence of a vector)

z

w

y

v

x

uwvu

zyx

v

SundermeyerMAR 550

Spring 2013 11

Linear Algebra and Calculus Review

Dot Product and Scalar Product Rules

uv is a scalar

uv = vu

u0 = 0 = 0u

uu = ||u||2

(ku)v = (k)uv = u(kv) for k scalar

u(v plusmn w) = uv plusmn uw

SundermeyerMAR 550

Spring 2013 12

Linear Algebra and Calculus Review

Cross Product Let

be two vectors of length 3 Then the cross product of the two vectors u and v is defined as

)(ˆ)(ˆ)(ˆ

detˆdetˆdetˆ

ˆˆˆ

det

122113312332

21

21

31

31

32

32

321

321

vuvukvuvujvuvui

vv

uuk

vv

uuj

vv

uui

vvv

uuu

kji

vu

Example

kjikji

kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ

713

214

ˆˆˆ

det713214

vu

Example (curl of a vector)

y

u

x

vk

z

u

x

wj

z

v

y

wi

wvuzyx

kji

wvuzyx

ˆˆˆ

ˆˆˆ

detv

321 uuuu 321 vvvv

SundermeyerMAR 550

Spring 2013 13

Linear Algebra and Calculus Review

Cross Product Rules

bull u times v is a vector

bull u times v is orthogonal to both u and v

bull u times 0 = 0 = 0 x u

bull u times u = 0

bull u times v = -(v times u)

bull (ku) times v = k(u times v) = u times (kv) for any scalar k

bull u times (v + w) = (u times v) + (u times w)

bull (v + w) times u = (v times u) + (w times u)

SundermeyerMAR 550

Spring 2013 14

Linear Algebra and Calculus Review

kk

a

a

a

ka

ka

ka

ka

ka

ka

a

a

a

kk

nnnn

AA

2

1

2

1

2

1

2

1

kz

Tjy

Tix

T

kz

Tj

y

Tix

T

z

T

y

T

x

TT

zyxT

ˆˆˆ

ˆˆˆ

NOTE In general for a vector A and a scalar k kA = Ak

However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie

SundermeyerMAR 550

Spring 2013 15

Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added

A + B

nmnmmm

nn

nmm

n

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 16

Linear Algebra and Calculus Review

Example

303020201010

303202101

300020001000

300200100

302010

321

Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers

bullA + B = B + A

bullA + (B + C) = (A + B) + C

bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A

bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0

bullk(A + B) = kA + kB

bull(k+p)A = kA + pA

bull(kp)A = k(pA)

SundermeyerMAR 550

Spring 2013 17

Linear Algebra and Calculus Review

nm

m

T

nmm a

a

a

aaa

aa

aa

aa

33

22

11312

1

3313

2212

n1

13

12

11n1131211

a

a

a

aaaaa

Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar

AT (also denoted Arsquo)

[m x n] [n x m]

Example

303

202

101

302010

321T

SundermeyerMAR 550

Spring 2013 18

Linear Algebra and Calculus Review

Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix

bull(AT)T = A

bull(kA)T = kAT

bull(A + B)T = AT + BT

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 4

Linear Algebra and Calculus Review

Examples (special matrices)

A square matrix has as many rows as it has columns Matrix A is square but matrix B is not

071

5122

543

A

5122

543B

A symmetric matrix is a square matrix in which xij = xji for all i and j

A symmetric matrix is equal to its transpose Matrix A is symmetric matrix B is not

0101

10122

121

A

0101

21210

121

B

SundermeyerMAR 550

Spring 2013 5

Linear Algebra and Calculus Review

A diagonal matrix is a symmetric matrix where all the off diagonal elements are 0 The matrix D is diagonal

An identity matrix is a diagonal matrix with only 1rsquos on the diagonal For any square matrix A the product IA = AI = A The identity matrix is generally denoted as I

IA = A

100

010

001

I

0101

10122

121

0101

10122

121

100

010

001

700

020

004

D

SundermeyerMAR 550

Spring 2013 6

Linear Algebra and Calculus Review

Example (system of equations)

Suppose we have a series of measurements of stream discharge and stage measured at n different times

time (day) = [0 14 28 42 56 70] stage (m) = [0612 0647 0580 0629 0688 0583]discharge (m3s) = [0330 0395 0241 0338 0531 0279]

Suppose we now wish to fit a rating curve to these measurements Let x = stage y = discharge then we can write this series of measurements as

yi = mxi + b with i = 1n

This in turn can be written as y = X b or

]12[ ]2[ ]1[

1

1

1

1

3

2

1

2

2

1

nn

b

m

x

x

x

x

y

y

y

y

nn

SundermeyerMAR 550

Spring 2013 7

Linear Algebra and Calculus Review

yi = mxi + b

y = X b

]12[ ]2[ ]1[

1

1

1

1

3

2

1

2

2

1

nn

b

m

x

x

x

x

y

y

y

y

nn

SundermeyerMAR 550

Spring 2013 8

Linear Algebra and Calculus Review

VectorsAdditionSubtraction Two vectors can be addedsubtracted if and only if they are of the same dimension

nnnn ba

ba

ba

b

b

b

a

a

a

22

11

2

1

2

1

BA

8

7

1

7

71

34

23

52

7

3

2

5

1

4

3

2Example

SundermeyerMAR 550

Spring 2013 9

Linear Algebra and Calculus Review

Scalar Multiplication If k is a scalar and A is a n-dimensional vector then

nn ka

ka

ka

a

a

a

kk

2

1

2

1

A

Example

10

40

50

52

202

252

5

20

25

2

Example

A + B ndash 3C where

2

1

10

2

1

1

6

3

2

CBA

2

1

27

626

313

3012

2

1

10

3

2

1

1

6

3

2

3CBA

SundermeyerMAR 550

Spring 2013 10

Linear Algebra and Calculus Review

Dot Product Let

be two vectors of length n Then the dot product of the two vectors u and v is defined as

nuuu 21 u nvvv 21 v

n

iiinn vuvuvuvu

12211 vu

A dot product is also an inner product

Example

33)23()72()11()34(27133214 vu

Example (divergence of a vector)

z

w

y

v

x

uwvu

zyx

v

SundermeyerMAR 550

Spring 2013 11

Linear Algebra and Calculus Review

Dot Product and Scalar Product Rules

uv is a scalar

uv = vu

u0 = 0 = 0u

uu = ||u||2

(ku)v = (k)uv = u(kv) for k scalar

u(v plusmn w) = uv plusmn uw

SundermeyerMAR 550

Spring 2013 12

Linear Algebra and Calculus Review

Cross Product Let

be two vectors of length 3 Then the cross product of the two vectors u and v is defined as

)(ˆ)(ˆ)(ˆ

detˆdetˆdetˆ

ˆˆˆ

det

122113312332

21

21

31

31

32

32

321

321

vuvukvuvujvuvui

vv

uuk

vv

uuj

vv

uui

vvv

uuu

kji

vu

Example

kjikji

kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ

713

214

ˆˆˆ

det713214

vu

Example (curl of a vector)

y

u

x

vk

z

u

x

wj

z

v

y

wi

wvuzyx

kji

wvuzyx

ˆˆˆ

ˆˆˆ

detv

321 uuuu 321 vvvv

SundermeyerMAR 550

Spring 2013 13

Linear Algebra and Calculus Review

Cross Product Rules

bull u times v is a vector

bull u times v is orthogonal to both u and v

bull u times 0 = 0 = 0 x u

bull u times u = 0

bull u times v = -(v times u)

bull (ku) times v = k(u times v) = u times (kv) for any scalar k

bull u times (v + w) = (u times v) + (u times w)

bull (v + w) times u = (v times u) + (w times u)

SundermeyerMAR 550

Spring 2013 14

Linear Algebra and Calculus Review

kk

a

a

a

ka

ka

ka

ka

ka

ka

a

a

a

kk

nnnn

AA

2

1

2

1

2

1

2

1

kz

Tjy

Tix

T

kz

Tj

y

Tix

T

z

T

y

T

x

TT

zyxT

ˆˆˆ

ˆˆˆ

NOTE In general for a vector A and a scalar k kA = Ak

However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie

SundermeyerMAR 550

Spring 2013 15

Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added

A + B

nmnmmm

nn

nmm

n

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 16

Linear Algebra and Calculus Review

Example

303020201010

303202101

300020001000

300200100

302010

321

Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers

bullA + B = B + A

bullA + (B + C) = (A + B) + C

bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A

bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0

bullk(A + B) = kA + kB

bull(k+p)A = kA + pA

bull(kp)A = k(pA)

SundermeyerMAR 550

Spring 2013 17

Linear Algebra and Calculus Review

nm

m

T

nmm a

a

a

aaa

aa

aa

aa

33

22

11312

1

3313

2212

n1

13

12

11n1131211

a

a

a

aaaaa

Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar

AT (also denoted Arsquo)

[m x n] [n x m]

Example

303

202

101

302010

321T

SundermeyerMAR 550

Spring 2013 18

Linear Algebra and Calculus Review

Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix

bull(AT)T = A

bull(kA)T = kAT

bull(A + B)T = AT + BT

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 5

Linear Algebra and Calculus Review

A diagonal matrix is a symmetric matrix where all the off diagonal elements are 0 The matrix D is diagonal

An identity matrix is a diagonal matrix with only 1rsquos on the diagonal For any square matrix A the product IA = AI = A The identity matrix is generally denoted as I

IA = A

100

010

001

I

0101

10122

121

0101

10122

121

100

010

001

700

020

004

D

SundermeyerMAR 550

Spring 2013 6

Linear Algebra and Calculus Review

Example (system of equations)

Suppose we have a series of measurements of stream discharge and stage measured at n different times

time (day) = [0 14 28 42 56 70] stage (m) = [0612 0647 0580 0629 0688 0583]discharge (m3s) = [0330 0395 0241 0338 0531 0279]

Suppose we now wish to fit a rating curve to these measurements Let x = stage y = discharge then we can write this series of measurements as

yi = mxi + b with i = 1n

This in turn can be written as y = X b or

]12[ ]2[ ]1[

1

1

1

1

3

2

1

2

2

1

nn

b

m

x

x

x

x

y

y

y

y

nn

SundermeyerMAR 550

Spring 2013 7

Linear Algebra and Calculus Review

yi = mxi + b

y = X b

]12[ ]2[ ]1[

1

1

1

1

3

2

1

2

2

1

nn

b

m

x

x

x

x

y

y

y

y

nn

SundermeyerMAR 550

Spring 2013 8

Linear Algebra and Calculus Review

VectorsAdditionSubtraction Two vectors can be addedsubtracted if and only if they are of the same dimension

nnnn ba

ba

ba

b

b

b

a

a

a

22

11

2

1

2

1

BA

8

7

1

7

71

34

23

52

7

3

2

5

1

4

3

2Example

SundermeyerMAR 550

Spring 2013 9

Linear Algebra and Calculus Review

Scalar Multiplication If k is a scalar and A is a n-dimensional vector then

nn ka

ka

ka

a

a

a

kk

2

1

2

1

A

Example

10

40

50

52

202

252

5

20

25

2

Example

A + B ndash 3C where

2

1

10

2

1

1

6

3

2

CBA

2

1

27

626

313

3012

2

1

10

3

2

1

1

6

3

2

3CBA

SundermeyerMAR 550

Spring 2013 10

Linear Algebra and Calculus Review

Dot Product Let

be two vectors of length n Then the dot product of the two vectors u and v is defined as

nuuu 21 u nvvv 21 v

n

iiinn vuvuvuvu

12211 vu

A dot product is also an inner product

Example

33)23()72()11()34(27133214 vu

Example (divergence of a vector)

z

w

y

v

x

uwvu

zyx

v

SundermeyerMAR 550

Spring 2013 11

Linear Algebra and Calculus Review

Dot Product and Scalar Product Rules

uv is a scalar

uv = vu

u0 = 0 = 0u

uu = ||u||2

(ku)v = (k)uv = u(kv) for k scalar

u(v plusmn w) = uv plusmn uw

SundermeyerMAR 550

Spring 2013 12

Linear Algebra and Calculus Review

Cross Product Let

be two vectors of length 3 Then the cross product of the two vectors u and v is defined as

)(ˆ)(ˆ)(ˆ

detˆdetˆdetˆ

ˆˆˆ

det

122113312332

21

21

31

31

32

32

321

321

vuvukvuvujvuvui

vv

uuk

vv

uuj

vv

uui

vvv

uuu

kji

vu

Example

kjikji

kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ

713

214

ˆˆˆ

det713214

vu

Example (curl of a vector)

y

u

x

vk

z

u

x

wj

z

v

y

wi

wvuzyx

kji

wvuzyx

ˆˆˆ

ˆˆˆ

detv

321 uuuu 321 vvvv

SundermeyerMAR 550

Spring 2013 13

Linear Algebra and Calculus Review

Cross Product Rules

bull u times v is a vector

bull u times v is orthogonal to both u and v

bull u times 0 = 0 = 0 x u

bull u times u = 0

bull u times v = -(v times u)

bull (ku) times v = k(u times v) = u times (kv) for any scalar k

bull u times (v + w) = (u times v) + (u times w)

bull (v + w) times u = (v times u) + (w times u)

SundermeyerMAR 550

Spring 2013 14

Linear Algebra and Calculus Review

kk

a

a

a

ka

ka

ka

ka

ka

ka

a

a

a

kk

nnnn

AA

2

1

2

1

2

1

2

1

kz

Tjy

Tix

T

kz

Tj

y

Tix

T

z

T

y

T

x

TT

zyxT

ˆˆˆ

ˆˆˆ

NOTE In general for a vector A and a scalar k kA = Ak

However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie

SundermeyerMAR 550

Spring 2013 15

Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added

A + B

nmnmmm

nn

nmm

n

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 16

Linear Algebra and Calculus Review

Example

303020201010

303202101

300020001000

300200100

302010

321

Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers

bullA + B = B + A

bullA + (B + C) = (A + B) + C

bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A

bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0

bullk(A + B) = kA + kB

bull(k+p)A = kA + pA

bull(kp)A = k(pA)

SundermeyerMAR 550

Spring 2013 17

Linear Algebra and Calculus Review

nm

m

T

nmm a

a

a

aaa

aa

aa

aa

33

22

11312

1

3313

2212

n1

13

12

11n1131211

a

a

a

aaaaa

Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar

AT (also denoted Arsquo)

[m x n] [n x m]

Example

303

202

101

302010

321T

SundermeyerMAR 550

Spring 2013 18

Linear Algebra and Calculus Review

Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix

bull(AT)T = A

bull(kA)T = kAT

bull(A + B)T = AT + BT

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 6

Linear Algebra and Calculus Review

Example (system of equations)

Suppose we have a series of measurements of stream discharge and stage measured at n different times

time (day) = [0 14 28 42 56 70] stage (m) = [0612 0647 0580 0629 0688 0583]discharge (m3s) = [0330 0395 0241 0338 0531 0279]

Suppose we now wish to fit a rating curve to these measurements Let x = stage y = discharge then we can write this series of measurements as

yi = mxi + b with i = 1n

This in turn can be written as y = X b or

]12[ ]2[ ]1[

1

1

1

1

3

2

1

2

2

1

nn

b

m

x

x

x

x

y

y

y

y

nn

SundermeyerMAR 550

Spring 2013 7

Linear Algebra and Calculus Review

yi = mxi + b

y = X b

]12[ ]2[ ]1[

1

1

1

1

3

2

1

2

2

1

nn

b

m

x

x

x

x

y

y

y

y

nn

SundermeyerMAR 550

Spring 2013 8

Linear Algebra and Calculus Review

VectorsAdditionSubtraction Two vectors can be addedsubtracted if and only if they are of the same dimension

nnnn ba

ba

ba

b

b

b

a

a

a

22

11

2

1

2

1

BA

8

7

1

7

71

34

23

52

7

3

2

5

1

4

3

2Example

SundermeyerMAR 550

Spring 2013 9

Linear Algebra and Calculus Review

Scalar Multiplication If k is a scalar and A is a n-dimensional vector then

nn ka

ka

ka

a

a

a

kk

2

1

2

1

A

Example

10

40

50

52

202

252

5

20

25

2

Example

A + B ndash 3C where

2

1

10

2

1

1

6

3

2

CBA

2

1

27

626

313

3012

2

1

10

3

2

1

1

6

3

2

3CBA

SundermeyerMAR 550

Spring 2013 10

Linear Algebra and Calculus Review

Dot Product Let

be two vectors of length n Then the dot product of the two vectors u and v is defined as

nuuu 21 u nvvv 21 v

n

iiinn vuvuvuvu

12211 vu

A dot product is also an inner product

Example

33)23()72()11()34(27133214 vu

Example (divergence of a vector)

z

w

y

v

x

uwvu

zyx

v

SundermeyerMAR 550

Spring 2013 11

Linear Algebra and Calculus Review

Dot Product and Scalar Product Rules

uv is a scalar

uv = vu

u0 = 0 = 0u

uu = ||u||2

(ku)v = (k)uv = u(kv) for k scalar

u(v plusmn w) = uv plusmn uw

SundermeyerMAR 550

Spring 2013 12

Linear Algebra and Calculus Review

Cross Product Let

be two vectors of length 3 Then the cross product of the two vectors u and v is defined as

)(ˆ)(ˆ)(ˆ

detˆdetˆdetˆ

ˆˆˆ

det

122113312332

21

21

31

31

32

32

321

321

vuvukvuvujvuvui

vv

uuk

vv

uuj

vv

uui

vvv

uuu

kji

vu

Example

kjikji

kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ

713

214

ˆˆˆ

det713214

vu

Example (curl of a vector)

y

u

x

vk

z

u

x

wj

z

v

y

wi

wvuzyx

kji

wvuzyx

ˆˆˆ

ˆˆˆ

detv

321 uuuu 321 vvvv

SundermeyerMAR 550

Spring 2013 13

Linear Algebra and Calculus Review

Cross Product Rules

bull u times v is a vector

bull u times v is orthogonal to both u and v

bull u times 0 = 0 = 0 x u

bull u times u = 0

bull u times v = -(v times u)

bull (ku) times v = k(u times v) = u times (kv) for any scalar k

bull u times (v + w) = (u times v) + (u times w)

bull (v + w) times u = (v times u) + (w times u)

SundermeyerMAR 550

Spring 2013 14

Linear Algebra and Calculus Review

kk

a

a

a

ka

ka

ka

ka

ka

ka

a

a

a

kk

nnnn

AA

2

1

2

1

2

1

2

1

kz

Tjy

Tix

T

kz

Tj

y

Tix

T

z

T

y

T

x

TT

zyxT

ˆˆˆ

ˆˆˆ

NOTE In general for a vector A and a scalar k kA = Ak

However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie

SundermeyerMAR 550

Spring 2013 15

Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added

A + B

nmnmmm

nn

nmm

n

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 16

Linear Algebra and Calculus Review

Example

303020201010

303202101

300020001000

300200100

302010

321

Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers

bullA + B = B + A

bullA + (B + C) = (A + B) + C

bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A

bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0

bullk(A + B) = kA + kB

bull(k+p)A = kA + pA

bull(kp)A = k(pA)

SundermeyerMAR 550

Spring 2013 17

Linear Algebra and Calculus Review

nm

m

T

nmm a

a

a

aaa

aa

aa

aa

33

22

11312

1

3313

2212

n1

13

12

11n1131211

a

a

a

aaaaa

Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar

AT (also denoted Arsquo)

[m x n] [n x m]

Example

303

202

101

302010

321T

SundermeyerMAR 550

Spring 2013 18

Linear Algebra and Calculus Review

Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix

bull(AT)T = A

bull(kA)T = kAT

bull(A + B)T = AT + BT

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 7

Linear Algebra and Calculus Review

yi = mxi + b

y = X b

]12[ ]2[ ]1[

1

1

1

1

3

2

1

2

2

1

nn

b

m

x

x

x

x

y

y

y

y

nn

SundermeyerMAR 550

Spring 2013 8

Linear Algebra and Calculus Review

VectorsAdditionSubtraction Two vectors can be addedsubtracted if and only if they are of the same dimension

nnnn ba

ba

ba

b

b

b

a

a

a

22

11

2

1

2

1

BA

8

7

1

7

71

34

23

52

7

3

2

5

1

4

3

2Example

SundermeyerMAR 550

Spring 2013 9

Linear Algebra and Calculus Review

Scalar Multiplication If k is a scalar and A is a n-dimensional vector then

nn ka

ka

ka

a

a

a

kk

2

1

2

1

A

Example

10

40

50

52

202

252

5

20

25

2

Example

A + B ndash 3C where

2

1

10

2

1

1

6

3

2

CBA

2

1

27

626

313

3012

2

1

10

3

2

1

1

6

3

2

3CBA

SundermeyerMAR 550

Spring 2013 10

Linear Algebra and Calculus Review

Dot Product Let

be two vectors of length n Then the dot product of the two vectors u and v is defined as

nuuu 21 u nvvv 21 v

n

iiinn vuvuvuvu

12211 vu

A dot product is also an inner product

Example

33)23()72()11()34(27133214 vu

Example (divergence of a vector)

z

w

y

v

x

uwvu

zyx

v

SundermeyerMAR 550

Spring 2013 11

Linear Algebra and Calculus Review

Dot Product and Scalar Product Rules

uv is a scalar

uv = vu

u0 = 0 = 0u

uu = ||u||2

(ku)v = (k)uv = u(kv) for k scalar

u(v plusmn w) = uv plusmn uw

SundermeyerMAR 550

Spring 2013 12

Linear Algebra and Calculus Review

Cross Product Let

be two vectors of length 3 Then the cross product of the two vectors u and v is defined as

)(ˆ)(ˆ)(ˆ

detˆdetˆdetˆ

ˆˆˆ

det

122113312332

21

21

31

31

32

32

321

321

vuvukvuvujvuvui

vv

uuk

vv

uuj

vv

uui

vvv

uuu

kji

vu

Example

kjikji

kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ

713

214

ˆˆˆ

det713214

vu

Example (curl of a vector)

y

u

x

vk

z

u

x

wj

z

v

y

wi

wvuzyx

kji

wvuzyx

ˆˆˆ

ˆˆˆ

detv

321 uuuu 321 vvvv

SundermeyerMAR 550

Spring 2013 13

Linear Algebra and Calculus Review

Cross Product Rules

bull u times v is a vector

bull u times v is orthogonal to both u and v

bull u times 0 = 0 = 0 x u

bull u times u = 0

bull u times v = -(v times u)

bull (ku) times v = k(u times v) = u times (kv) for any scalar k

bull u times (v + w) = (u times v) + (u times w)

bull (v + w) times u = (v times u) + (w times u)

SundermeyerMAR 550

Spring 2013 14

Linear Algebra and Calculus Review

kk

a

a

a

ka

ka

ka

ka

ka

ka

a

a

a

kk

nnnn

AA

2

1

2

1

2

1

2

1

kz

Tjy

Tix

T

kz

Tj

y

Tix

T

z

T

y

T

x

TT

zyxT

ˆˆˆ

ˆˆˆ

NOTE In general for a vector A and a scalar k kA = Ak

However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie

SundermeyerMAR 550

Spring 2013 15

Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added

A + B

nmnmmm

nn

nmm

n

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 16

Linear Algebra and Calculus Review

Example

303020201010

303202101

300020001000

300200100

302010

321

Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers

bullA + B = B + A

bullA + (B + C) = (A + B) + C

bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A

bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0

bullk(A + B) = kA + kB

bull(k+p)A = kA + pA

bull(kp)A = k(pA)

SundermeyerMAR 550

Spring 2013 17

Linear Algebra and Calculus Review

nm

m

T

nmm a

a

a

aaa

aa

aa

aa

33

22

11312

1

3313

2212

n1

13

12

11n1131211

a

a

a

aaaaa

Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar

AT (also denoted Arsquo)

[m x n] [n x m]

Example

303

202

101

302010

321T

SundermeyerMAR 550

Spring 2013 18

Linear Algebra and Calculus Review

Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix

bull(AT)T = A

bull(kA)T = kAT

bull(A + B)T = AT + BT

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 8

Linear Algebra and Calculus Review

VectorsAdditionSubtraction Two vectors can be addedsubtracted if and only if they are of the same dimension

nnnn ba

ba

ba

b

b

b

a

a

a

22

11

2

1

2

1

BA

8

7

1

7

71

34

23

52

7

3

2

5

1

4

3

2Example

SundermeyerMAR 550

Spring 2013 9

Linear Algebra and Calculus Review

Scalar Multiplication If k is a scalar and A is a n-dimensional vector then

nn ka

ka

ka

a

a

a

kk

2

1

2

1

A

Example

10

40

50

52

202

252

5

20

25

2

Example

A + B ndash 3C where

2

1

10

2

1

1

6

3

2

CBA

2

1

27

626

313

3012

2

1

10

3

2

1

1

6

3

2

3CBA

SundermeyerMAR 550

Spring 2013 10

Linear Algebra and Calculus Review

Dot Product Let

be two vectors of length n Then the dot product of the two vectors u and v is defined as

nuuu 21 u nvvv 21 v

n

iiinn vuvuvuvu

12211 vu

A dot product is also an inner product

Example

33)23()72()11()34(27133214 vu

Example (divergence of a vector)

z

w

y

v

x

uwvu

zyx

v

SundermeyerMAR 550

Spring 2013 11

Linear Algebra and Calculus Review

Dot Product and Scalar Product Rules

uv is a scalar

uv = vu

u0 = 0 = 0u

uu = ||u||2

(ku)v = (k)uv = u(kv) for k scalar

u(v plusmn w) = uv plusmn uw

SundermeyerMAR 550

Spring 2013 12

Linear Algebra and Calculus Review

Cross Product Let

be two vectors of length 3 Then the cross product of the two vectors u and v is defined as

)(ˆ)(ˆ)(ˆ

detˆdetˆdetˆ

ˆˆˆ

det

122113312332

21

21

31

31

32

32

321

321

vuvukvuvujvuvui

vv

uuk

vv

uuj

vv

uui

vvv

uuu

kji

vu

Example

kjikji

kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ

713

214

ˆˆˆ

det713214

vu

Example (curl of a vector)

y

u

x

vk

z

u

x

wj

z

v

y

wi

wvuzyx

kji

wvuzyx

ˆˆˆ

ˆˆˆ

detv

321 uuuu 321 vvvv

SundermeyerMAR 550

Spring 2013 13

Linear Algebra and Calculus Review

Cross Product Rules

bull u times v is a vector

bull u times v is orthogonal to both u and v

bull u times 0 = 0 = 0 x u

bull u times u = 0

bull u times v = -(v times u)

bull (ku) times v = k(u times v) = u times (kv) for any scalar k

bull u times (v + w) = (u times v) + (u times w)

bull (v + w) times u = (v times u) + (w times u)

SundermeyerMAR 550

Spring 2013 14

Linear Algebra and Calculus Review

kk

a

a

a

ka

ka

ka

ka

ka

ka

a

a

a

kk

nnnn

AA

2

1

2

1

2

1

2

1

kz

Tjy

Tix

T

kz

Tj

y

Tix

T

z

T

y

T

x

TT

zyxT

ˆˆˆ

ˆˆˆ

NOTE In general for a vector A and a scalar k kA = Ak

However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie

SundermeyerMAR 550

Spring 2013 15

Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added

A + B

nmnmmm

nn

nmm

n

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 16

Linear Algebra and Calculus Review

Example

303020201010

303202101

300020001000

300200100

302010

321

Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers

bullA + B = B + A

bullA + (B + C) = (A + B) + C

bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A

bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0

bullk(A + B) = kA + kB

bull(k+p)A = kA + pA

bull(kp)A = k(pA)

SundermeyerMAR 550

Spring 2013 17

Linear Algebra and Calculus Review

nm

m

T

nmm a

a

a

aaa

aa

aa

aa

33

22

11312

1

3313

2212

n1

13

12

11n1131211

a

a

a

aaaaa

Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar

AT (also denoted Arsquo)

[m x n] [n x m]

Example

303

202

101

302010

321T

SundermeyerMAR 550

Spring 2013 18

Linear Algebra and Calculus Review

Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix

bull(AT)T = A

bull(kA)T = kAT

bull(A + B)T = AT + BT

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 9

Linear Algebra and Calculus Review

Scalar Multiplication If k is a scalar and A is a n-dimensional vector then

nn ka

ka

ka

a

a

a

kk

2

1

2

1

A

Example

10

40

50

52

202

252

5

20

25

2

Example

A + B ndash 3C where

2

1

10

2

1

1

6

3

2

CBA

2

1

27

626

313

3012

2

1

10

3

2

1

1

6

3

2

3CBA

SundermeyerMAR 550

Spring 2013 10

Linear Algebra and Calculus Review

Dot Product Let

be two vectors of length n Then the dot product of the two vectors u and v is defined as

nuuu 21 u nvvv 21 v

n

iiinn vuvuvuvu

12211 vu

A dot product is also an inner product

Example

33)23()72()11()34(27133214 vu

Example (divergence of a vector)

z

w

y

v

x

uwvu

zyx

v

SundermeyerMAR 550

Spring 2013 11

Linear Algebra and Calculus Review

Dot Product and Scalar Product Rules

uv is a scalar

uv = vu

u0 = 0 = 0u

uu = ||u||2

(ku)v = (k)uv = u(kv) for k scalar

u(v plusmn w) = uv plusmn uw

SundermeyerMAR 550

Spring 2013 12

Linear Algebra and Calculus Review

Cross Product Let

be two vectors of length 3 Then the cross product of the two vectors u and v is defined as

)(ˆ)(ˆ)(ˆ

detˆdetˆdetˆ

ˆˆˆ

det

122113312332

21

21

31

31

32

32

321

321

vuvukvuvujvuvui

vv

uuk

vv

uuj

vv

uui

vvv

uuu

kji

vu

Example

kjikji

kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ

713

214

ˆˆˆ

det713214

vu

Example (curl of a vector)

y

u

x

vk

z

u

x

wj

z

v

y

wi

wvuzyx

kji

wvuzyx

ˆˆˆ

ˆˆˆ

detv

321 uuuu 321 vvvv

SundermeyerMAR 550

Spring 2013 13

Linear Algebra and Calculus Review

Cross Product Rules

bull u times v is a vector

bull u times v is orthogonal to both u and v

bull u times 0 = 0 = 0 x u

bull u times u = 0

bull u times v = -(v times u)

bull (ku) times v = k(u times v) = u times (kv) for any scalar k

bull u times (v + w) = (u times v) + (u times w)

bull (v + w) times u = (v times u) + (w times u)

SundermeyerMAR 550

Spring 2013 14

Linear Algebra and Calculus Review

kk

a

a

a

ka

ka

ka

ka

ka

ka

a

a

a

kk

nnnn

AA

2

1

2

1

2

1

2

1

kz

Tjy

Tix

T

kz

Tj

y

Tix

T

z

T

y

T

x

TT

zyxT

ˆˆˆ

ˆˆˆ

NOTE In general for a vector A and a scalar k kA = Ak

However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie

SundermeyerMAR 550

Spring 2013 15

Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added

A + B

nmnmmm

nn

nmm

n

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 16

Linear Algebra and Calculus Review

Example

303020201010

303202101

300020001000

300200100

302010

321

Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers

bullA + B = B + A

bullA + (B + C) = (A + B) + C

bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A

bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0

bullk(A + B) = kA + kB

bull(k+p)A = kA + pA

bull(kp)A = k(pA)

SundermeyerMAR 550

Spring 2013 17

Linear Algebra and Calculus Review

nm

m

T

nmm a

a

a

aaa

aa

aa

aa

33

22

11312

1

3313

2212

n1

13

12

11n1131211

a

a

a

aaaaa

Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar

AT (also denoted Arsquo)

[m x n] [n x m]

Example

303

202

101

302010

321T

SundermeyerMAR 550

Spring 2013 18

Linear Algebra and Calculus Review

Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix

bull(AT)T = A

bull(kA)T = kAT

bull(A + B)T = AT + BT

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 10

Linear Algebra and Calculus Review

Dot Product Let

be two vectors of length n Then the dot product of the two vectors u and v is defined as

nuuu 21 u nvvv 21 v

n

iiinn vuvuvuvu

12211 vu

A dot product is also an inner product

Example

33)23()72()11()34(27133214 vu

Example (divergence of a vector)

z

w

y

v

x

uwvu

zyx

v

SundermeyerMAR 550

Spring 2013 11

Linear Algebra and Calculus Review

Dot Product and Scalar Product Rules

uv is a scalar

uv = vu

u0 = 0 = 0u

uu = ||u||2

(ku)v = (k)uv = u(kv) for k scalar

u(v plusmn w) = uv plusmn uw

SundermeyerMAR 550

Spring 2013 12

Linear Algebra and Calculus Review

Cross Product Let

be two vectors of length 3 Then the cross product of the two vectors u and v is defined as

)(ˆ)(ˆ)(ˆ

detˆdetˆdetˆ

ˆˆˆ

det

122113312332

21

21

31

31

32

32

321

321

vuvukvuvujvuvui

vv

uuk

vv

uuj

vv

uui

vvv

uuu

kji

vu

Example

kjikji

kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ

713

214

ˆˆˆ

det713214

vu

Example (curl of a vector)

y

u

x

vk

z

u

x

wj

z

v

y

wi

wvuzyx

kji

wvuzyx

ˆˆˆ

ˆˆˆ

detv

321 uuuu 321 vvvv

SundermeyerMAR 550

Spring 2013 13

Linear Algebra and Calculus Review

Cross Product Rules

bull u times v is a vector

bull u times v is orthogonal to both u and v

bull u times 0 = 0 = 0 x u

bull u times u = 0

bull u times v = -(v times u)

bull (ku) times v = k(u times v) = u times (kv) for any scalar k

bull u times (v + w) = (u times v) + (u times w)

bull (v + w) times u = (v times u) + (w times u)

SundermeyerMAR 550

Spring 2013 14

Linear Algebra and Calculus Review

kk

a

a

a

ka

ka

ka

ka

ka

ka

a

a

a

kk

nnnn

AA

2

1

2

1

2

1

2

1

kz

Tjy

Tix

T

kz

Tj

y

Tix

T

z

T

y

T

x

TT

zyxT

ˆˆˆ

ˆˆˆ

NOTE In general for a vector A and a scalar k kA = Ak

However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie

SundermeyerMAR 550

Spring 2013 15

Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added

A + B

nmnmmm

nn

nmm

n

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 16

Linear Algebra and Calculus Review

Example

303020201010

303202101

300020001000

300200100

302010

321

Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers

bullA + B = B + A

bullA + (B + C) = (A + B) + C

bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A

bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0

bullk(A + B) = kA + kB

bull(k+p)A = kA + pA

bull(kp)A = k(pA)

SundermeyerMAR 550

Spring 2013 17

Linear Algebra and Calculus Review

nm

m

T

nmm a

a

a

aaa

aa

aa

aa

33

22

11312

1

3313

2212

n1

13

12

11n1131211

a

a

a

aaaaa

Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar

AT (also denoted Arsquo)

[m x n] [n x m]

Example

303

202

101

302010

321T

SundermeyerMAR 550

Spring 2013 18

Linear Algebra and Calculus Review

Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix

bull(AT)T = A

bull(kA)T = kAT

bull(A + B)T = AT + BT

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 11

Linear Algebra and Calculus Review

Dot Product and Scalar Product Rules

uv is a scalar

uv = vu

u0 = 0 = 0u

uu = ||u||2

(ku)v = (k)uv = u(kv) for k scalar

u(v plusmn w) = uv plusmn uw

SundermeyerMAR 550

Spring 2013 12

Linear Algebra and Calculus Review

Cross Product Let

be two vectors of length 3 Then the cross product of the two vectors u and v is defined as

)(ˆ)(ˆ)(ˆ

detˆdetˆdetˆ

ˆˆˆ

det

122113312332

21

21

31

31

32

32

321

321

vuvukvuvujvuvui

vv

uuk

vv

uuj

vv

uui

vvv

uuu

kji

vu

Example

kjikji

kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ

713

214

ˆˆˆ

det713214

vu

Example (curl of a vector)

y

u

x

vk

z

u

x

wj

z

v

y

wi

wvuzyx

kji

wvuzyx

ˆˆˆ

ˆˆˆ

detv

321 uuuu 321 vvvv

SundermeyerMAR 550

Spring 2013 13

Linear Algebra and Calculus Review

Cross Product Rules

bull u times v is a vector

bull u times v is orthogonal to both u and v

bull u times 0 = 0 = 0 x u

bull u times u = 0

bull u times v = -(v times u)

bull (ku) times v = k(u times v) = u times (kv) for any scalar k

bull u times (v + w) = (u times v) + (u times w)

bull (v + w) times u = (v times u) + (w times u)

SundermeyerMAR 550

Spring 2013 14

Linear Algebra and Calculus Review

kk

a

a

a

ka

ka

ka

ka

ka

ka

a

a

a

kk

nnnn

AA

2

1

2

1

2

1

2

1

kz

Tjy

Tix

T

kz

Tj

y

Tix

T

z

T

y

T

x

TT

zyxT

ˆˆˆ

ˆˆˆ

NOTE In general for a vector A and a scalar k kA = Ak

However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie

SundermeyerMAR 550

Spring 2013 15

Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added

A + B

nmnmmm

nn

nmm

n

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 16

Linear Algebra and Calculus Review

Example

303020201010

303202101

300020001000

300200100

302010

321

Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers

bullA + B = B + A

bullA + (B + C) = (A + B) + C

bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A

bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0

bullk(A + B) = kA + kB

bull(k+p)A = kA + pA

bull(kp)A = k(pA)

SundermeyerMAR 550

Spring 2013 17

Linear Algebra and Calculus Review

nm

m

T

nmm a

a

a

aaa

aa

aa

aa

33

22

11312

1

3313

2212

n1

13

12

11n1131211

a

a

a

aaaaa

Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar

AT (also denoted Arsquo)

[m x n] [n x m]

Example

303

202

101

302010

321T

SundermeyerMAR 550

Spring 2013 18

Linear Algebra and Calculus Review

Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix

bull(AT)T = A

bull(kA)T = kAT

bull(A + B)T = AT + BT

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 12

Linear Algebra and Calculus Review

Cross Product Let

be two vectors of length 3 Then the cross product of the two vectors u and v is defined as

)(ˆ)(ˆ)(ˆ

detˆdetˆdetˆ

ˆˆˆ

det

122113312332

21

21

31

31

32

32

321

321

vuvukvuvujvuvui

vv

uuk

vv

uuj

vv

uui

vvv

uuu

kji

vu

Example

kjikji

kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ

713

214

ˆˆˆ

det713214

vu

Example (curl of a vector)

y

u

x

vk

z

u

x

wj

z

v

y

wi

wvuzyx

kji

wvuzyx

ˆˆˆ

ˆˆˆ

detv

321 uuuu 321 vvvv

SundermeyerMAR 550

Spring 2013 13

Linear Algebra and Calculus Review

Cross Product Rules

bull u times v is a vector

bull u times v is orthogonal to both u and v

bull u times 0 = 0 = 0 x u

bull u times u = 0

bull u times v = -(v times u)

bull (ku) times v = k(u times v) = u times (kv) for any scalar k

bull u times (v + w) = (u times v) + (u times w)

bull (v + w) times u = (v times u) + (w times u)

SundermeyerMAR 550

Spring 2013 14

Linear Algebra and Calculus Review

kk

a

a

a

ka

ka

ka

ka

ka

ka

a

a

a

kk

nnnn

AA

2

1

2

1

2

1

2

1

kz

Tjy

Tix

T

kz

Tj

y

Tix

T

z

T

y

T

x

TT

zyxT

ˆˆˆ

ˆˆˆ

NOTE In general for a vector A and a scalar k kA = Ak

However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie

SundermeyerMAR 550

Spring 2013 15

Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added

A + B

nmnmmm

nn

nmm

n

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 16

Linear Algebra and Calculus Review

Example

303020201010

303202101

300020001000

300200100

302010

321

Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers

bullA + B = B + A

bullA + (B + C) = (A + B) + C

bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A

bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0

bullk(A + B) = kA + kB

bull(k+p)A = kA + pA

bull(kp)A = k(pA)

SundermeyerMAR 550

Spring 2013 17

Linear Algebra and Calculus Review

nm

m

T

nmm a

a

a

aaa

aa

aa

aa

33

22

11312

1

3313

2212

n1

13

12

11n1131211

a

a

a

aaaaa

Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar

AT (also denoted Arsquo)

[m x n] [n x m]

Example

303

202

101

302010

321T

SundermeyerMAR 550

Spring 2013 18

Linear Algebra and Calculus Review

Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix

bull(AT)T = A

bull(kA)T = kAT

bull(A + B)T = AT + BT

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 13

Linear Algebra and Calculus Review

Cross Product Rules

bull u times v is a vector

bull u times v is orthogonal to both u and v

bull u times 0 = 0 = 0 x u

bull u times u = 0

bull u times v = -(v times u)

bull (ku) times v = k(u times v) = u times (kv) for any scalar k

bull u times (v + w) = (u times v) + (u times w)

bull (v + w) times u = (v times u) + (w times u)

SundermeyerMAR 550

Spring 2013 14

Linear Algebra and Calculus Review

kk

a

a

a

ka

ka

ka

ka

ka

ka

a

a

a

kk

nnnn

AA

2

1

2

1

2

1

2

1

kz

Tjy

Tix

T

kz

Tj

y

Tix

T

z

T

y

T

x

TT

zyxT

ˆˆˆ

ˆˆˆ

NOTE In general for a vector A and a scalar k kA = Ak

However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie

SundermeyerMAR 550

Spring 2013 15

Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added

A + B

nmnmmm

nn

nmm

n

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 16

Linear Algebra and Calculus Review

Example

303020201010

303202101

300020001000

300200100

302010

321

Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers

bullA + B = B + A

bullA + (B + C) = (A + B) + C

bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A

bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0

bullk(A + B) = kA + kB

bull(k+p)A = kA + pA

bull(kp)A = k(pA)

SundermeyerMAR 550

Spring 2013 17

Linear Algebra and Calculus Review

nm

m

T

nmm a

a

a

aaa

aa

aa

aa

33

22

11312

1

3313

2212

n1

13

12

11n1131211

a

a

a

aaaaa

Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar

AT (also denoted Arsquo)

[m x n] [n x m]

Example

303

202

101

302010

321T

SundermeyerMAR 550

Spring 2013 18

Linear Algebra and Calculus Review

Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix

bull(AT)T = A

bull(kA)T = kAT

bull(A + B)T = AT + BT

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 14

Linear Algebra and Calculus Review

kk

a

a

a

ka

ka

ka

ka

ka

ka

a

a

a

kk

nnnn

AA

2

1

2

1

2

1

2

1

kz

Tjy

Tix

T

kz

Tj

y

Tix

T

z

T

y

T

x

TT

zyxT

ˆˆˆ

ˆˆˆ

NOTE In general for a vector A and a scalar k kA = Ak

However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie

SundermeyerMAR 550

Spring 2013 15

Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added

A + B

nmnmmm

nn

nmm

n

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 16

Linear Algebra and Calculus Review

Example

303020201010

303202101

300020001000

300200100

302010

321

Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers

bullA + B = B + A

bullA + (B + C) = (A + B) + C

bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A

bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0

bullk(A + B) = kA + kB

bull(k+p)A = kA + pA

bull(kp)A = k(pA)

SundermeyerMAR 550

Spring 2013 17

Linear Algebra and Calculus Review

nm

m

T

nmm a

a

a

aaa

aa

aa

aa

33

22

11312

1

3313

2212

n1

13

12

11n1131211

a

a

a

aaaaa

Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar

AT (also denoted Arsquo)

[m x n] [n x m]

Example

303

202

101

302010

321T

SundermeyerMAR 550

Spring 2013 18

Linear Algebra and Calculus Review

Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix

bull(AT)T = A

bull(kA)T = kAT

bull(A + B)T = AT + BT

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 15

Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added

A + B

nmnmmm

nn

nmm

n

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Linear Algebra and Calculus Review

SundermeyerMAR 550

Spring 2013 16

Linear Algebra and Calculus Review

Example

303020201010

303202101

300020001000

300200100

302010

321

Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers

bullA + B = B + A

bullA + (B + C) = (A + B) + C

bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A

bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0

bullk(A + B) = kA + kB

bull(k+p)A = kA + pA

bull(kp)A = k(pA)

SundermeyerMAR 550

Spring 2013 17

Linear Algebra and Calculus Review

nm

m

T

nmm a

a

a

aaa

aa

aa

aa

33

22

11312

1

3313

2212

n1

13

12

11n1131211

a

a

a

aaaaa

Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar

AT (also denoted Arsquo)

[m x n] [n x m]

Example

303

202

101

302010

321T

SundermeyerMAR 550

Spring 2013 18

Linear Algebra and Calculus Review

Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix

bull(AT)T = A

bull(kA)T = kAT

bull(A + B)T = AT + BT

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 16

Linear Algebra and Calculus Review

Example

303020201010

303202101

300020001000

300200100

302010

321

Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers

bullA + B = B + A

bullA + (B + C) = (A + B) + C

bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A

bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0

bullk(A + B) = kA + kB

bull(k+p)A = kA + pA

bull(kp)A = k(pA)

SundermeyerMAR 550

Spring 2013 17

Linear Algebra and Calculus Review

nm

m

T

nmm a

a

a

aaa

aa

aa

aa

33

22

11312

1

3313

2212

n1

13

12

11n1131211

a

a

a

aaaaa

Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar

AT (also denoted Arsquo)

[m x n] [n x m]

Example

303

202

101

302010

321T

SundermeyerMAR 550

Spring 2013 18

Linear Algebra and Calculus Review

Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix

bull(AT)T = A

bull(kA)T = kAT

bull(A + B)T = AT + BT

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 17

Linear Algebra and Calculus Review

nm

m

T

nmm a

a

a

aaa

aa

aa

aa

33

22

11312

1

3313

2212

n1

13

12

11n1131211

a

a

a

aaaaa

Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar

AT (also denoted Arsquo)

[m x n] [n x m]

Example

303

202

101

302010

321T

SundermeyerMAR 550

Spring 2013 18

Linear Algebra and Calculus Review

Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix

bull(AT)T = A

bull(kA)T = kAT

bull(A + B)T = AT + BT

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 18

Linear Algebra and Calculus Review

Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix

bull(AT)T = A

bull(kA)T = kAT

bull(A + B)T = AT + BT

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 19

Linear Algebra and Calculus Review

Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar

Scalar Multiplication sA

nmm

n

nmm

n

asas

asas

asas

asasasas

aa

aa

aa

aaaa

s

1

3313

2212

1312111

1

3313

2212

1312111

Example

2412

812

63

23 4

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 20

Linear Algebra and Calculus Review

mnn

m

nmm

n

bb

bb

bb

bbbb

aa

aa

aa

aaaa

1

3313

2212

1312111

1

3313

2212

1312111

Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]

= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B

Example (inner (dot) product)

32)63()52()41(

6

5

4

321

[1 x 3][3 x 1] = [1 x 1]

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 21

Linear Algebra and Calculus Review

Example (outer product)

18126

15105

1284

362616

352515

342414

321

6

5

4

Example (general matrix product)

1400140

14014

)3030()2020()1010()330()220()110(

)303()202()101()33()22()11(

303

202

101

302010

321

[2 x 3] [3 x 2] [2 x 2]

[3 x 1][1 x 3] = [3 x 3]

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 22

Linear Algebra and Calculus Review

Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed

bullIA = A BI = B

bullA(BC) = (AB)C

bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC

bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA

bullk(AB) = (kA)B = A(kB)

bull(AB)T = BTAT

NOTE In general matrix multiplication is not commutative AB ne BA

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 23

Linear Algebra and Calculus Review

Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse

Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B

NOTE Consider general matrix expression

A X = B A-1 A X = A-1 B

A-1 A X = A-1 B 1 X = A-1 B X = A-1 B

Also note not all matrices are invertible eg the matrix has no inverse

31

00

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 24

Linear Algebra and Calculus Review

Special Matrix Algebra Rules in Matlab

Matrix + Scalar Addition A + s

sasa

sasa

sasa

sasasasa

s

aa

aa

aa

aaaa

nmm

n

nmm

n

1

3313

2212

1312111

1

3313

2212

1312111

Example

130120110

103102101100

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321

SundermeyerMAR 550

Spring 2013 25

Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab

Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)

mnnmnm

mn

mnn

m

nmm

n

baba

baba

baba

babababa

bb

bb

bb

bbbb

aa

aa

aa

aaaa

11

33331313

22221212

11313121211111

1

3313

2212

1312111

1

3313

2212

1312111

Example

900040001000

904010

300200100

302010

302010

321