sundermeyer mar 550 spring 2013 1 laboratory in oceanography: data and methods mar550, spring 2013...
TRANSCRIPT
SundermeyerMAR 550
Spring 2013 1
Laboratory in Oceanography Data and Methods
MAR550 Spring 2013
Miles A Sundermeyer
Linear Algebra amp Calculus Review
SundermeyerMAR 550
Spring 2013 2
Nomenclature
scalar A scalar is a variable that only has magnitude eg a speed of 40 kmh 10 a (42 + 7) log10(a)
vector A geometric entity with both length and direction a quantity comprising both magnitude and direction eg a velocity of 40 kmh north velocity u position x = (x y z)
array An indexed set or group of elements also can be used to represent vectors eg
row vectorarray column vectorarray
21 13 8 5 3 2 1 1
21
13
8
5
3
2
1
1
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 3
matrix A rectangular table of elements (or entries) which may be numbers or more generally any abstract quantities that can be added and multiplied effectively a generalized array or vector - a collection of numbers ordered by rows and columns
[2 x 3] matrix
[m x n] matrix
302010
321
nmm
n
aa
aa
aa
aaaa
1
3313
2212
1312111
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 4
Linear Algebra and Calculus Review
Examples (special matrices)
A square matrix has as many rows as it has columns Matrix A is square but matrix B is not
071
5122
543
A
5122
543B
A symmetric matrix is a square matrix in which xij = xji for all i and j
A symmetric matrix is equal to its transpose Matrix A is symmetric matrix B is not
0101
10122
121
A
0101
21210
121
B
SundermeyerMAR 550
Spring 2013 5
Linear Algebra and Calculus Review
A diagonal matrix is a symmetric matrix where all the off diagonal elements are 0 The matrix D is diagonal
An identity matrix is a diagonal matrix with only 1rsquos on the diagonal For any square matrix A the product IA = AI = A The identity matrix is generally denoted as I
IA = A
100
010
001
I
0101
10122
121
0101
10122
121
100
010
001
700
020
004
D
SundermeyerMAR 550
Spring 2013 6
Linear Algebra and Calculus Review
Example (system of equations)
Suppose we have a series of measurements of stream discharge and stage measured at n different times
time (day) = [0 14 28 42 56 70] stage (m) = [0612 0647 0580 0629 0688 0583]discharge (m3s) = [0330 0395 0241 0338 0531 0279]
Suppose we now wish to fit a rating curve to these measurements Let x = stage y = discharge then we can write this series of measurements as
yi = mxi + b with i = 1n
This in turn can be written as y = X b or
]12[ ]2[ ]1[
1
1
1
1
3
2
1
2
2
1
nn
b
m
x
x
x
x
y
y
y
y
nn
SundermeyerMAR 550
Spring 2013 7
Linear Algebra and Calculus Review
yi = mxi + b
y = X b
]12[ ]2[ ]1[
1
1
1
1
3
2
1
2
2
1
nn
b
m
x
x
x
x
y
y
y
y
nn
SundermeyerMAR 550
Spring 2013 8
Linear Algebra and Calculus Review
VectorsAdditionSubtraction Two vectors can be addedsubtracted if and only if they are of the same dimension
nnnn ba
ba
ba
b
b
b
a
a
a
22
11
2
1
2
1
BA
8
7
1
7
71
34
23
52
7
3
2
5
1
4
3
2Example
SundermeyerMAR 550
Spring 2013 9
Linear Algebra and Calculus Review
Scalar Multiplication If k is a scalar and A is a n-dimensional vector then
nn ka
ka
ka
a
a
a
kk
2
1
2
1
A
Example
10
40
50
52
202
252
5
20
25
2
Example
A + B ndash 3C where
2
1
10
2
1
1
6
3
2
CBA
2
1
27
626
313
3012
2
1
10
3
2
1
1
6
3
2
3CBA
SundermeyerMAR 550
Spring 2013 10
Linear Algebra and Calculus Review
Dot Product Let
be two vectors of length n Then the dot product of the two vectors u and v is defined as
nuuu 21 u nvvv 21 v
n
iiinn vuvuvuvu
12211 vu
A dot product is also an inner product
Example
33)23()72()11()34(27133214 vu
Example (divergence of a vector)
z
w
y
v
x
uwvu
zyx
v
SundermeyerMAR 550
Spring 2013 11
Linear Algebra and Calculus Review
Dot Product and Scalar Product Rules
uv is a scalar
uv = vu
u0 = 0 = 0u
uu = ||u||2
(ku)v = (k)uv = u(kv) for k scalar
u(v plusmn w) = uv plusmn uw
SundermeyerMAR 550
Spring 2013 12
Linear Algebra and Calculus Review
Cross Product Let
be two vectors of length 3 Then the cross product of the two vectors u and v is defined as
)(ˆ)(ˆ)(ˆ
detˆdetˆdetˆ
ˆˆˆ
det
122113312332
21
21
31
31
32
32
321
321
vuvukvuvujvuvui
vv
uuk
vv
uuj
vv
uui
vvv
uuu
kji
vu
Example
kjikji
kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ
713
214
ˆˆˆ
det713214
vu
Example (curl of a vector)
y
u
x
vk
z
u
x
wj
z
v
y
wi
wvuzyx
kji
wvuzyx
ˆˆˆ
ˆˆˆ
detv
321 uuuu 321 vvvv
SundermeyerMAR 550
Spring 2013 13
Linear Algebra and Calculus Review
Cross Product Rules
bull u times v is a vector
bull u times v is orthogonal to both u and v
bull u times 0 = 0 = 0 x u
bull u times u = 0
bull u times v = -(v times u)
bull (ku) times v = k(u times v) = u times (kv) for any scalar k
bull u times (v + w) = (u times v) + (u times w)
bull (v + w) times u = (v times u) + (w times u)
SundermeyerMAR 550
Spring 2013 14
Linear Algebra and Calculus Review
kk
a
a
a
ka
ka
ka
ka
ka
ka
a
a
a
kk
nnnn
AA
2
1
2
1
2
1
2
1
kz
Tjy
Tix
T
kz
Tj
y
Tix
T
z
T
y
T
x
TT
zyxT
ˆˆˆ
ˆˆˆ
NOTE In general for a vector A and a scalar k kA = Ak
However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie
SundermeyerMAR 550
Spring 2013 15
Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added
A + B
nmnmmm
nn
nmm
n
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 16
Linear Algebra and Calculus Review
Example
303020201010
303202101
300020001000
300200100
302010
321
Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers
bullA + B = B + A
bullA + (B + C) = (A + B) + C
bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A
bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0
bullk(A + B) = kA + kB
bull(k+p)A = kA + pA
bull(kp)A = k(pA)
SundermeyerMAR 550
Spring 2013 17
Linear Algebra and Calculus Review
nm
m
T
nmm a
a
a
aaa
aa
aa
aa
33
22
11312
1
3313
2212
n1
13
12
11n1131211
a
a
a
aaaaa
Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar
AT (also denoted Arsquo)
[m x n] [n x m]
Example
303
202
101
302010
321T
SundermeyerMAR 550
Spring 2013 18
Linear Algebra and Calculus Review
Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix
bull(AT)T = A
bull(kA)T = kAT
bull(A + B)T = AT + BT
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 2
Nomenclature
scalar A scalar is a variable that only has magnitude eg a speed of 40 kmh 10 a (42 + 7) log10(a)
vector A geometric entity with both length and direction a quantity comprising both magnitude and direction eg a velocity of 40 kmh north velocity u position x = (x y z)
array An indexed set or group of elements also can be used to represent vectors eg
row vectorarray column vectorarray
21 13 8 5 3 2 1 1
21
13
8
5
3
2
1
1
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 3
matrix A rectangular table of elements (or entries) which may be numbers or more generally any abstract quantities that can be added and multiplied effectively a generalized array or vector - a collection of numbers ordered by rows and columns
[2 x 3] matrix
[m x n] matrix
302010
321
nmm
n
aa
aa
aa
aaaa
1
3313
2212
1312111
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 4
Linear Algebra and Calculus Review
Examples (special matrices)
A square matrix has as many rows as it has columns Matrix A is square but matrix B is not
071
5122
543
A
5122
543B
A symmetric matrix is a square matrix in which xij = xji for all i and j
A symmetric matrix is equal to its transpose Matrix A is symmetric matrix B is not
0101
10122
121
A
0101
21210
121
B
SundermeyerMAR 550
Spring 2013 5
Linear Algebra and Calculus Review
A diagonal matrix is a symmetric matrix where all the off diagonal elements are 0 The matrix D is diagonal
An identity matrix is a diagonal matrix with only 1rsquos on the diagonal For any square matrix A the product IA = AI = A The identity matrix is generally denoted as I
IA = A
100
010
001
I
0101
10122
121
0101
10122
121
100
010
001
700
020
004
D
SundermeyerMAR 550
Spring 2013 6
Linear Algebra and Calculus Review
Example (system of equations)
Suppose we have a series of measurements of stream discharge and stage measured at n different times
time (day) = [0 14 28 42 56 70] stage (m) = [0612 0647 0580 0629 0688 0583]discharge (m3s) = [0330 0395 0241 0338 0531 0279]
Suppose we now wish to fit a rating curve to these measurements Let x = stage y = discharge then we can write this series of measurements as
yi = mxi + b with i = 1n
This in turn can be written as y = X b or
]12[ ]2[ ]1[
1
1
1
1
3
2
1
2
2
1
nn
b
m
x
x
x
x
y
y
y
y
nn
SundermeyerMAR 550
Spring 2013 7
Linear Algebra and Calculus Review
yi = mxi + b
y = X b
]12[ ]2[ ]1[
1
1
1
1
3
2
1
2
2
1
nn
b
m
x
x
x
x
y
y
y
y
nn
SundermeyerMAR 550
Spring 2013 8
Linear Algebra and Calculus Review
VectorsAdditionSubtraction Two vectors can be addedsubtracted if and only if they are of the same dimension
nnnn ba
ba
ba
b
b
b
a
a
a
22
11
2
1
2
1
BA
8
7
1
7
71
34
23
52
7
3
2
5
1
4
3
2Example
SundermeyerMAR 550
Spring 2013 9
Linear Algebra and Calculus Review
Scalar Multiplication If k is a scalar and A is a n-dimensional vector then
nn ka
ka
ka
a
a
a
kk
2
1
2
1
A
Example
10
40
50
52
202
252
5
20
25
2
Example
A + B ndash 3C where
2
1
10
2
1
1
6
3
2
CBA
2
1
27
626
313
3012
2
1
10
3
2
1
1
6
3
2
3CBA
SundermeyerMAR 550
Spring 2013 10
Linear Algebra and Calculus Review
Dot Product Let
be two vectors of length n Then the dot product of the two vectors u and v is defined as
nuuu 21 u nvvv 21 v
n
iiinn vuvuvuvu
12211 vu
A dot product is also an inner product
Example
33)23()72()11()34(27133214 vu
Example (divergence of a vector)
z
w
y
v
x
uwvu
zyx
v
SundermeyerMAR 550
Spring 2013 11
Linear Algebra and Calculus Review
Dot Product and Scalar Product Rules
uv is a scalar
uv = vu
u0 = 0 = 0u
uu = ||u||2
(ku)v = (k)uv = u(kv) for k scalar
u(v plusmn w) = uv plusmn uw
SundermeyerMAR 550
Spring 2013 12
Linear Algebra and Calculus Review
Cross Product Let
be two vectors of length 3 Then the cross product of the two vectors u and v is defined as
)(ˆ)(ˆ)(ˆ
detˆdetˆdetˆ
ˆˆˆ
det
122113312332
21
21
31
31
32
32
321
321
vuvukvuvujvuvui
vv
uuk
vv
uuj
vv
uui
vvv
uuu
kji
vu
Example
kjikji
kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ
713
214
ˆˆˆ
det713214
vu
Example (curl of a vector)
y
u
x
vk
z
u
x
wj
z
v
y
wi
wvuzyx
kji
wvuzyx
ˆˆˆ
ˆˆˆ
detv
321 uuuu 321 vvvv
SundermeyerMAR 550
Spring 2013 13
Linear Algebra and Calculus Review
Cross Product Rules
bull u times v is a vector
bull u times v is orthogonal to both u and v
bull u times 0 = 0 = 0 x u
bull u times u = 0
bull u times v = -(v times u)
bull (ku) times v = k(u times v) = u times (kv) for any scalar k
bull u times (v + w) = (u times v) + (u times w)
bull (v + w) times u = (v times u) + (w times u)
SundermeyerMAR 550
Spring 2013 14
Linear Algebra and Calculus Review
kk
a
a
a
ka
ka
ka
ka
ka
ka
a
a
a
kk
nnnn
AA
2
1
2
1
2
1
2
1
kz
Tjy
Tix
T
kz
Tj
y
Tix
T
z
T
y
T
x
TT
zyxT
ˆˆˆ
ˆˆˆ
NOTE In general for a vector A and a scalar k kA = Ak
However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie
SundermeyerMAR 550
Spring 2013 15
Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added
A + B
nmnmmm
nn
nmm
n
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 16
Linear Algebra and Calculus Review
Example
303020201010
303202101
300020001000
300200100
302010
321
Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers
bullA + B = B + A
bullA + (B + C) = (A + B) + C
bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A
bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0
bullk(A + B) = kA + kB
bull(k+p)A = kA + pA
bull(kp)A = k(pA)
SundermeyerMAR 550
Spring 2013 17
Linear Algebra and Calculus Review
nm
m
T
nmm a
a
a
aaa
aa
aa
aa
33
22
11312
1
3313
2212
n1
13
12
11n1131211
a
a
a
aaaaa
Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar
AT (also denoted Arsquo)
[m x n] [n x m]
Example
303
202
101
302010
321T
SundermeyerMAR 550
Spring 2013 18
Linear Algebra and Calculus Review
Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix
bull(AT)T = A
bull(kA)T = kAT
bull(A + B)T = AT + BT
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 3
matrix A rectangular table of elements (or entries) which may be numbers or more generally any abstract quantities that can be added and multiplied effectively a generalized array or vector - a collection of numbers ordered by rows and columns
[2 x 3] matrix
[m x n] matrix
302010
321
nmm
n
aa
aa
aa
aaaa
1
3313
2212
1312111
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 4
Linear Algebra and Calculus Review
Examples (special matrices)
A square matrix has as many rows as it has columns Matrix A is square but matrix B is not
071
5122
543
A
5122
543B
A symmetric matrix is a square matrix in which xij = xji for all i and j
A symmetric matrix is equal to its transpose Matrix A is symmetric matrix B is not
0101
10122
121
A
0101
21210
121
B
SundermeyerMAR 550
Spring 2013 5
Linear Algebra and Calculus Review
A diagonal matrix is a symmetric matrix where all the off diagonal elements are 0 The matrix D is diagonal
An identity matrix is a diagonal matrix with only 1rsquos on the diagonal For any square matrix A the product IA = AI = A The identity matrix is generally denoted as I
IA = A
100
010
001
I
0101
10122
121
0101
10122
121
100
010
001
700
020
004
D
SundermeyerMAR 550
Spring 2013 6
Linear Algebra and Calculus Review
Example (system of equations)
Suppose we have a series of measurements of stream discharge and stage measured at n different times
time (day) = [0 14 28 42 56 70] stage (m) = [0612 0647 0580 0629 0688 0583]discharge (m3s) = [0330 0395 0241 0338 0531 0279]
Suppose we now wish to fit a rating curve to these measurements Let x = stage y = discharge then we can write this series of measurements as
yi = mxi + b with i = 1n
This in turn can be written as y = X b or
]12[ ]2[ ]1[
1
1
1
1
3
2
1
2
2
1
nn
b
m
x
x
x
x
y
y
y
y
nn
SundermeyerMAR 550
Spring 2013 7
Linear Algebra and Calculus Review
yi = mxi + b
y = X b
]12[ ]2[ ]1[
1
1
1
1
3
2
1
2
2
1
nn
b
m
x
x
x
x
y
y
y
y
nn
SundermeyerMAR 550
Spring 2013 8
Linear Algebra and Calculus Review
VectorsAdditionSubtraction Two vectors can be addedsubtracted if and only if they are of the same dimension
nnnn ba
ba
ba
b
b
b
a
a
a
22
11
2
1
2
1
BA
8
7
1
7
71
34
23
52
7
3
2
5
1
4
3
2Example
SundermeyerMAR 550
Spring 2013 9
Linear Algebra and Calculus Review
Scalar Multiplication If k is a scalar and A is a n-dimensional vector then
nn ka
ka
ka
a
a
a
kk
2
1
2
1
A
Example
10
40
50
52
202
252
5
20
25
2
Example
A + B ndash 3C where
2
1
10
2
1
1
6
3
2
CBA
2
1
27
626
313
3012
2
1
10
3
2
1
1
6
3
2
3CBA
SundermeyerMAR 550
Spring 2013 10
Linear Algebra and Calculus Review
Dot Product Let
be two vectors of length n Then the dot product of the two vectors u and v is defined as
nuuu 21 u nvvv 21 v
n
iiinn vuvuvuvu
12211 vu
A dot product is also an inner product
Example
33)23()72()11()34(27133214 vu
Example (divergence of a vector)
z
w
y
v
x
uwvu
zyx
v
SundermeyerMAR 550
Spring 2013 11
Linear Algebra and Calculus Review
Dot Product and Scalar Product Rules
uv is a scalar
uv = vu
u0 = 0 = 0u
uu = ||u||2
(ku)v = (k)uv = u(kv) for k scalar
u(v plusmn w) = uv plusmn uw
SundermeyerMAR 550
Spring 2013 12
Linear Algebra and Calculus Review
Cross Product Let
be two vectors of length 3 Then the cross product of the two vectors u and v is defined as
)(ˆ)(ˆ)(ˆ
detˆdetˆdetˆ
ˆˆˆ
det
122113312332
21
21
31
31
32
32
321
321
vuvukvuvujvuvui
vv
uuk
vv
uuj
vv
uui
vvv
uuu
kji
vu
Example
kjikji
kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ
713
214
ˆˆˆ
det713214
vu
Example (curl of a vector)
y
u
x
vk
z
u
x
wj
z
v
y
wi
wvuzyx
kji
wvuzyx
ˆˆˆ
ˆˆˆ
detv
321 uuuu 321 vvvv
SundermeyerMAR 550
Spring 2013 13
Linear Algebra and Calculus Review
Cross Product Rules
bull u times v is a vector
bull u times v is orthogonal to both u and v
bull u times 0 = 0 = 0 x u
bull u times u = 0
bull u times v = -(v times u)
bull (ku) times v = k(u times v) = u times (kv) for any scalar k
bull u times (v + w) = (u times v) + (u times w)
bull (v + w) times u = (v times u) + (w times u)
SundermeyerMAR 550
Spring 2013 14
Linear Algebra and Calculus Review
kk
a
a
a
ka
ka
ka
ka
ka
ka
a
a
a
kk
nnnn
AA
2
1
2
1
2
1
2
1
kz
Tjy
Tix
T
kz
Tj
y
Tix
T
z
T
y
T
x
TT
zyxT
ˆˆˆ
ˆˆˆ
NOTE In general for a vector A and a scalar k kA = Ak
However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie
SundermeyerMAR 550
Spring 2013 15
Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added
A + B
nmnmmm
nn
nmm
n
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 16
Linear Algebra and Calculus Review
Example
303020201010
303202101
300020001000
300200100
302010
321
Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers
bullA + B = B + A
bullA + (B + C) = (A + B) + C
bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A
bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0
bullk(A + B) = kA + kB
bull(k+p)A = kA + pA
bull(kp)A = k(pA)
SundermeyerMAR 550
Spring 2013 17
Linear Algebra and Calculus Review
nm
m
T
nmm a
a
a
aaa
aa
aa
aa
33
22
11312
1
3313
2212
n1
13
12
11n1131211
a
a
a
aaaaa
Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar
AT (also denoted Arsquo)
[m x n] [n x m]
Example
303
202
101
302010
321T
SundermeyerMAR 550
Spring 2013 18
Linear Algebra and Calculus Review
Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix
bull(AT)T = A
bull(kA)T = kAT
bull(A + B)T = AT + BT
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 4
Linear Algebra and Calculus Review
Examples (special matrices)
A square matrix has as many rows as it has columns Matrix A is square but matrix B is not
071
5122
543
A
5122
543B
A symmetric matrix is a square matrix in which xij = xji for all i and j
A symmetric matrix is equal to its transpose Matrix A is symmetric matrix B is not
0101
10122
121
A
0101
21210
121
B
SundermeyerMAR 550
Spring 2013 5
Linear Algebra and Calculus Review
A diagonal matrix is a symmetric matrix where all the off diagonal elements are 0 The matrix D is diagonal
An identity matrix is a diagonal matrix with only 1rsquos on the diagonal For any square matrix A the product IA = AI = A The identity matrix is generally denoted as I
IA = A
100
010
001
I
0101
10122
121
0101
10122
121
100
010
001
700
020
004
D
SundermeyerMAR 550
Spring 2013 6
Linear Algebra and Calculus Review
Example (system of equations)
Suppose we have a series of measurements of stream discharge and stage measured at n different times
time (day) = [0 14 28 42 56 70] stage (m) = [0612 0647 0580 0629 0688 0583]discharge (m3s) = [0330 0395 0241 0338 0531 0279]
Suppose we now wish to fit a rating curve to these measurements Let x = stage y = discharge then we can write this series of measurements as
yi = mxi + b with i = 1n
This in turn can be written as y = X b or
]12[ ]2[ ]1[
1
1
1
1
3
2
1
2
2
1
nn
b
m
x
x
x
x
y
y
y
y
nn
SundermeyerMAR 550
Spring 2013 7
Linear Algebra and Calculus Review
yi = mxi + b
y = X b
]12[ ]2[ ]1[
1
1
1
1
3
2
1
2
2
1
nn
b
m
x
x
x
x
y
y
y
y
nn
SundermeyerMAR 550
Spring 2013 8
Linear Algebra and Calculus Review
VectorsAdditionSubtraction Two vectors can be addedsubtracted if and only if they are of the same dimension
nnnn ba
ba
ba
b
b
b
a
a
a
22
11
2
1
2
1
BA
8
7
1
7
71
34
23
52
7
3
2
5
1
4
3
2Example
SundermeyerMAR 550
Spring 2013 9
Linear Algebra and Calculus Review
Scalar Multiplication If k is a scalar and A is a n-dimensional vector then
nn ka
ka
ka
a
a
a
kk
2
1
2
1
A
Example
10
40
50
52
202
252
5
20
25
2
Example
A + B ndash 3C where
2
1
10
2
1
1
6
3
2
CBA
2
1
27
626
313
3012
2
1
10
3
2
1
1
6
3
2
3CBA
SundermeyerMAR 550
Spring 2013 10
Linear Algebra and Calculus Review
Dot Product Let
be two vectors of length n Then the dot product of the two vectors u and v is defined as
nuuu 21 u nvvv 21 v
n
iiinn vuvuvuvu
12211 vu
A dot product is also an inner product
Example
33)23()72()11()34(27133214 vu
Example (divergence of a vector)
z
w
y
v
x
uwvu
zyx
v
SundermeyerMAR 550
Spring 2013 11
Linear Algebra and Calculus Review
Dot Product and Scalar Product Rules
uv is a scalar
uv = vu
u0 = 0 = 0u
uu = ||u||2
(ku)v = (k)uv = u(kv) for k scalar
u(v plusmn w) = uv plusmn uw
SundermeyerMAR 550
Spring 2013 12
Linear Algebra and Calculus Review
Cross Product Let
be two vectors of length 3 Then the cross product of the two vectors u and v is defined as
)(ˆ)(ˆ)(ˆ
detˆdetˆdetˆ
ˆˆˆ
det
122113312332
21
21
31
31
32
32
321
321
vuvukvuvujvuvui
vv
uuk
vv
uuj
vv
uui
vvv
uuu
kji
vu
Example
kjikji
kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ
713
214
ˆˆˆ
det713214
vu
Example (curl of a vector)
y
u
x
vk
z
u
x
wj
z
v
y
wi
wvuzyx
kji
wvuzyx
ˆˆˆ
ˆˆˆ
detv
321 uuuu 321 vvvv
SundermeyerMAR 550
Spring 2013 13
Linear Algebra and Calculus Review
Cross Product Rules
bull u times v is a vector
bull u times v is orthogonal to both u and v
bull u times 0 = 0 = 0 x u
bull u times u = 0
bull u times v = -(v times u)
bull (ku) times v = k(u times v) = u times (kv) for any scalar k
bull u times (v + w) = (u times v) + (u times w)
bull (v + w) times u = (v times u) + (w times u)
SundermeyerMAR 550
Spring 2013 14
Linear Algebra and Calculus Review
kk
a
a
a
ka
ka
ka
ka
ka
ka
a
a
a
kk
nnnn
AA
2
1
2
1
2
1
2
1
kz
Tjy
Tix
T
kz
Tj
y
Tix
T
z
T
y
T
x
TT
zyxT
ˆˆˆ
ˆˆˆ
NOTE In general for a vector A and a scalar k kA = Ak
However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie
SundermeyerMAR 550
Spring 2013 15
Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added
A + B
nmnmmm
nn
nmm
n
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 16
Linear Algebra and Calculus Review
Example
303020201010
303202101
300020001000
300200100
302010
321
Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers
bullA + B = B + A
bullA + (B + C) = (A + B) + C
bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A
bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0
bullk(A + B) = kA + kB
bull(k+p)A = kA + pA
bull(kp)A = k(pA)
SundermeyerMAR 550
Spring 2013 17
Linear Algebra and Calculus Review
nm
m
T
nmm a
a
a
aaa
aa
aa
aa
33
22
11312
1
3313
2212
n1
13
12
11n1131211
a
a
a
aaaaa
Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar
AT (also denoted Arsquo)
[m x n] [n x m]
Example
303
202
101
302010
321T
SundermeyerMAR 550
Spring 2013 18
Linear Algebra and Calculus Review
Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix
bull(AT)T = A
bull(kA)T = kAT
bull(A + B)T = AT + BT
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 5
Linear Algebra and Calculus Review
A diagonal matrix is a symmetric matrix where all the off diagonal elements are 0 The matrix D is diagonal
An identity matrix is a diagonal matrix with only 1rsquos on the diagonal For any square matrix A the product IA = AI = A The identity matrix is generally denoted as I
IA = A
100
010
001
I
0101
10122
121
0101
10122
121
100
010
001
700
020
004
D
SundermeyerMAR 550
Spring 2013 6
Linear Algebra and Calculus Review
Example (system of equations)
Suppose we have a series of measurements of stream discharge and stage measured at n different times
time (day) = [0 14 28 42 56 70] stage (m) = [0612 0647 0580 0629 0688 0583]discharge (m3s) = [0330 0395 0241 0338 0531 0279]
Suppose we now wish to fit a rating curve to these measurements Let x = stage y = discharge then we can write this series of measurements as
yi = mxi + b with i = 1n
This in turn can be written as y = X b or
]12[ ]2[ ]1[
1
1
1
1
3
2
1
2
2
1
nn
b
m
x
x
x
x
y
y
y
y
nn
SundermeyerMAR 550
Spring 2013 7
Linear Algebra and Calculus Review
yi = mxi + b
y = X b
]12[ ]2[ ]1[
1
1
1
1
3
2
1
2
2
1
nn
b
m
x
x
x
x
y
y
y
y
nn
SundermeyerMAR 550
Spring 2013 8
Linear Algebra and Calculus Review
VectorsAdditionSubtraction Two vectors can be addedsubtracted if and only if they are of the same dimension
nnnn ba
ba
ba
b
b
b
a
a
a
22
11
2
1
2
1
BA
8
7
1
7
71
34
23
52
7
3
2
5
1
4
3
2Example
SundermeyerMAR 550
Spring 2013 9
Linear Algebra and Calculus Review
Scalar Multiplication If k is a scalar and A is a n-dimensional vector then
nn ka
ka
ka
a
a
a
kk
2
1
2
1
A
Example
10
40
50
52
202
252
5
20
25
2
Example
A + B ndash 3C where
2
1
10
2
1
1
6
3
2
CBA
2
1
27
626
313
3012
2
1
10
3
2
1
1
6
3
2
3CBA
SundermeyerMAR 550
Spring 2013 10
Linear Algebra and Calculus Review
Dot Product Let
be two vectors of length n Then the dot product of the two vectors u and v is defined as
nuuu 21 u nvvv 21 v
n
iiinn vuvuvuvu
12211 vu
A dot product is also an inner product
Example
33)23()72()11()34(27133214 vu
Example (divergence of a vector)
z
w
y
v
x
uwvu
zyx
v
SundermeyerMAR 550
Spring 2013 11
Linear Algebra and Calculus Review
Dot Product and Scalar Product Rules
uv is a scalar
uv = vu
u0 = 0 = 0u
uu = ||u||2
(ku)v = (k)uv = u(kv) for k scalar
u(v plusmn w) = uv plusmn uw
SundermeyerMAR 550
Spring 2013 12
Linear Algebra and Calculus Review
Cross Product Let
be two vectors of length 3 Then the cross product of the two vectors u and v is defined as
)(ˆ)(ˆ)(ˆ
detˆdetˆdetˆ
ˆˆˆ
det
122113312332
21
21
31
31
32
32
321
321
vuvukvuvujvuvui
vv
uuk
vv
uuj
vv
uui
vvv
uuu
kji
vu
Example
kjikji
kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ
713
214
ˆˆˆ
det713214
vu
Example (curl of a vector)
y
u
x
vk
z
u
x
wj
z
v
y
wi
wvuzyx
kji
wvuzyx
ˆˆˆ
ˆˆˆ
detv
321 uuuu 321 vvvv
SundermeyerMAR 550
Spring 2013 13
Linear Algebra and Calculus Review
Cross Product Rules
bull u times v is a vector
bull u times v is orthogonal to both u and v
bull u times 0 = 0 = 0 x u
bull u times u = 0
bull u times v = -(v times u)
bull (ku) times v = k(u times v) = u times (kv) for any scalar k
bull u times (v + w) = (u times v) + (u times w)
bull (v + w) times u = (v times u) + (w times u)
SundermeyerMAR 550
Spring 2013 14
Linear Algebra and Calculus Review
kk
a
a
a
ka
ka
ka
ka
ka
ka
a
a
a
kk
nnnn
AA
2
1
2
1
2
1
2
1
kz
Tjy
Tix
T
kz
Tj
y
Tix
T
z
T
y
T
x
TT
zyxT
ˆˆˆ
ˆˆˆ
NOTE In general for a vector A and a scalar k kA = Ak
However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie
SundermeyerMAR 550
Spring 2013 15
Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added
A + B
nmnmmm
nn
nmm
n
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 16
Linear Algebra and Calculus Review
Example
303020201010
303202101
300020001000
300200100
302010
321
Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers
bullA + B = B + A
bullA + (B + C) = (A + B) + C
bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A
bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0
bullk(A + B) = kA + kB
bull(k+p)A = kA + pA
bull(kp)A = k(pA)
SundermeyerMAR 550
Spring 2013 17
Linear Algebra and Calculus Review
nm
m
T
nmm a
a
a
aaa
aa
aa
aa
33
22
11312
1
3313
2212
n1
13
12
11n1131211
a
a
a
aaaaa
Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar
AT (also denoted Arsquo)
[m x n] [n x m]
Example
303
202
101
302010
321T
SundermeyerMAR 550
Spring 2013 18
Linear Algebra and Calculus Review
Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix
bull(AT)T = A
bull(kA)T = kAT
bull(A + B)T = AT + BT
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 6
Linear Algebra and Calculus Review
Example (system of equations)
Suppose we have a series of measurements of stream discharge and stage measured at n different times
time (day) = [0 14 28 42 56 70] stage (m) = [0612 0647 0580 0629 0688 0583]discharge (m3s) = [0330 0395 0241 0338 0531 0279]
Suppose we now wish to fit a rating curve to these measurements Let x = stage y = discharge then we can write this series of measurements as
yi = mxi + b with i = 1n
This in turn can be written as y = X b or
]12[ ]2[ ]1[
1
1
1
1
3
2
1
2
2
1
nn
b
m
x
x
x
x
y
y
y
y
nn
SundermeyerMAR 550
Spring 2013 7
Linear Algebra and Calculus Review
yi = mxi + b
y = X b
]12[ ]2[ ]1[
1
1
1
1
3
2
1
2
2
1
nn
b
m
x
x
x
x
y
y
y
y
nn
SundermeyerMAR 550
Spring 2013 8
Linear Algebra and Calculus Review
VectorsAdditionSubtraction Two vectors can be addedsubtracted if and only if they are of the same dimension
nnnn ba
ba
ba
b
b
b
a
a
a
22
11
2
1
2
1
BA
8
7
1
7
71
34
23
52
7
3
2
5
1
4
3
2Example
SundermeyerMAR 550
Spring 2013 9
Linear Algebra and Calculus Review
Scalar Multiplication If k is a scalar and A is a n-dimensional vector then
nn ka
ka
ka
a
a
a
kk
2
1
2
1
A
Example
10
40
50
52
202
252
5
20
25
2
Example
A + B ndash 3C where
2
1
10
2
1
1
6
3
2
CBA
2
1
27
626
313
3012
2
1
10
3
2
1
1
6
3
2
3CBA
SundermeyerMAR 550
Spring 2013 10
Linear Algebra and Calculus Review
Dot Product Let
be two vectors of length n Then the dot product of the two vectors u and v is defined as
nuuu 21 u nvvv 21 v
n
iiinn vuvuvuvu
12211 vu
A dot product is also an inner product
Example
33)23()72()11()34(27133214 vu
Example (divergence of a vector)
z
w
y
v
x
uwvu
zyx
v
SundermeyerMAR 550
Spring 2013 11
Linear Algebra and Calculus Review
Dot Product and Scalar Product Rules
uv is a scalar
uv = vu
u0 = 0 = 0u
uu = ||u||2
(ku)v = (k)uv = u(kv) for k scalar
u(v plusmn w) = uv plusmn uw
SundermeyerMAR 550
Spring 2013 12
Linear Algebra and Calculus Review
Cross Product Let
be two vectors of length 3 Then the cross product of the two vectors u and v is defined as
)(ˆ)(ˆ)(ˆ
detˆdetˆdetˆ
ˆˆˆ
det
122113312332
21
21
31
31
32
32
321
321
vuvukvuvujvuvui
vv
uuk
vv
uuj
vv
uui
vvv
uuu
kji
vu
Example
kjikji
kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ
713
214
ˆˆˆ
det713214
vu
Example (curl of a vector)
y
u
x
vk
z
u
x
wj
z
v
y
wi
wvuzyx
kji
wvuzyx
ˆˆˆ
ˆˆˆ
detv
321 uuuu 321 vvvv
SundermeyerMAR 550
Spring 2013 13
Linear Algebra and Calculus Review
Cross Product Rules
bull u times v is a vector
bull u times v is orthogonal to both u and v
bull u times 0 = 0 = 0 x u
bull u times u = 0
bull u times v = -(v times u)
bull (ku) times v = k(u times v) = u times (kv) for any scalar k
bull u times (v + w) = (u times v) + (u times w)
bull (v + w) times u = (v times u) + (w times u)
SundermeyerMAR 550
Spring 2013 14
Linear Algebra and Calculus Review
kk
a
a
a
ka
ka
ka
ka
ka
ka
a
a
a
kk
nnnn
AA
2
1
2
1
2
1
2
1
kz
Tjy
Tix
T
kz
Tj
y
Tix
T
z
T
y
T
x
TT
zyxT
ˆˆˆ
ˆˆˆ
NOTE In general for a vector A and a scalar k kA = Ak
However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie
SundermeyerMAR 550
Spring 2013 15
Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added
A + B
nmnmmm
nn
nmm
n
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 16
Linear Algebra and Calculus Review
Example
303020201010
303202101
300020001000
300200100
302010
321
Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers
bullA + B = B + A
bullA + (B + C) = (A + B) + C
bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A
bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0
bullk(A + B) = kA + kB
bull(k+p)A = kA + pA
bull(kp)A = k(pA)
SundermeyerMAR 550
Spring 2013 17
Linear Algebra and Calculus Review
nm
m
T
nmm a
a
a
aaa
aa
aa
aa
33
22
11312
1
3313
2212
n1
13
12
11n1131211
a
a
a
aaaaa
Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar
AT (also denoted Arsquo)
[m x n] [n x m]
Example
303
202
101
302010
321T
SundermeyerMAR 550
Spring 2013 18
Linear Algebra and Calculus Review
Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix
bull(AT)T = A
bull(kA)T = kAT
bull(A + B)T = AT + BT
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 7
Linear Algebra and Calculus Review
yi = mxi + b
y = X b
]12[ ]2[ ]1[
1
1
1
1
3
2
1
2
2
1
nn
b
m
x
x
x
x
y
y
y
y
nn
SundermeyerMAR 550
Spring 2013 8
Linear Algebra and Calculus Review
VectorsAdditionSubtraction Two vectors can be addedsubtracted if and only if they are of the same dimension
nnnn ba
ba
ba
b
b
b
a
a
a
22
11
2
1
2
1
BA
8
7
1
7
71
34
23
52
7
3
2
5
1
4
3
2Example
SundermeyerMAR 550
Spring 2013 9
Linear Algebra and Calculus Review
Scalar Multiplication If k is a scalar and A is a n-dimensional vector then
nn ka
ka
ka
a
a
a
kk
2
1
2
1
A
Example
10
40
50
52
202
252
5
20
25
2
Example
A + B ndash 3C where
2
1
10
2
1
1
6
3
2
CBA
2
1
27
626
313
3012
2
1
10
3
2
1
1
6
3
2
3CBA
SundermeyerMAR 550
Spring 2013 10
Linear Algebra and Calculus Review
Dot Product Let
be two vectors of length n Then the dot product of the two vectors u and v is defined as
nuuu 21 u nvvv 21 v
n
iiinn vuvuvuvu
12211 vu
A dot product is also an inner product
Example
33)23()72()11()34(27133214 vu
Example (divergence of a vector)
z
w
y
v
x
uwvu
zyx
v
SundermeyerMAR 550
Spring 2013 11
Linear Algebra and Calculus Review
Dot Product and Scalar Product Rules
uv is a scalar
uv = vu
u0 = 0 = 0u
uu = ||u||2
(ku)v = (k)uv = u(kv) for k scalar
u(v plusmn w) = uv plusmn uw
SundermeyerMAR 550
Spring 2013 12
Linear Algebra and Calculus Review
Cross Product Let
be two vectors of length 3 Then the cross product of the two vectors u and v is defined as
)(ˆ)(ˆ)(ˆ
detˆdetˆdetˆ
ˆˆˆ
det
122113312332
21
21
31
31
32
32
321
321
vuvukvuvujvuvui
vv
uuk
vv
uuj
vv
uui
vvv
uuu
kji
vu
Example
kjikji
kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ
713
214
ˆˆˆ
det713214
vu
Example (curl of a vector)
y
u
x
vk
z
u
x
wj
z
v
y
wi
wvuzyx
kji
wvuzyx
ˆˆˆ
ˆˆˆ
detv
321 uuuu 321 vvvv
SundermeyerMAR 550
Spring 2013 13
Linear Algebra and Calculus Review
Cross Product Rules
bull u times v is a vector
bull u times v is orthogonal to both u and v
bull u times 0 = 0 = 0 x u
bull u times u = 0
bull u times v = -(v times u)
bull (ku) times v = k(u times v) = u times (kv) for any scalar k
bull u times (v + w) = (u times v) + (u times w)
bull (v + w) times u = (v times u) + (w times u)
SundermeyerMAR 550
Spring 2013 14
Linear Algebra and Calculus Review
kk
a
a
a
ka
ka
ka
ka
ka
ka
a
a
a
kk
nnnn
AA
2
1
2
1
2
1
2
1
kz
Tjy
Tix
T
kz
Tj
y
Tix
T
z
T
y
T
x
TT
zyxT
ˆˆˆ
ˆˆˆ
NOTE In general for a vector A and a scalar k kA = Ak
However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie
SundermeyerMAR 550
Spring 2013 15
Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added
A + B
nmnmmm
nn
nmm
n
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 16
Linear Algebra and Calculus Review
Example
303020201010
303202101
300020001000
300200100
302010
321
Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers
bullA + B = B + A
bullA + (B + C) = (A + B) + C
bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A
bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0
bullk(A + B) = kA + kB
bull(k+p)A = kA + pA
bull(kp)A = k(pA)
SundermeyerMAR 550
Spring 2013 17
Linear Algebra and Calculus Review
nm
m
T
nmm a
a
a
aaa
aa
aa
aa
33
22
11312
1
3313
2212
n1
13
12
11n1131211
a
a
a
aaaaa
Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar
AT (also denoted Arsquo)
[m x n] [n x m]
Example
303
202
101
302010
321T
SundermeyerMAR 550
Spring 2013 18
Linear Algebra and Calculus Review
Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix
bull(AT)T = A
bull(kA)T = kAT
bull(A + B)T = AT + BT
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 8
Linear Algebra and Calculus Review
VectorsAdditionSubtraction Two vectors can be addedsubtracted if and only if they are of the same dimension
nnnn ba
ba
ba
b
b
b
a
a
a
22
11
2
1
2
1
BA
8
7
1
7
71
34
23
52
7
3
2
5
1
4
3
2Example
SundermeyerMAR 550
Spring 2013 9
Linear Algebra and Calculus Review
Scalar Multiplication If k is a scalar and A is a n-dimensional vector then
nn ka
ka
ka
a
a
a
kk
2
1
2
1
A
Example
10
40
50
52
202
252
5
20
25
2
Example
A + B ndash 3C where
2
1
10
2
1
1
6
3
2
CBA
2
1
27
626
313
3012
2
1
10
3
2
1
1
6
3
2
3CBA
SundermeyerMAR 550
Spring 2013 10
Linear Algebra and Calculus Review
Dot Product Let
be two vectors of length n Then the dot product of the two vectors u and v is defined as
nuuu 21 u nvvv 21 v
n
iiinn vuvuvuvu
12211 vu
A dot product is also an inner product
Example
33)23()72()11()34(27133214 vu
Example (divergence of a vector)
z
w
y
v
x
uwvu
zyx
v
SundermeyerMAR 550
Spring 2013 11
Linear Algebra and Calculus Review
Dot Product and Scalar Product Rules
uv is a scalar
uv = vu
u0 = 0 = 0u
uu = ||u||2
(ku)v = (k)uv = u(kv) for k scalar
u(v plusmn w) = uv plusmn uw
SundermeyerMAR 550
Spring 2013 12
Linear Algebra and Calculus Review
Cross Product Let
be two vectors of length 3 Then the cross product of the two vectors u and v is defined as
)(ˆ)(ˆ)(ˆ
detˆdetˆdetˆ
ˆˆˆ
det
122113312332
21
21
31
31
32
32
321
321
vuvukvuvujvuvui
vv
uuk
vv
uuj
vv
uui
vvv
uuu
kji
vu
Example
kjikji
kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ
713
214
ˆˆˆ
det713214
vu
Example (curl of a vector)
y
u
x
vk
z
u
x
wj
z
v
y
wi
wvuzyx
kji
wvuzyx
ˆˆˆ
ˆˆˆ
detv
321 uuuu 321 vvvv
SundermeyerMAR 550
Spring 2013 13
Linear Algebra and Calculus Review
Cross Product Rules
bull u times v is a vector
bull u times v is orthogonal to both u and v
bull u times 0 = 0 = 0 x u
bull u times u = 0
bull u times v = -(v times u)
bull (ku) times v = k(u times v) = u times (kv) for any scalar k
bull u times (v + w) = (u times v) + (u times w)
bull (v + w) times u = (v times u) + (w times u)
SundermeyerMAR 550
Spring 2013 14
Linear Algebra and Calculus Review
kk
a
a
a
ka
ka
ka
ka
ka
ka
a
a
a
kk
nnnn
AA
2
1
2
1
2
1
2
1
kz
Tjy
Tix
T
kz
Tj
y
Tix
T
z
T
y
T
x
TT
zyxT
ˆˆˆ
ˆˆˆ
NOTE In general for a vector A and a scalar k kA = Ak
However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie
SundermeyerMAR 550
Spring 2013 15
Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added
A + B
nmnmmm
nn
nmm
n
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 16
Linear Algebra and Calculus Review
Example
303020201010
303202101
300020001000
300200100
302010
321
Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers
bullA + B = B + A
bullA + (B + C) = (A + B) + C
bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A
bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0
bullk(A + B) = kA + kB
bull(k+p)A = kA + pA
bull(kp)A = k(pA)
SundermeyerMAR 550
Spring 2013 17
Linear Algebra and Calculus Review
nm
m
T
nmm a
a
a
aaa
aa
aa
aa
33
22
11312
1
3313
2212
n1
13
12
11n1131211
a
a
a
aaaaa
Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar
AT (also denoted Arsquo)
[m x n] [n x m]
Example
303
202
101
302010
321T
SundermeyerMAR 550
Spring 2013 18
Linear Algebra and Calculus Review
Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix
bull(AT)T = A
bull(kA)T = kAT
bull(A + B)T = AT + BT
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 9
Linear Algebra and Calculus Review
Scalar Multiplication If k is a scalar and A is a n-dimensional vector then
nn ka
ka
ka
a
a
a
kk
2
1
2
1
A
Example
10
40
50
52
202
252
5
20
25
2
Example
A + B ndash 3C where
2
1
10
2
1
1
6
3
2
CBA
2
1
27
626
313
3012
2
1
10
3
2
1
1
6
3
2
3CBA
SundermeyerMAR 550
Spring 2013 10
Linear Algebra and Calculus Review
Dot Product Let
be two vectors of length n Then the dot product of the two vectors u and v is defined as
nuuu 21 u nvvv 21 v
n
iiinn vuvuvuvu
12211 vu
A dot product is also an inner product
Example
33)23()72()11()34(27133214 vu
Example (divergence of a vector)
z
w
y
v
x
uwvu
zyx
v
SundermeyerMAR 550
Spring 2013 11
Linear Algebra and Calculus Review
Dot Product and Scalar Product Rules
uv is a scalar
uv = vu
u0 = 0 = 0u
uu = ||u||2
(ku)v = (k)uv = u(kv) for k scalar
u(v plusmn w) = uv plusmn uw
SundermeyerMAR 550
Spring 2013 12
Linear Algebra and Calculus Review
Cross Product Let
be two vectors of length 3 Then the cross product of the two vectors u and v is defined as
)(ˆ)(ˆ)(ˆ
detˆdetˆdetˆ
ˆˆˆ
det
122113312332
21
21
31
31
32
32
321
321
vuvukvuvujvuvui
vv
uuk
vv
uuj
vv
uui
vvv
uuu
kji
vu
Example
kjikji
kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ
713
214
ˆˆˆ
det713214
vu
Example (curl of a vector)
y
u
x
vk
z
u
x
wj
z
v
y
wi
wvuzyx
kji
wvuzyx
ˆˆˆ
ˆˆˆ
detv
321 uuuu 321 vvvv
SundermeyerMAR 550
Spring 2013 13
Linear Algebra and Calculus Review
Cross Product Rules
bull u times v is a vector
bull u times v is orthogonal to both u and v
bull u times 0 = 0 = 0 x u
bull u times u = 0
bull u times v = -(v times u)
bull (ku) times v = k(u times v) = u times (kv) for any scalar k
bull u times (v + w) = (u times v) + (u times w)
bull (v + w) times u = (v times u) + (w times u)
SundermeyerMAR 550
Spring 2013 14
Linear Algebra and Calculus Review
kk
a
a
a
ka
ka
ka
ka
ka
ka
a
a
a
kk
nnnn
AA
2
1
2
1
2
1
2
1
kz
Tjy
Tix
T
kz
Tj
y
Tix
T
z
T
y
T
x
TT
zyxT
ˆˆˆ
ˆˆˆ
NOTE In general for a vector A and a scalar k kA = Ak
However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie
SundermeyerMAR 550
Spring 2013 15
Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added
A + B
nmnmmm
nn
nmm
n
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 16
Linear Algebra and Calculus Review
Example
303020201010
303202101
300020001000
300200100
302010
321
Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers
bullA + B = B + A
bullA + (B + C) = (A + B) + C
bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A
bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0
bullk(A + B) = kA + kB
bull(k+p)A = kA + pA
bull(kp)A = k(pA)
SundermeyerMAR 550
Spring 2013 17
Linear Algebra and Calculus Review
nm
m
T
nmm a
a
a
aaa
aa
aa
aa
33
22
11312
1
3313
2212
n1
13
12
11n1131211
a
a
a
aaaaa
Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar
AT (also denoted Arsquo)
[m x n] [n x m]
Example
303
202
101
302010
321T
SundermeyerMAR 550
Spring 2013 18
Linear Algebra and Calculus Review
Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix
bull(AT)T = A
bull(kA)T = kAT
bull(A + B)T = AT + BT
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 10
Linear Algebra and Calculus Review
Dot Product Let
be two vectors of length n Then the dot product of the two vectors u and v is defined as
nuuu 21 u nvvv 21 v
n
iiinn vuvuvuvu
12211 vu
A dot product is also an inner product
Example
33)23()72()11()34(27133214 vu
Example (divergence of a vector)
z
w
y
v
x
uwvu
zyx
v
SundermeyerMAR 550
Spring 2013 11
Linear Algebra and Calculus Review
Dot Product and Scalar Product Rules
uv is a scalar
uv = vu
u0 = 0 = 0u
uu = ||u||2
(ku)v = (k)uv = u(kv) for k scalar
u(v plusmn w) = uv plusmn uw
SundermeyerMAR 550
Spring 2013 12
Linear Algebra and Calculus Review
Cross Product Let
be two vectors of length 3 Then the cross product of the two vectors u and v is defined as
)(ˆ)(ˆ)(ˆ
detˆdetˆdetˆ
ˆˆˆ
det
122113312332
21
21
31
31
32
32
321
321
vuvukvuvujvuvui
vv
uuk
vv
uuj
vv
uui
vvv
uuu
kji
vu
Example
kjikji
kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ
713
214
ˆˆˆ
det713214
vu
Example (curl of a vector)
y
u
x
vk
z
u
x
wj
z
v
y
wi
wvuzyx
kji
wvuzyx
ˆˆˆ
ˆˆˆ
detv
321 uuuu 321 vvvv
SundermeyerMAR 550
Spring 2013 13
Linear Algebra and Calculus Review
Cross Product Rules
bull u times v is a vector
bull u times v is orthogonal to both u and v
bull u times 0 = 0 = 0 x u
bull u times u = 0
bull u times v = -(v times u)
bull (ku) times v = k(u times v) = u times (kv) for any scalar k
bull u times (v + w) = (u times v) + (u times w)
bull (v + w) times u = (v times u) + (w times u)
SundermeyerMAR 550
Spring 2013 14
Linear Algebra and Calculus Review
kk
a
a
a
ka
ka
ka
ka
ka
ka
a
a
a
kk
nnnn
AA
2
1
2
1
2
1
2
1
kz
Tjy
Tix
T
kz
Tj
y
Tix
T
z
T
y
T
x
TT
zyxT
ˆˆˆ
ˆˆˆ
NOTE In general for a vector A and a scalar k kA = Ak
However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie
SundermeyerMAR 550
Spring 2013 15
Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added
A + B
nmnmmm
nn
nmm
n
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 16
Linear Algebra and Calculus Review
Example
303020201010
303202101
300020001000
300200100
302010
321
Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers
bullA + B = B + A
bullA + (B + C) = (A + B) + C
bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A
bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0
bullk(A + B) = kA + kB
bull(k+p)A = kA + pA
bull(kp)A = k(pA)
SundermeyerMAR 550
Spring 2013 17
Linear Algebra and Calculus Review
nm
m
T
nmm a
a
a
aaa
aa
aa
aa
33
22
11312
1
3313
2212
n1
13
12
11n1131211
a
a
a
aaaaa
Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar
AT (also denoted Arsquo)
[m x n] [n x m]
Example
303
202
101
302010
321T
SundermeyerMAR 550
Spring 2013 18
Linear Algebra and Calculus Review
Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix
bull(AT)T = A
bull(kA)T = kAT
bull(A + B)T = AT + BT
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 11
Linear Algebra and Calculus Review
Dot Product and Scalar Product Rules
uv is a scalar
uv = vu
u0 = 0 = 0u
uu = ||u||2
(ku)v = (k)uv = u(kv) for k scalar
u(v plusmn w) = uv plusmn uw
SundermeyerMAR 550
Spring 2013 12
Linear Algebra and Calculus Review
Cross Product Let
be two vectors of length 3 Then the cross product of the two vectors u and v is defined as
)(ˆ)(ˆ)(ˆ
detˆdetˆdetˆ
ˆˆˆ
det
122113312332
21
21
31
31
32
32
321
321
vuvukvuvujvuvui
vv
uuk
vv
uuj
vv
uui
vvv
uuu
kji
vu
Example
kjikji
kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ
713
214
ˆˆˆ
det713214
vu
Example (curl of a vector)
y
u
x
vk
z
u
x
wj
z
v
y
wi
wvuzyx
kji
wvuzyx
ˆˆˆ
ˆˆˆ
detv
321 uuuu 321 vvvv
SundermeyerMAR 550
Spring 2013 13
Linear Algebra and Calculus Review
Cross Product Rules
bull u times v is a vector
bull u times v is orthogonal to both u and v
bull u times 0 = 0 = 0 x u
bull u times u = 0
bull u times v = -(v times u)
bull (ku) times v = k(u times v) = u times (kv) for any scalar k
bull u times (v + w) = (u times v) + (u times w)
bull (v + w) times u = (v times u) + (w times u)
SundermeyerMAR 550
Spring 2013 14
Linear Algebra and Calculus Review
kk
a
a
a
ka
ka
ka
ka
ka
ka
a
a
a
kk
nnnn
AA
2
1
2
1
2
1
2
1
kz
Tjy
Tix
T
kz
Tj
y
Tix
T
z
T
y
T
x
TT
zyxT
ˆˆˆ
ˆˆˆ
NOTE In general for a vector A and a scalar k kA = Ak
However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie
SundermeyerMAR 550
Spring 2013 15
Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added
A + B
nmnmmm
nn
nmm
n
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 16
Linear Algebra and Calculus Review
Example
303020201010
303202101
300020001000
300200100
302010
321
Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers
bullA + B = B + A
bullA + (B + C) = (A + B) + C
bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A
bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0
bullk(A + B) = kA + kB
bull(k+p)A = kA + pA
bull(kp)A = k(pA)
SundermeyerMAR 550
Spring 2013 17
Linear Algebra and Calculus Review
nm
m
T
nmm a
a
a
aaa
aa
aa
aa
33
22
11312
1
3313
2212
n1
13
12
11n1131211
a
a
a
aaaaa
Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar
AT (also denoted Arsquo)
[m x n] [n x m]
Example
303
202
101
302010
321T
SundermeyerMAR 550
Spring 2013 18
Linear Algebra and Calculus Review
Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix
bull(AT)T = A
bull(kA)T = kAT
bull(A + B)T = AT + BT
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 12
Linear Algebra and Calculus Review
Cross Product Let
be two vectors of length 3 Then the cross product of the two vectors u and v is defined as
)(ˆ)(ˆ)(ˆ
detˆdetˆdetˆ
ˆˆˆ
det
122113312332
21
21
31
31
32
32
321
321
vuvukvuvujvuvui
vv
uuk
vv
uuj
vv
uui
vvv
uuu
kji
vu
Example
kjikji
kjiˆ1ˆ22ˆ5)34(ˆ)628(ˆ)27(ˆ
713
214
ˆˆˆ
det713214
vu
Example (curl of a vector)
y
u
x
vk
z
u
x
wj
z
v
y
wi
wvuzyx
kji
wvuzyx
ˆˆˆ
ˆˆˆ
detv
321 uuuu 321 vvvv
SundermeyerMAR 550
Spring 2013 13
Linear Algebra and Calculus Review
Cross Product Rules
bull u times v is a vector
bull u times v is orthogonal to both u and v
bull u times 0 = 0 = 0 x u
bull u times u = 0
bull u times v = -(v times u)
bull (ku) times v = k(u times v) = u times (kv) for any scalar k
bull u times (v + w) = (u times v) + (u times w)
bull (v + w) times u = (v times u) + (w times u)
SundermeyerMAR 550
Spring 2013 14
Linear Algebra and Calculus Review
kk
a
a
a
ka
ka
ka
ka
ka
ka
a
a
a
kk
nnnn
AA
2
1
2
1
2
1
2
1
kz
Tjy
Tix
T
kz
Tj
y
Tix
T
z
T
y
T
x
TT
zyxT
ˆˆˆ
ˆˆˆ
NOTE In general for a vector A and a scalar k kA = Ak
However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie
SundermeyerMAR 550
Spring 2013 15
Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added
A + B
nmnmmm
nn
nmm
n
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 16
Linear Algebra and Calculus Review
Example
303020201010
303202101
300020001000
300200100
302010
321
Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers
bullA + B = B + A
bullA + (B + C) = (A + B) + C
bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A
bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0
bullk(A + B) = kA + kB
bull(k+p)A = kA + pA
bull(kp)A = k(pA)
SundermeyerMAR 550
Spring 2013 17
Linear Algebra and Calculus Review
nm
m
T
nmm a
a
a
aaa
aa
aa
aa
33
22
11312
1
3313
2212
n1
13
12
11n1131211
a
a
a
aaaaa
Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar
AT (also denoted Arsquo)
[m x n] [n x m]
Example
303
202
101
302010
321T
SundermeyerMAR 550
Spring 2013 18
Linear Algebra and Calculus Review
Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix
bull(AT)T = A
bull(kA)T = kAT
bull(A + B)T = AT + BT
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 13
Linear Algebra and Calculus Review
Cross Product Rules
bull u times v is a vector
bull u times v is orthogonal to both u and v
bull u times 0 = 0 = 0 x u
bull u times u = 0
bull u times v = -(v times u)
bull (ku) times v = k(u times v) = u times (kv) for any scalar k
bull u times (v + w) = (u times v) + (u times w)
bull (v + w) times u = (v times u) + (w times u)
SundermeyerMAR 550
Spring 2013 14
Linear Algebra and Calculus Review
kk
a
a
a
ka
ka
ka
ka
ka
ka
a
a
a
kk
nnnn
AA
2
1
2
1
2
1
2
1
kz
Tjy
Tix
T
kz
Tj
y
Tix
T
z
T
y
T
x
TT
zyxT
ˆˆˆ
ˆˆˆ
NOTE In general for a vector A and a scalar k kA = Ak
However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie
SundermeyerMAR 550
Spring 2013 15
Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added
A + B
nmnmmm
nn
nmm
n
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 16
Linear Algebra and Calculus Review
Example
303020201010
303202101
300020001000
300200100
302010
321
Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers
bullA + B = B + A
bullA + (B + C) = (A + B) + C
bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A
bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0
bullk(A + B) = kA + kB
bull(k+p)A = kA + pA
bull(kp)A = k(pA)
SundermeyerMAR 550
Spring 2013 17
Linear Algebra and Calculus Review
nm
m
T
nmm a
a
a
aaa
aa
aa
aa
33
22
11312
1
3313
2212
n1
13
12
11n1131211
a
a
a
aaaaa
Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar
AT (also denoted Arsquo)
[m x n] [n x m]
Example
303
202
101
302010
321T
SundermeyerMAR 550
Spring 2013 18
Linear Algebra and Calculus Review
Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix
bull(AT)T = A
bull(kA)T = kAT
bull(A + B)T = AT + BT
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 14
Linear Algebra and Calculus Review
kk
a
a
a
ka
ka
ka
ka
ka
ka
a
a
a
kk
nnnn
AA
2
1
2
1
2
1
2
1
kz
Tjy
Tix
T
kz
Tj
y
Tix
T
z
T
y
T
x
TT
zyxT
ˆˆˆ
ˆˆˆ
NOTE In general for a vector A and a scalar k kA = Ak
However when computing the gradient of a scalar the scalar product is not commutative because itself is not commutative ie
SundermeyerMAR 550
Spring 2013 15
Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added
A + B
nmnmmm
nn
nmm
n
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 16
Linear Algebra and Calculus Review
Example
303020201010
303202101
300020001000
300200100
302010
321
Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers
bullA + B = B + A
bullA + (B + C) = (A + B) + C
bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A
bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0
bullk(A + B) = kA + kB
bull(k+p)A = kA + pA
bull(kp)A = k(pA)
SundermeyerMAR 550
Spring 2013 17
Linear Algebra and Calculus Review
nm
m
T
nmm a
a
a
aaa
aa
aa
aa
33
22
11312
1
3313
2212
n1
13
12
11n1131211
a
a
a
aaaaa
Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar
AT (also denoted Arsquo)
[m x n] [n x m]
Example
303
202
101
302010
321T
SundermeyerMAR 550
Spring 2013 18
Linear Algebra and Calculus Review
Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix
bull(AT)T = A
bull(kA)T = kAT
bull(A + B)T = AT + BT
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 15
Matrix Algebra Matrix Addition To add two matrices they both must have the same number of rows and the same number of columns The elements of the two matrices are simply added together element by element Matrix subtraction works in the same way except the elements are subtracted rather than added
A + B
nmnmmm
nn
nmm
n
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Linear Algebra and Calculus Review
SundermeyerMAR 550
Spring 2013 16
Linear Algebra and Calculus Review
Example
303020201010
303202101
300020001000
300200100
302010
321
Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers
bullA + B = B + A
bullA + (B + C) = (A + B) + C
bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A
bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0
bullk(A + B) = kA + kB
bull(k+p)A = kA + pA
bull(kp)A = k(pA)
SundermeyerMAR 550
Spring 2013 17
Linear Algebra and Calculus Review
nm
m
T
nmm a
a
a
aaa
aa
aa
aa
33
22
11312
1
3313
2212
n1
13
12
11n1131211
a
a
a
aaaaa
Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar
AT (also denoted Arsquo)
[m x n] [n x m]
Example
303
202
101
302010
321T
SundermeyerMAR 550
Spring 2013 18
Linear Algebra and Calculus Review
Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix
bull(AT)T = A
bull(kA)T = kAT
bull(A + B)T = AT + BT
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 16
Linear Algebra and Calculus Review
Example
303020201010
303202101
300020001000
300200100
302010
321
Matrix Addition Rules Let A B and C denote arbitrary [m x n] matrices where m and n are fixed Let k and p denote arbitrary real numbers
bullA + B = B + A
bullA + (B + C) = (A + B) + C
bullThere is an [m x n] matrix of 0rsquos such that 0 + A = A for each A
bullFor each A there is an [m x n] matrix ndashA such that A + (-A) = 0
bullk(A + B) = kA + kB
bull(k+p)A = kA + pA
bull(kp)A = k(pA)
SundermeyerMAR 550
Spring 2013 17
Linear Algebra and Calculus Review
nm
m
T
nmm a
a
a
aaa
aa
aa
aa
33
22
11312
1
3313
2212
n1
13
12
11n1131211
a
a
a
aaaaa
Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar
AT (also denoted Arsquo)
[m x n] [n x m]
Example
303
202
101
302010
321T
SundermeyerMAR 550
Spring 2013 18
Linear Algebra and Calculus Review
Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix
bull(AT)T = A
bull(kA)T = kAT
bull(A + B)T = AT + BT
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 17
Linear Algebra and Calculus Review
nm
m
T
nmm a
a
a
aaa
aa
aa
aa
33
22
11312
1
3313
2212
n1
13
12
11n1131211
a
a
a
aaaaa
Matrix Transpose Let A and B denote matrices of the same size and let k denote a scalar
AT (also denoted Arsquo)
[m x n] [n x m]
Example
303
202
101
302010
321T
SundermeyerMAR 550
Spring 2013 18
Linear Algebra and Calculus Review
Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix
bull(AT)T = A
bull(kA)T = kAT
bull(A + B)T = AT + BT
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 18
Linear Algebra and Calculus Review
Matrix Transpose RulesIf A is an [m x n] matrix then AT is an [n x m] matrix
bull(AT)T = A
bull(kA)T = kAT
bull(A + B)T = AT + BT
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 19
Linear Algebra and Calculus Review
Matrix Multiplication There are several rules for matrix multiplication The first concerns the multiplication between a matrix and a scalar Here each element in the product matrix is simply the element in the matrix multiplied by the scalar
Scalar Multiplication sA
nmm
n
nmm
n
asas
asas
asas
asasasas
aa
aa
aa
aaaa
s
1
3313
2212
1312111
1
3313
2212
1312111
Example
2412
812
63
23 4
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 20
Linear Algebra and Calculus Review
mnn
m
nmm
n
bb
bb
bb
bbbb
aa
aa
aa
aaaa
1
3313
2212
1312111
1
3313
2212
1312111
Matrix Product AB This is multiplication of a matrix by another matrix Here the number of columns in the first matrix must equal the number of rows in the second matrix eg [m times n][n times m] = [m times m]
= [m times m] matrix whose (ij) entry is the dot product of the ith row of A and the jth column of B
Example (inner (dot) product)
32)63()52()41(
6
5
4
321
[1 x 3][3 x 1] = [1 x 1]
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 21
Linear Algebra and Calculus Review
Example (outer product)
18126
15105
1284
362616
352515
342414
321
6
5
4
Example (general matrix product)
1400140
14014
)3030()2020()1010()330()220()110(
)303()202()101()33()22()11(
303
202
101
302010
321
[2 x 3] [3 x 2] [2 x 2]
[3 x 1][1 x 3] = [3 x 3]
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 22
Linear Algebra and Calculus Review
Matrix Multiplication Rules Assume that k is an arbitrary scalar and that A B and C are matrices of sizes such that the indicated operations can be performed
bullIA = A BI = B
bullA(BC) = (AB)C
bullA(B + C) = AB + AC A(B ndash C) = AB ndash AC
bull(B + C)A = BA + CA (B ndash C)A = BA ndash CA
bullk(AB) = (kA)B = A(kB)
bull(AB)T = BTAT
NOTE In general matrix multiplication is not commutative AB ne BA
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 23
Linear Algebra and Calculus Review
Matrix Division There is no simple division operation per se for matrices This is handled more generally by left and right multiplication by a matrix inverse
Matrix Inverse The inverse of a matrix is defined by the followingAB = I = BA if and only if A is the inverse of B We then write AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE Consider general matrix expression
A X = B A-1 A X = A-1 B
A-1 A X = A-1 B 1 X = A-1 B X = A-1 B
Also note not all matrices are invertible eg the matrix has no inverse
31
00
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab
Matrix + Scalar Addition A + s
sasa
sasa
sasa
sasasasa
s
aa
aa
aa
aaaa
nmm
n
nmm
n
1
3313
2212
1312111
1
3313
2212
1312111
Example
130120110
103102101100
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321
SundermeyerMAR 550
Spring 2013 25
Linear Algebra and Calculus ReviewSpecial Matrix Algebra Rules in Matlab
Matrix times matrix lsquodotrsquo multiplication A B (similar for lsquodotrsquo division A B)
mnnmnm
mn
mnn
m
nmm
n
baba
baba
baba
babababa
bb
bb
bb
bbbb
aa
aa
aa
aaaa
11
33331313
22221212
11313121211111
1
3313
2212
1312111
1
3313
2212
1312111
Example
900040001000
904010
300200100
302010
302010
321