march 7, 2006lecture 8aslide #1 matrix algebra, or: is this torture really necessary?! what for?...

March 7, 2006 Lecture 8a Slide #1

Matrix Algebra, or:Is this torture really necessary?!

• What for?

– Permits compact, intuitive depiction of regression analysis

– Flexible, in that it can handle any number of independent

variables

– Generally used in statistical presentation, for OLS and

other techniques

• You need to be able to interpret it.


The Basics

• Matrix form:

A =10 5 8−12 1 0 ⎡ ⎣ ⎢

⎤ ⎦ ⎥, where a23 =0

Elements of a matrix can be indentified as:

A=a11 a12 a13

a21 a22 a23

⎡

⎣ ⎢ ⎤

⎦ ⎥


Transpose (“prime”) of a Matrix

A =10 5 8−12 1 0 ⎡ ⎣ ⎢

⎤ ⎦ ⎥

′ A =10 −125 18 0

⎡

⎣

⎢ ⎢

⎤

⎦

⎥ ⎥


Vectors

• Vectors are essentially single rows or columns:

Y =

6−1811

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

′ Y = 6 −1 8 11[ ]


Adding Matrices

Addition works only if matrices have the same dimension:

X1 +X2

=4 −3

2 0

⎡

⎣ ⎢

⎤

⎦ ⎥ +

8 1

4 −5

⎡

⎣ ⎢

⎤

⎦ ⎥

=4+8 −3+1

2+4 0+(−5)

⎡

⎣ ⎢

⎤

⎦ ⎥ =

12 −2

6 −5

⎡

⎣ ⎢

⎤

⎦ ⎥


Multiplication of MatricesDimensions: A(r*q) * B(q*c) = C(r*c),

So the number of columns in the first matrix must matchthe number of rows in the second matrix

2 5

1 0

6 −2

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥ ×

4 2 1

5 7 2

⎡

⎣ ⎢

⎤

⎦ ⎥

=

(2×4)+(5×5) (2×2)+(5×7) (2×1) +(5×2)

(1×4) +(0×5) (1×2)+(0×7) (1×1)+(0×2)

(6×4)+(−2×5) (6×2)+(−2×7) (6×1)+(−2×2)

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

=

33 39 12

4 2 1

14 −2 2

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥


Rules for Matrix Multiplication

• Are matrices conformable? A x B = C (r x q) (q x c) (r x c)• Vector times a matrix: A x B = C (r x c) (c x 1) (r x 1)• Row and column vectors: A x B = C (r x 1) (1 x p) (r x p)

A x B = C (1 x r) (r x 1) (1 x 1) a scalar


Identity Matrices

Square matrices with 1’s on diagonal and 0’s elsewhere:

I4 =

1 0 0 00 1 0 00 0 1 00 0 0 1

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

Identity matrices act like 1’s in familiar algebra:

I x B = B (r x r) (r x c) (r x c)

4 x 4 identity matrix


Matrix Inversion

Acts a bit like division in algebra: any matrix multipliedby its inverse is equal to the identity matrix:

CC−1 =C−1C =I

The text uses the following example:

C =10 43 11 ⎡ ⎣ ⎢

⎤ ⎦ ⎥ soC−1 =

0.112245 −0.04082−0.03061 0.102041 ⎡ ⎣ ⎢

⎤ ⎦ ⎥

Inversion works only for square matrices


Finding the Identity Matrix:An Example

C =3 12 4 ⎡ ⎣ ⎢

⎤ ⎦ ⎥×C−1 =

a cb d ⎡ ⎣ ⎢

⎤ ⎦ ⎥=I

1 00 1 ⎡ ⎣ ⎢

⎤ ⎦ ⎥

2a + 4b = 0 so 2a = -4b and a = -2b3a + b = 1 so 3(-2b) + b = 1, and -5b=1 so b = -1/5

Therefore: a = -2(-1/5) so a = 2/5

3c + d = 0 so d = -3c2c + 4d = 1 so 2c + 4(-3c) = 1 and -10c = 1 so c = -1/10

Therefore d = -3(-1/10) so d = 3/10


Example Continued

Now we can check and see the result of C x C-1:

C−1 =

25

− 110

−15

310

⎡

⎣

⎢ ⎢

⎤

⎦

⎥ ⎥×C =

3 12 4 ⎡ ⎣ ⎢

⎤ ⎦ ⎥

=( 25×3) + (−1

10×2) (2

5×1) + (−1

10×4)

(−15×3) + ( 3

10×2) (−1

5×1) + ( 3

10×4)

⎡

⎣

⎢ ⎢

⎤

⎦

⎥ ⎥

=1 00 1 ⎡ ⎣ ⎢

⎤ ⎦ ⎥


Regression in Matrix Form• Assume a model using n observations, with K-1

Xi (independent) variables

Y (n×1) is a column vector of the observed dependent variable

ˆ Y (n×1) is a column vector of predicted Y values

X (n×K) each column is of observations on an X, first column 1's

B (K ×1) a row vector of regression coefficients (first is b0)

U (n×1) is a column vector of n residual values


Regression in Matrix Form

Y =XB+Uˆ Y =XB

B=( ′ X X)−1 ′ XY

Note: we can’t uniquely define (X’X) if anycolumn in the X matrix is a linear function ofany other column(s) in X. Why is that?


The X’X Matrix

( ′ X X) =

n X1∑ X2∑ X3∑X1∑ X1

2∑ X1X2∑ X1X3∑X2∑ X2X1∑ X2

2∑ X2X3∑X3∑ X3X1∑ X3X2∑ X3

2∑

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

Note that you can obtain the basis for all thenecessary means, variances and covariancesfrom the (X’X) matrix


An Example of Matrix Regression

Using a sample of 7 observations, where X hasElements {X0, X1, X2, X3}

[ ]49.004.006.196.3=B

11.0

58.0

11.0

49.0

41.0

98.0

48.0

10.11

9.58

4.89

2.51

4.41

10.02

6.48

=Y

5281

4371

5431

6911

4621

3271

4541

10

9

5

3

4

11

6

−

⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

−

−

−

−

=

⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

=

⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

= UXY


New Dataset: Scientists

• Sample of AAAS members– US and EU– Collected in 2002– Focus on science, security, GCC…

• Available on the class data page– See the codebook and do file


Application of Multivariate Regression Analysis

• Predict expected temperature change (c4_34_tc), using the following independent variables:– Age (c5_3_age)

– Gender (c5_4_gen)

– Ideology (c4_1_ide))

– Fragile nature (c4_2_nat)

– US or EU (recoded: usa_c)

• Run the model• Evaluate the Output• Draw Initial Conclusions


Break for analysis...

• Feel free to work in groups

• Discuss Analyses

• Take 20 minutes

march 7, 2006lecture 8aslide #1 matrix algebra, or: is this torture really necessary?! what for?...

Documents