Multiple Linear Regression

Multiple Regression Model A regression model that contains more than

one regressor variable. Multiple Linear Regression Model

A multiple regression model that is a linear function of the unknown parameters b0, b1, b2, and so on.

Examples:

Nonlinear:

iiii

ipipiii

xxY

xxxY

)exp(log 2211010

1,122110

ix

iieY 1

0

3

Intercept α predicts where the regression plane crosses the Y axis

Slope for variable X1 (β1) predicts the change in Y per unit X1 holding X2 constant

The slope for variable X2 (β2) predicts the change in Y per unit X2 holding X1 constant

Estimates for the multiple slope coefficients are derived by minimizing ∑residuals2 to derive this multiple regression model:

The standard error of the regression is based on the ∑residuals2:

A multiple regression analysis involves estimation, testing, and diagnostic procedures designed to fit the multiple regression model to a set of data.

The Method of Least SquaresThe prediction equation

is the line that minimizes SSE, the sum of squares of the deviations of the observed values y from the predicted values

kk xxxyE 22110)(

kk xbxbxbby 22110ˆ

.y

When you perform multiple regression analysis, use a step-by-step approach:1. Obtain the fitted prediction model.2. Use the analysis of variance F test and R 2 to determine how

well the model fits the data.3. Check the t tests for the partial regression coefficients to see

which ones are contributing significant information in the presence of the others.

4. If you choose to compare several different models, use R 2(adj) to compare their effectiveness

5. Use-computer generated residual plots to check for violation

of the regression assumptions.

First time to Use MLR in Excel

Multiple Regression 8

Open MS Excel and Click this icon

. Click on the Excel

options.

Multiple Regression 9

Click Add- ins and Select Analysis

ToolPak from the Add-ins List.

Multiple Regression 10

Click on Excel Add-ins.Then Click Go...

Multiple Regression 11

Check on Analysis

ToolPak, ThenClick OK.

Multiple Regression 12

To check:Go to Data Menu and Go

to Data analysis Tab.

EXAMPLEEXAMPLE

14

The example on the next slide is about the result on the quizzes and summative test of 20 students in TLE subjects.

The Hypothesis: Is there any relationship between the

student’s quiz results to his summative test result @ .05 level of significance.

Multiple Regression 15

Students Quiz 1 Quiz 2 Quiz 3 Summative Teststudent1 73 80 75 152student2 93 88 93 185student3 89 91 90 180student4 96 98 100 196student5 73 66 70 142student6 53 46 55 101student7 69 74 77 149student8 47 56 60 115student9 87 79 90 175student10 79 70 88 164student11 69 70 73 141student12 70 65 74 141student13 93 95 91 184student14 79 80 73 152student15 70 73 78 148student16 93 89 96 192student17 78 75 68 147student18 81 90 93 183student19 88 92 86 177student20 78 83 77 159

Excel Demo

Multiple Regression 17

Multiple Regression 18

Then, P-value (0.9309) > .05 therefore Accept Ho.So, there is a significant relationship between the students’ quiz 1 result and their summative test result

Multiple Regression 19

Then, P-value (0.65706) > .05 therefore Accept Ho.So, there is a significant relationship between the students’ quiz 2 result and their summative test result.

Multiple Regression 20

Then, P-value (0.241742) > .05 therefore Accept Ho.So, there is a significant relationship between the students’ quiz 3 result and their summative test result

Excel Regression Analysis Output Explained: Multiple Regression

These are the “Goodness of Fit” measures. They tell you how well the calculated linear equation fits your data.

1. Multiple R. This is the correlation coefficient. It tells you how strong the linear relationship is. For example, a value of 1 means a perfect positive relationship and a value of zero means no relationship at all. It is the square root of r squared 

2. R square. This is r2, the Coefficient of Determination. It tells you how many points fall on the regression line. for example, 80% means that 80% of the variation of y-values around the mean are explained by the x-values. In other words, 80% of the values fit the model.

3. Adjusted R square. The adjusted R-squared adjust for the number of terms in a model. You’ll want to use this instead of #2 if you have more than one x variable.

4. Standard Error of the regression: An estimate of the standard deviation of the error μ. This is not the same as the standard error in descriptive statistics! The standard error of the regression is the precision that the regression coefficient is measured; if the coefficient is large compared to the standard error, then the coefficient is probably different from 0.

5. Observations. Number of observations in the sample.

SS = Sum of Squares. Regression MS = Regression SS /

Regression degrees of freedom. Residual MS = mean square error

(Residual SS / Residual degrees of freedom).

F: Overall F test for the null hypothesis.

Significance F: The associated P-Value.

Coefficient: Gives you the least squares estimate.

Standard Error: The least squares estimate of the standard

error. T Statistic:

The T Statistic for the null hypothesis vs. the alternate hypothesis.

P Value: Gives you the p-value for the hypothesis test.

Lower 95%: The lower boundary for the confidence

interval. Upper 95%:

The upper boundary for the confidence interval.

Thank you