Lecture 24: Thurs. Dec. 4

• Extra sum of squares F-tests (10.3)• R-squared statistic (10.4.1)• Residual plots (11.2)• Influential observations (11.3, 11.4.3 – very

brief)• Course summary• More advanced statistics courses

Model Fits• Parallel Regression Lines Model

• Separate Regression Lines Model

• How do we test whether parallel regression lines model is appropriate ( )?

Parameter Estimates Term Estimate Std Error Prob>|t| Lower 95% Upper 95%

Intercept 152.53408 6.282065 <.0001 139.94454 165.12362 I-Manager A 62.178074 5.194105 <.0001 51.768855 72.587293 I-Manager B 8.3438732 5.317727 0.1224 -2.31309 19.000836 Run Size 0.2454321 0.025265 <.0001 0.1947992 0.296065

Parameter Estimates Term Estimate Std Error Prob>|t| Lower 95% Upper 95%

Intercept 149.7477 8.084041 <.0001 133.53317 165.96224 I-Manager A 52.786451 12.352 <.0001 28.011482 77.561419 I-Manager B 35.398214 14.51216 0.0181 6.2905144 64.505913 Run Size 0.2592431 0.036053 <.0001 0.1869294 0.3315568 I-Manager A*Run Size

0.0480219 0.056812 0.4018 -0.065928 0.161972

I-Manager B*Run Size

-0.118178 0.061188 0.0588 -0.240906 0.0045504

0: 540 H

Extra Sum of Squares F-tests• Suppose we want to test whether multiple coefficients are

equal to zero, e.g.,

test• t-tests, either individually or in combination cannot be used

to test such a hypothesis involving more than one parameter.• F-test for joint significance of several terms

lmassIITYPElmasslenergy nebirdnebat 3210},|{

0: 210 H

model full from of Estimate testedbeing betas ofNumber model full of errors squared of Sum

- model reduced of errors squared of Sum

2 statisticF

Extra Sum of Squares F-test

• Under , the F-statistic has an F distribution with number of betas being tested, n-(p+1) degrees of freedom.

• p-value can be found by using Table A.4 or creating a Formula in JMP with probability, F distribution and the putting the value of the F-statistic for F and the appropriate degrees of freedom. This gives the P(F random variable with degrees of freedom < observed F-statistic) which equals 1 – p-value

zero equal testedbeing sbeta' all:0H

Extra Sum of Squares F-test example

• Testing parallel regression lines model (H0, reduced model ) vs. separate regression lines model (full model) in manager example

• Full model:

• Reduced model:

• F-statistic

• p-value: P(F random variable with 2,53 df > 3.29)

Summary of Fit Root Mean Square Error 15.7761 Analysis of Variance Source DF Sum of Squares Mean Square F Ratio

Model 5 67040.713 13408.1 53.8728 Error 53 13190.915 248.9 Prob > F

Analysis of Variance Source DF Sum of Squares Mean Square F Ratio

Model 3 65401.417 21800.5 80.8502 Error 55 14830.210 269.6 Prob > F

29.37761.152

915.13190210.14830

2

F

045.0955.01

Second Example of F-test• For echolocation study, in parallel regression model, test

• Full model:

• Reduced model:

• F-statistic:

• p-value: P(F random variable with 2,16 degrees of freedom > 0.43) = 1-0.342 = 0.658

lmassIISPECIESlmasslenergy nebirdnebat 3210}||{ 0: 210 HSummary of Fit Root Mean Square Error 0.185963 Analysis of Variance Source DF Sum of Squares Mean Square F Ratio

Model 3 29.421483 9.80716 283.5887 Error 16 0.553317 0.03458 Prob > F

Analysis of Variance Source DF Sum of Squares Mean Square F Ratio

Model 1 29.391909 29.3919 907.6384 Error 18 0.582891 0.0324 Prob > F

F

43.0185963.0

2553317.0582891.0

2

F

Manager Example Findings

• The runs supervised by Manager a appear abnormally time consuming. Manager b has high initial fixed setup costs, but the time per unit is the best of the three. Manager c has the lowest fixed costs and per unit production time in between managers a and b.

• Adjustments to marginal analysis via regression only control for possible differences in size among production runs. Other differences might be relevant, e.g., difficulty of production runs. It could be that Manager a supervised most difficult production runs.

Parameter Estimates Term Estimate Std Error Prob>|t| Lower 95% Upper 95%

Intercept 149.7477 8.084041 <.0001 133.53317 165.96224 I-Manager A 52.786451 12.352 <.0001 28.011482 77.561419 I-Manager B 35.398214 14.51216 0.0181 6.2905144 64.505913 Run Size 0.2592431 0.036053 <.0001 0.1869294 0.3315568 I-Manager A*Run Size

0.0480219 0.056812 0.4018 -0.065928 0.161972

I-Manager B*Run Size

-0.118178 0.061188 0.0588 -0.240906 0.0045504

Special Cases of F-test• Multiple Regression Model:

• If we want to test if one equals zero, e.g., , F-test is equivalent to t-test.

• Suppose we want to test , i.e., null hypothesis is that the mean of Y does not depend on any of the explanatory variables.

• JMP automatically computes this test under Analysis of Variance, Prob>F. For separate regression lines model, strong evidence that mean run time does depend on at least one of run size, manager.

ppXXXY 110}|{

0: 10 pH

Analysis of Variance Source DF Sum of Squares Mean Square F Ratio

Model 5 67040.713 13408.1 53.8728 Error 53 13190.915 248.9 Prob > F

C. Total 58 80231.627 <.0001

0: 10 H

The R-Squared Statistic• For separate regression lines model in production time

example,

• Similar interpretation as in simple linear regression. The R-squared statistic is the proportion of the variation in y explained by the multiple regression model

• Total Sum of Squares: • Residual Sum of Squares:

Summary of Fit RSquare 0.83559

squares of sum Totalsquares of sum Residual - squares of sum Total2 R

21

)(

n

i i yy

n

i ippii xxy1

2110 )ˆˆˆ(

Assumptions of Multiple Linear Regression Model

• Assumptions of multiple linear regression:– For each subpopulation ,

• (A-1A)• (A-1B) • (A-1C) The distribution of is normal[Distribution of residuals should not depend on ]

– (A-2) The observations are independent of one another

pxx ,...,1

ppp XXXXY 1101 },...,|{2

1 ),...,|( pXXYVar

pXXY ,...,| 1

pxx ,...,1

Checking/Refining Model

• Tools for checking (A-1A) and (A-1B)– Residual plots versus predicted (fitted) values– Residual plots versus explanatory variables – If model is correct, there should be no pattern in the

residual plots• Tool for checking (A-1C)

– Normal quantile plot• Tool for checking (A-2)

– Residual plot versus time or spatial order of observations

pxx ,,1

Residual Plots for Echolocation Study

• Model:

• Residual vs. predicted plot suggests that variance is not constant and possible nonlinearity.

MassIITYPEMassEnergy nebirdnebat 3210},|{ Residual by Predicted Plot

-5

0

5

10

15

EN

ER

GY

Res

idua

l

0 1020304050ENERGY Predicted

Residual plots for echolocation study

• Model massIITYPEMassEnergy nebirdnebat 3210},|{

Residual by Predicted Plot

-5

0

5

10

15

EN

ER

GY

Res

idua

l

0 1020304050ENERGY Predicted

Coded Residual Plots

• For multiple regression involving nominal variables, a plot of the residuals versus a continuous explanatory variable with codes for the nominal variable is very useful.

-5

0

5

10

15

Res

idua

l EN

ER

GY

0 100 300 500 700MASS

Residual Plots for Transformed Model

• Model:

• Residual Plots for Transformed Model

• Transformed model appears to satisfy Assumptions (1-B) and Assumptions (1-C).

lmassIITYPElmasslenergy nebirdnebat 3210},|{

Residual by Predicted Plot

-0.3

-0.1

0.1

0.3

Log

Ene

rgy

Res

idua

l

.0.51.02.03.04.0Log Energy Predicted

-0.3-0.2-0.1

00.10.20.30.4

Res

idua

l Log

Ene

rgy

1 2 3 4 5 6 7Log Mass

Normal Quantile Plot

• To check Assumption 1-C [populations are normal], we can use a normal quantile plot of the residuals as with simple linear regression.

-0.3-0.2-0.1

00.10.20.30.4

.01 .05.10 .25 .50 .75 .90.95 .99

-2 -1 0 1 2 3Normal Quantile Plot

Dealing with Influential Observations

• By influential observations, we mean one or several observations whose removal causes a different conclusion or course of action.

• Display 11.8 provides a strategy for dealing with suspected influential cases.

Cook’s Distance

• Cook’s distance is a statistic that can be used to flag observations which are influential.

• After fit model, click on red triangle next to Response, Save columns, Cook’s D influence.

• Cook’s distance of close to or larger than 1 indicates a large influence.

Course Summary Cont.

• Techniques: – Methods for comparing two groups– Methods for comparing more than two groups– Simple and multiple linear regression for

predicting a response variable based on explanatory variables and (with a random experiment) finding the causal effect of explanatory variables on a response variable.

Course Summary Cont.• Key messages:

– Scope of inference: randomized experiments vs. observational studies, random samples vs. nonrandom samples. Always use randomized experiments and random samples if possible.

– p-values only assesses whether there is strong evidence against the null hypothesis. They do not provide information about either practical significance. Confidence intervals are needed to assess practical significance.

– When designing a study, choose a sample size that is large enough so that it will be unlikely that the confidence interval will contain both the null hypothesis and a practically significant alternative.

Course Summary Cont.

• Key messages:– Beware of multiple comparisons and data snooping.

Use Tukey-Kramer method or Bonferroni to adjust for multiple comparisons.

– Simple/multiple linear regression is a powerful method for making predictions and understanding causation in a randomized experiment. But beware of extrapolation and making causal statements when the explanatory variables were not randomly assigned.

More Statistics?• Stat 210: Sample Survey Design. Will be offered next

year.• Stat 202: Intermediate Statistics. Offered next fall.• Stat 431: Statistical Inference. Will be offered this

spring (as well as throughout next year).• Stat 430: Probability. Offered this spring.• Stat 500: Applied Regression and Analysis of

Variance. Offered next fall.• Stat 501: Introduction to Nonparametric Methods and

Log-linear models. Offered this spring.