chapter 10: inferential for regression 1

1

Chapter 10: Inferential for Regression

http://jonfwilkins.blogspot.com/2011_08_01_archive.html

2

10.1: Simple Linear Regression10.2: More Detail about Simple Linear Regression

Goals• Describe the simple linear regression model (review – Ch. 2).• Be able to perform the method (with the output from software

packages, Lab 8).• Use diagnostic plots to check the assumptions.• Be able to perform inference on the slope (Confidence interval and

hypothesis test).• Be able to determine if there is an association between the response

and explanatory variables.• Be able to perform a hypothesis test using the correlation coefficient.• Be able to state the similarities and differences between a confidence

interval for a mean response and a prediction interval and in which situations each would be used (if there is time)

3

Conditions for Linear Regression

• We have n (x,y) pairs.• For any fixed x, y ~ N(y, )

• Each yi is independent of the other yj’s.

• y = 0 + 1x

4

Model for Linear Regression

yi = 0 + 1x + i

Data = Fit + Error

5

Linear Regressiony = b0 + b1x• y is an unbiased estimator for y

• b0 is an unbiased estimator for 0

• b1 is an unbiased estimator for 1

6

Linear Regression

i i xy y

1 2xx xi

x x y y SS sb r

SS sx x

b0 = y - b1x�

ei = yi - yi

7

Other SS and df

• Total

dft = n - 1• Model

dfm = 1

8

ANOVA table for Linear Regression

Source df SS MS

Model (Regression) 1 Σ(yi - y)2

Error n – 2 Σ(yi - yi)2

Total n - 1 Σ(yi - y)2

SSM

dfm

SSE

dfe

SST

dft

9

Conditions for Linear Regression• SRS• Observations are independent of each other.• The relationship is linear in the population.• The response, y, is normally distribution around

the population regression line.• The standard deviation of the response is constant.• Important plots:– Scatter plot– Residual plot– Histogram/Normal quantile plot of the

residuals.

10

Residual Plots

11

Example: Linear Regression 1The cetane number is a critical property in specifying the ignition

quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil.

a) Verify the assumptions required for linear regression.b) Determine the equation of the fitted line.c) What is a point estimate of the true average cetane number

whose iodine value is 100?d) Estimate the value of σ.e) What proportion of the observed variation in cetane number

that can be attributed to the iodine value?

12

Example: Linear Regression 1 (cont.)x: 132.0 129.0 120.0 113.2 105.0 92.0 84.0y: 46.0 48.0 51.0 52.1 54.0 52.0 59.0x: 83.2 88.4 59.0 80.0 81.5 71.0 69.2y: 58.7 61.6 64.0 61.4 54.6 58.8 58.0

13

Example: SLR 1 - Scatterplot

14

Example: SLR 1 – Residual Plot

15

Example: SLR 1 – Normality

16






17

Example: SLR 1 – Fitted Line x: 132.0 129.0 120.0 113.2 105.0 92.0 84.0y: 46.0 48.0 51.0 52.1 54.0 52.0 59.0x: 83.2 88.4 59.0 80.0 81.5 71.0 69.2y: 58.7 61.6 64.0 61.4 54.6 58.8 58.0

r = -0.88925 sx = 22.8755 sy = 5.3864yQ = 55.657 xQ = 93.393

18

Example: SLR – fitted line

19






20






21

Example: SLR - sx: 132.0 129.0 120.0 113.2 105.0 92.0 84.0y: 46.0 48.0 51.0 52.1 54.0 52.0 59.0x: 83.2 88.4 59.0 80.0 81.5 71.0 69.2y: 58.7 61.6 64.0 61.4 54.6 58.8 58.0

Analysis of VarianceSource DF Sum of

SquaresMean

SquareF

ValuePr > F

Model 1 298.25443 298.25443 45.35 <.0001Error 12 78.91986 6.57665Corrected Total

13 377.17429

22






23

Example: SLR 1x: 132.0 129.0 120.0 113.2 105.0 92.0 84.0y: 46.0 48.0 51.0 52.1 54.0 52.0 59.0x: 83.2 88.4 59.0 80.0 81.5 71.0 69.2y: 58.7 61.6 64.0 61.4 54.6 58.8 58.0


SquaresMean

SquareF

ValuePr > F


13 377.17429

24

Confidence Interval• Point estimates–b0 is an unbiased estimator for 0

–b1 is an unbiased estimator for 1

• Assumptions– SRS– linearity–Constant standard deviation of residuals–Normality• If y is normal, then both b0 and b1 are

normal• If y is not normal, there is still CLT

25

Standard deviation for b1

(Bonus on HW)

26

Confidence Interval for 1

𝑏1± 𝑡𝑐𝑜𝑙𝑢𝑚𝑛∗ (𝑑𝑓 )𝑆𝐸𝑏1=𝑏1±𝑡𝑐𝑜𝑙𝑢𝑚𝑛

∗ (𝑑𝑓 ) √ 𝑀𝑆𝐸𝑆𝑥𝑥

27

Example: SLR 1 - InferenceThe cetane number is a critical property in specifying the

ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil.

e) What is the 95% Confidence Interval for the population

slope?f) Is the model useful (that is, is there a useful linear

relationship between iodine value and cetane number)?

28

Example: SLR 1x: 132.0 129.0 120.0 113.2 105.0 92.0 84.0y: 46.0 48.0 51.0 52.1 54.0 52.0 59.0x: 83.2 88.4 59.0 80.0 81.5 71.0 69.2y: 58.7 61.6 64.0 61.4 54.6 58.8 58.0


SquaresMean

SquareF

ValuePr > F


13 377.17429b1 = -0.209 Sxx = 6802.77

29

Example: SLR 1 – CI.

We are 95% confidence that the population slope is between -0.277 and -0.141.

30

Example: SLR – fitted line

31

LR Hypothesis Test: SummaryNull hypothesis: H0: 1 = Δ

Test statistic:

Note: A two-sided test with Δ = 0 is called a model utility test

Alternative Hypothesis

P-Value

Upper-tailed Ha: 1 > Δ P(T ≥ t)Lower-tailed Ha: 1 < Δ P(T ≤ t)two-sided Ha: 1 ≠ Δ 2P(T ≥ |t|)

32

Example: SLR 1 - InferenceThe cetane number is a critical property in specifying the


e) What is the 95% Confidence Interval for the population

slope?f) Is the model useful (that is, is there a useful linear

relationship between iodine value and cetane number)?

33

Example: SLR 1 - HT

The data does provide strong support (P = 2.13 x 10-5) to the claim that there is a linear relationship between iodine value and cetane number.

34

ANOVA table for Linear Regression

Source df SS MS

Model (Regression) 1 Σ(yi - y)2

Error n – 2 Σ(yi - yi)2

Total n - 1 Σ(yi - y)2

SSM

dfm

SSE

dfe

SST

dft

35

LR Hypothesis Test: SummaryNull hypothesis: H0: there is an association between

x and yTest statistic: F

P-value: P = P(F > Ftest), dfn = dfm, dfd = dfe

36

Example: LR - InferenceThe cetane number is a critical property in specifying

the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil.

g) Perform the hypothesis test using the F test statistic.h) Perform the hypothesis using the population

correlation coefficient

37

Example: LR – Inference - ANOVA

Parameter EstimatesVariable DF Parameter

EstimateStandard

Errort Value Pr > |t|

Intercept 1 75.21243 2.98363 25.21 <.0001iodine 1 -0.20939 0.03109 -6.73 <.0001


SquaresMean

SquareF

ValuePr > F


13 377.17429

38

Example: LR – Inference (cont)


39

Inference for Correlation: Assumptions

• (x,y) are independent• (x,y) is normal• Linear relationship between x and y• Constant variance for the residuals.

40

LR Hypothesis Test: SummaryNull hypothesis: H0: = 0

Test statistic:

Alternative Hypothesis

P-Value

Upper-tailed Ha: > Δ P(T ≥ t)Lower-tailed Ha: < Δ P(T ≤ t)two-sided Ha: ≠ Δ 2P(T ≥ |t|)

41

Example: LR - InferenceThe cetane number is a critical property in specifying

the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil.

g) Perform the hypothesis test using the F test statistic.h) Perform the hypothesis using the population

correlation coefficient.

42

Example: LR – Inference - ANOVA

Parameter EstimatesVariable DF Parameter

EstimateStandard

Errort Value Pr > |t|

Intercept 1 75.21243 2.98363 25.21 <.0001iodine 1 -0.20939 0.03109 -6.73 <.0001


SquaresMean

SquareF

ValuePr > F


13 377.17429

43

Example: LR - Inference


44

SE hµ�

𝑆𝐸 ��h=√𝑀𝑆𝐸 [ 1𝑛+(𝑥h−𝑥 )2

∑ (𝑥 𝑖−𝑥 )2 ]

45

Example: LR - InferenceThe cetane number is a critical property in specifying the


i) What is the 95% confidence interval for the cetane

number with the iodine value is 100.j) Predict the cetane number for the next sample of

biofuel that contains an iodine value of 100.

46

Example: LR – Inference Analysis of Variance

Source DF Sum ofSquares

MeanSquare

F Value

Pr > F


13 377.17429

� = 54.313Sxx = 6802.77 xQ = 93.393

47

Example: SLR (cont)

We are 95% confident that the population mean cetane number is between 52.754 and 55.872 when the iodine value is 100.

48

SEy

Variance Components of prediction value1) Variance associate with the mean response

2) Variance associated with the observation

49

Example: LR - InferenceThe cetane number is a critical property in specifying the


i) What is the 95% confidence interval for the cetane

number with the iodine value is 100.j) Predict the cetane number for the next sample of

biofuel that contains an iodine value of 100.

50

Example: LR – Inference Analysis of Variance

Source DF Sum ofSquares

MeanSquare

F Value

Pr > F


13 377.17429

� = 54.313Sxx = 6802.77 xQ = 93.393

51

Example: SLR (cont)

We are 95% confident that the next cetane number is between 48.512 and 60.114 when the iodine value is 100.

Mean response: (52.754, 55.872)Prediction interval: (48.512. 60.114)

52

CI for mean responsePrediction interval

53

Example: Confidence/Prediction Band

54

Multiple Regression: Examples 11) A portrait studio operates in cities of medium

size and specializes in portraits of children. They want to open a store in a other similar community, but want to be able to predict sales.

2) So that only students that succeed are accepted into college, the registrar’s office wants to be able to predict GPA from entering high school students.

3) A researcher studied the effects of the charge rate and the temperature on the life of a new type of power cell in a preliminary small-scale experiment.

55

Multiple Regression: Examples 24) An experiment was run to investigate the yield

of tomato plants as a function of the amount of water levels. A series of plots were randomized to different water levels and at the end of the season, the yield of the plants was determined.

5) Fernandez-Juricic et al. (2003) examined the effect of human disturbance on the nesting of house sparrows (Passer domesticus). They counted breeding sparrows per hectare in 18 parks in Madrid, Spain, and also counted the number of people per minute walking through each park (both measurement variables).

chapter 10: inferential for regression 1

Documents

y i y

linear regression b

y pairs

simple linear regression

regression http

population regression

fit error slide

fixed x