advanced statistical methods005

8/11/2019 Advanced Statistical Methods005

1/78


2/78


3/78


4/78

11-1 Empirical Models

Many problems in engineering and science involve

exploring the relationships between two or more variables.

Regression analysisis a statistical technique that is

very useful for these types of problems.

For example, in a chemical process, suppose that the

yield of the product is related to the process-operating

temperature.Regression analysis can be used to build a model to

predict yield at a given temperature level.


5/78



6/78


Figure 11-1Scatter Diagram of oxygen purity versushydrocarbon level from Table 11-1.


7/78


Based on the scatter diagram, it is probably reasonable toassume that the mean of the random variable Y is related toxby

the following straight-line relationship:

where the slope and intercept of the line are called regression

coefficients.

The simple linear regression modelis given by

where is the random error term.


8/78


We think of the regression model as an empirical model.Suppose that the mean and variance of are 0 and 2,

respectively, then

The variance of Ygivenxis


9/78


The true regression model is a line of mean values:

where 1can be interpreted as the change in themean of Yfor a unit change inx.

Also, the variability of Y at a particular value ofxis

determined by the error variance, 2.

This implies there is a distribution of Y-values at

eachxand that the variance of this distribution is the

same at eachx.


10/78


Figure 11-2The distribution of Yfor a given value of

xfor the oxygen purity-hydrocarbon data.


11/78

11-2 Simple Linear Regression

The case of simple linear regression considersa single regressoror predictorx and a

dependentor response variableY.

The expected value of Yat each level ofxis arandom variable:

We assume that each observation, Y, can bedescribed by the model


12/78


Suppose that we have n pairs of observations (x1,y1), (x2, y2), , (xn, yn).

Figure 11-3

Deviations of thedata from the

estimated

regression model.


13/78


The method of least squaresis used to

estimate the parameters, 0and 1by minimizing

the sum of the squares of the vertical deviations in

Figure 11-3.

Figure 11-3

Deviations of thedata from the

estimated

regression model.


14/78


Using Equation 11-2,the nobservations in the

sample can be expressed as

The sum of the squares of the deviations of the

observations from the true regression line is


15/78



16/78



17/78


Definition


18/78



19/78


Notation


20/78


21/78


Example 11-1


22/78


Example 11-1

Figure 11-4Scatter

plot of oxygen

purity y versus

hydrocarbon level xand regression

model = 74.20 +

14.97x.


23/78


Example 11-1


24/78


25/78


Estimating

2

The error sum of squares is

It can be shown that the expected value of the

error sum of squares is E(SSE) = (n2)2.


26/78


Estimating

2

An unbiased estimatorof 2 is

where SSEcan be easily computed using

11 3 P ti f th L t S


27/78

11-3 Properties of the Least Squares

Estimators

Slope Properties

Intercept Properties

11 4 H th i T t i Si l Li


28/78

11-4 Hypothesis Tests in Simple Linear

Regression

11-4.1 Use of t-Tests

Suppose we wish to test

An appropriate test statistic would be



29/78


Regression


We would reject the null hypothesis if

The test statistic could also be written as:



30/78


Regression


Suppose we wish to test

An appropriate test statistic would be


31/78

11 4 H pothesis Tests in Simple Linear


32/78


Regression


An important special case of the hypotheses of

Equation 11-18 is

These hypotheses relate to the significance of regression.

Failureto reject H0is equivalent to concluding that there

is no linear relationship betweenxand Y.

11 4 Hypothesis Tests in Simple Linear


33/78


Regression

Figure 11-5The hypothesis H0: 1= 0 is not rejected.



34/78


Regression

Figure 11-6The hypothesis H0: 1= 0 is rejected.



35/78


Regression

Example 11-2



36/78


Regression

11-4.2 Analysis of Variance Approach to TestSignificance of Regression

The analysis of varianceidentity is

Symbolically,



37/78


Regression


If the null hypothesis, H0: 1= 0 is true, the statistic

follows theF1,n-2distribution and we would reject if

f0>f,1,n-2.



38/78


Regression


The quantities, MSRand MSEare called mean squares.

Analysis of variancetable:



39/78


Regression

Example 11-3



40/78


Regression


41/78

11-5 Confidence Intervals

11-5.1 Confidence Intervals on the Slope and Intercept

Definition


42/78


Example 11-4


43/78


11-5.2 Confidence Interval on the Mean Response

Definition


44/78


Example 11-5


45/78


Example 11-5


46/78


Example 11-5


47/78


Example 11-5

Figure 11-7

Scatter diagram ofoxygen purity data

from Example 11-1

with fitted

regression line and

95 percent

confidence limits

on Y|x0.


48/78

11-6 Prediction of New Observations

Ifx0is the value of the regressor variable of interest,

is the point estimator of the new or future value of the

response, Y0.


49/78


Definition


50/78


Example 11-6


51/78


Example 11-6


52/78


Example 11-6

Figure 11-8Scatter

diagram of oxygen

purity data from

Example 11-1 withfitted regression line,

95% prediction limits

(outer lines) , and

95% confidencelimits on Y|x0.

11 A f i


53/78

11-7 Adequacy of the Regression Model

Fitting a regression model requires severalassumptions.

1. Errors are uncorrelated random variables with

mean zero;

2. Errors have constant variance; and,

3. Errors be normally distributed.

The analyst should always consider the validity of

these assumptions to be doubtful and conduct

analyses to examine the adequacy of the model

11 7 Ad f h R i M d l


54/78


11-7.1 Residual Analysis

The residuals from a regression model are ei=yi- i, whereyi

is an actual observation and iis the corresponding fitted value

from the regression model.

Analysis of the residuals is frequently helpful in checking the

assumption that the errors are approximately normally distributed

with constant variance, and in determining whether additional

terms in the model would be useful.

11 7 Ad f h R i M d l


55/78


11-7.1 Residual Analysis

Figure 11-9Patterns

for residual plots. (a)

satisfactory, (b)

funnel, (c) doublebow, (d) nonlinear.

[Adapted from

Montgomery, Peck,

and Vining (2001).]

11 7 Ad f th R i M d l


56/78


Example 11-7



57/78


Example 11-7



58/78


Example 11-7

Figure 11-10Normal

probability plot of

residuals, Example

11-7.



59/78


Example 11-7

Figure 11-11Plot of

residuals versus

predicted oxygen

purity,

, Example

11-7.



60/78


11-7.2 Coefficient of Determination (R2)

The quantity

is called the coefficient of determinationand is often

used to judge the adequacy of a regression model.

0 R2

1;

We often refer (loosely) to R2as the amount of

variability in the data explained or accounted for by the

regression model.



61/78


11-7.2 Coefficient of Determination (R2)

For the oxygen purity regression model,

R2= SSR/SST

= 152.13/173.38

= 0.877

Thus, the model accounts for 87.7% of the

variability in the data.

11 8 C l ti


62/78

11-8 Correlation


63/78

11 8 Correlation


64/78

11-8 Correlation

It is often useful to test the hypotheses

The appropriate test statistic for these hypotheses is

Reject H0if |t0| > t/2,n-2.

11 8 Correlation


65/78

11-8 Correlation

The test procedure for the hypothesis

where 00 is somewhat more complicated. In this

case, the appropriate test statistic is

Reject H0if |z0| > z/2.

11 8 Correlation


66/78

11-8 Correlation

The approximate 100(1- )% confidence interval is

11 8 Correlation


67/78

11-8 Correlation

Example 11-8

11 8 Correlation


68/78

11-8 Correlation

Figure 11-13Scatter plot of wire bond strength versus wire

length, Example 11-8.

11 8 Correlation


69/78

11-8 Correlation

Minitab Output for Example 11-8

11 8 Correlation


70/78

11-8 Correlation

Example 11-8 (continued)

11 8 Correlation


71/78

11-8 Correlation


11 8 Correlation


72/78

11-8 Correlation


11-9 Transformation and Logistic Regression


73/78

11-9 Transformation and Logistic Regression

11-9 Transformation and Logistic


74/78

Regression

Example 11-9

Table 11-5Observed Values

and Regressor Variable for

Example 11-9.

iy

ix



75/78

Regression

Example 11-9 (Continued)



76/78

Regression

Example 11-9 (Continued)


77/78


78/78

advanced statistical methods005

Documents