chapter 5 - موقع المهندس حماده شعبان ch17.docx · web viewfor these cases, a...

43
July 2013 Least-Square Regression Chapter 17 ى ج ر ي ف عام ل ع ا ف نل ل ة ي ن ا ج م ات وت ن ل ا مة ه سا م ل ا ن’ ع لاغ ت- لا ا ت طا/ خ ا/ ي ا/ و ات4 لاخط م ة وري ر ضها را ت ة ي ص ن رسالة ت260 4444 9 ا/و ي ن رو كي ل- لا د ا رت لي ا تPhysics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economy ان ب عM ش مادة ح م.9 4444 260 info@ eng-hs.com ولة جل م ئ سا م رح وM شً ا ات ج م ن عي ق و م ل ا تeng-hs. com , eng-hs. net

Upload: lyminh

Post on 02-May-2018

233 views

Category:

Documents


8 download

TRANSCRIPT

Page 1: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

Least-Square Regression

Chapter 17

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

Page 2: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

Where substantial error is associated with data, polynomial

interpolation is inappropriate and may yield unsatisfactory results when

used to predict intermediate values. Experimentally data is often of this

type. For example, the following figure (a) shows seven experimentally

derived data points showing significant variability. The data indicates that

higher values of y are associated with higher values of x.

Now, if a sixth-order interpolating polynomial is fitted to this data (fig b), it

will pass exactly through all of the points. However, because of the

variability in the data, the curve oscillates widely in the interval between

the points. In particular, the interpolated values at x = 1.5 and x = 6.5

appear to be well beyond the range suggested by the data.

A more appropriate strategy is to derive an approximating function

that fits the shape . Fig (c) illustrates how a straight line can be used to

generally characterize the trend of the data without passing through any

particular point.

One way to determine the line in figure (c) is to look at the plotted

data and then sketch a “best” line through the points. Such approaches

are not enough because they are arbitrary. That is, unless the points

define a perfect straight line (in which case, interpolation would be

appropriate), different analysis would draw different lines.

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

أناس نظرة أغلبنا يقلق مابنا يهتمون يكادون ال قد

اإلطالق على

Page 3: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

To avoid this, some criterion

must be devised to establish a basis for the fit. One way to do this is to

derive a curve that minimizes the discrepancy between the data points

and the curve. One technique for doing

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

لتحصل تتعب أن إماأو تحب، ما على

على نفسك ستجبر

Page 4: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

this is called least-squares regression.

17.1 Linear Regression

The simple example of a least-squares approximation is fitting a

straight line to a set of paired observations: (x1,y1), (x2,y2), …, (xn,yn).

The mathematical expression for the straight line is

y = a0 + a1x + e

where a0 and a1 are coefficients representing the intercept and the slope,

respectively, and e is the error between the model and the observations,

which can be represented by rearranging the previous equation as

e = y – a0 – a1x

thus, the error is the discrepancy between the true value of y and the

approximate value, a0 + a1x, predicted by the linear equation.

17.1.1 Criteria for the “best” fit

One strategy for fitting a “best” line through the data would be to

minimize the sum of the residual errors for all the available data, as in

∑i=1

n

ei=∑i=1

n

( y i−a0−a1 x i )

where n = total number of points. However, this is an inadequate criterion,

as illustrated by the next figure, which shows the fit of a straight line to

two points. فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالة

اإللكتروني أو 9 4444 260 بالبريد Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economy

شعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرحhs. com , eng-hs. net

Page 5: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

Obviously, the best fit is the line connecting the points.

However, any straight line passing through the midpoint of the connecting

line results in a minimum value of the previous equation equal to zero

because the errors cancel.

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

قدراتكلشخص التي الحدود حسب

عندما سعادتك كميعتقد إنجازات تحقق

تعجر حتما الناسإنك

Page 6: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

Therefore, another logical criterion might be to minimize the sum of the

absolute values of the discrepancies, as in

∑i=1

n

|ei|=∑i=1

n

|y i−ao−a1 xi|

The previous fig (b) demonstrates why this criterion is also inadequate.

For the four points shown, any straight line falling within the dashed lines

will minimize the sum of the absolute values. Thus, this criterion also does

not yield a unique best fit.

A third strategy for fitting a best line is the minimax criterion.

In this technique, the line is chosen that minimizes the maximum distance

that an individual point falls from the line. As shown in previous fig (c), this

strategy is ill-suited for regression because it gives big effect to an outlier,

that is, a single point with a large error.

A strategy that overcomes the shortcomings of the previous

approaches is to minimize the sum of the squares of the residuals between

the measured y and the y calculated with the linear model.

Sr=∑i=1

n

e i2=¿∑

i=1

n

( y i ,measured− y i ,model )2=∑

i=1

n

( y i−a0−a1 xi)2¿

This criterion has a number of advantages, including the fact that it yields

a unique line for a give set of data.

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

إذا إنك الحياة، عجائب من لعلهالقمة، دون هو ما رفضتكل

عليه فإنكستحصل

Page 7: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

17.1.2 Least-Squares fit of a straight line

To determine values of a0 and a1, the previous equation is

differentiated with respect to each coefficient:∂Sr∂a0

=−2∑( yi−a0−a1 xi)

∂Sr∂a1

=−2∑ [( y i−a0−a1 xi)x i ]

Note that we have simplified the summation symbols; unless otherwise

indicated, all summation are from i = 1 to n. Setting these derivatives equal

to zero will result in a minimum Sr.

0=∑ yi−∑ a0−¿∑ a1 xi ¿

0=∑ yi x i−∑ a0 xi−¿∑ a1 x i2¿

Now, realizing that ∑ a0 = na0, we can express the equations as a set of two

simultaneous linear equations with two unknowns (a0 and a1):

na0+(∑ x i )a1=∑ y i (17.4)

(∑ x i )a0+ (∑ x i2 )a1=∑ xi y i

These are called the normal equations. They can be solved simultaneously

a1=n∑ x i y i−∑ x i∑ yin∑ x i

2−(∑ x i)2

This result can then be used in conjunction with Eq. (17.4) to solve forفيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالة

اإللكتروني أو 9 4444 260 بالبريد Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economy

شعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرحhs. com , eng-hs. net

Page 8: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

a0= y−a1 x

where y and x are the means of y and x, respectively.

Example 17.1 Linear Regression

Problem Statement:

Fit a straight line to the x and y values in the first two columns of the next

table

Solution:

The following quantities can be computed

n = 7 ∑ x i y i=119.5 ∑ x i2=140

∑ x i=28 x=287

=4

∑ yi=24 y=247

=3.428571

Using the previous two equations,

a1=7 (119.5 )−28(24)

7 (140 )−(28)2 =0.8392857

a0=3.428571−0.8392857 ( 4 )=0.07142857

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

تخيله يمكن ماوما تحقيقه، يمكنلن تحقيقه يمكن

طريقا نعدم

Page 9: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

Therefore, the least-square fit is

y=0.07142857+0.8392857 x

The line, along with the data, is shown in the first figure (c).

17.1.3 Quantification of Error of Linear Regression

Any line other than the one computed in the previous example results

in a larger sum of the squares of the residuals. Thus, the line is unique and

in terms of our chosen criterion is a “best” line through the points.

A number of additional properties of this fit can be explained by examining

more closely the way in which residuals were computed.

Recall that the sum of the squares is defined as

Sr=∑i=1

n

e i2=∑

i=1

n

( y i−a0−a1x i)2

Notice the similarity between the previous equation and

St=∑ ( y i− y )2

The similarity can be extended further for cases where (1) the spread of

the points around the line is of similar magnitude along the entire range of

the data and (2) the distribution of these points about the line is normal.

It can be demonstrated that if these criteria are met, least-square

regression will provide the best (that is, the most likely) estimates of a0

and a1.

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

وصديقك مرة عدوك احذرانقلب فإن ألفمرةأعلم فهو الصديق

Page 10: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

In addition, if these criteria are met, a “standard deviation” of the

regression line can be determined as

sy / x=√ Srn−2

where sy / x is called the standard error of the estimate. The subscript

notation “y / x” designates that the error is for a predicted value of y

corresponding to a particular value of x.

Also, notice that we now divide by n-2 because two data derived estimates

– a0 and a1 – were used to compute Sr; thus, we have lost two degrees of

freedom.

Another justification for dividing by n-2 is that there is no such thing as the

“spread of data” around a straight line connecting two points..

The standard error of the estimate quantifies the spread of the data.

However, sy / x quantifies the spread around the regression line as shown in

the next figure (b) in contrast to the original standard deviation Sy that

quantified the spread around the mean ( fig (a)).

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

للبر وسيلة أفضلتعد ال أن بالوعد

) بونابارت) نابليون

Page 11: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

The above concepts can be used to quantify the “goodness” of our fit. This

is particularly useful for comparison of several regressions

(next figure). To do this, we return to the original data and determine the

total sum of the squares around the mean for the dependent variable

(in our case, y). This quantity is designated as St. This is the magnitude of

the residual error associated with the dependent variable prior to

regression. After performing the regression, we can compute Sr, the sum

of the squares of the residuals around the regression line.

This characterizes the residual error that remains after the regression.

It is, therefore, sometimes called the unexplained sum of the squares.

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

إلى أسديتجميال إذاتذكره أن فاحذر إنسان

إليك إنسان أسدى وإنتنساه أن فاحذر جميال

Page 12: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

The difference between the two quantifies, St – Sr, quantifies the

improvement or error reduction due to describing the data in terms of a

straight line rather than as an average value.

Because the magnitude of this quantity is scale-dependent, the difference

is normalized to St to yield

r2=St−SrS t

where r2 is called the coefficient of determination and r is the correlation

coefficient (= √r2).

For a perfect fit, Sr = 0 and r = r2 = 1, signifying that the line explains

100 percent of the variability of the data. For r = r2 = 0, Sr = St and the fit

represents no improvement.

An alternative formulation for r that is more convenient for computer

implementations is

r=n∑ x i y i−(∑ x i)(∑ y i)

√n∑ x i2−(∑ xi)

2 √n∑ y i2−(∑ y i)

2

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

وأنتغاضب . . . تكلمحديث أعظم فستقولحياتك طوال عليه تندم

Page 13: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

Example 17.2 Estimate of errors for the linear least-Squares Fit

Problem Statement:

Compute the total standard deviation, the standard error of the estimate,

and the correlation coefficient for the data in Example 17.1

Solution:

The summations are performed and represented in the previous

example’s table. The standard deviation is

Sy=√ S tn−1

=√ 22.71437−1

=1.9457

and the standard error of the estimate is

Sy / x=√ Srn−2

=√ 2.99117−2

=0.7735

Thus, because Sy / x<S y, the linear regression model is efficient.

The extent of the improvement is quantified by

r2=St−SrS t

=22.7143−2.991122.7143

=0.868

or

r=√0.868=0.932

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

كاملة لسنة احتجت ربمامن لكن لكسبصديق،دقيقة في خسارته السهل

Page 14: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

These results indicate that 86.8 percent of the original uncertainty has

been explained by the linear model.

17.1.5 Linearization of Nonlinear Relationships

Linear regression provides a powerful technique for fitting a best line

to data. However, it is predicated on the fact that the relationship

between the dependent and independent variables is linear.

This is not always the case and the first step in any regression analysis

should be to plot and visually inspect the data to know whether a linear

model applies. For example, the next figure shows some data that is

obviously curvilinear. In some cases, techniques such as polynomial

regression, are appropriate. For example, transformations can be used to

express the data in a form that is compatible with linear regression.

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

فقد إنسان الثرثارالسمع نعمة

Page 15: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

One example is the exponential model

y=α1 eβ1 x (17.2)

where α 1 and β1are constants. As shown in the next figure, the equation

represents a nonlinear relationship (for β1≠0) between x and y.

Another example of a nonlinear model is the simple power equation

y=a2xβ2 (17.13)

where α 2 and β2 are constant coefficients. As shown in the previous figure,

the equation ( for β2 ≠ 0 or 1) is nonlinear.

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

السعادة ينشر ممن كنممن ذهبوالتكن أينما

متىذهب وراءه يخلفها

Page 16: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

A third example of a nonlinear model is the saturation-growth-rate

equation

y=α3x

β3+x (17.4)

Where α 3 and β3 are constant coefficients. This model also represents a

nonlinear relationship between y and x, that levels off as x increases.

A simpler alternative is to use mathematical manipulations to

transform the equations into a linear form. Then, simple linear regression

can be employed to fit the equations to data.

Equation (17.2) can be linearized by taking its natural logarithm

ln y=ln α1+β1 x ln e But because ln e = 1,ln y=ln α1+β1 x

Thus, a plot of ln y versus x will yield a straight line with a slope of β1 and

an intercept of ln α 1 (previous fig d).

Equation (17.3) is linearized by taking its base-10 logarithm to givelog y=β2 log x+ log α2

Thus, a plot of y versus log x will yield a straight line with a slope of β2 and

an intercept of log α 2 ( previous fig e).

Equation (17.14) is linearized by inverting it to give

1y=β3

α3

1x+ 1α3

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

Page 17: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

Thus, a plot of 1/ y versus 1/ x will be linear, with a slope of β3/α3 and an

intercept of 1/α 3 (previous fig f).

In their transformed forms, these models can use linear regression to

evaluate the constant coefficients. They could then be transformed back

to their original state and used for predictive purposes.

Example 17.4 illustrates this procedure for Eq. (17.3)

Example 17.4 Linearization of a Power Equation

Problem Statement:

Fit Eq.(17.13) to the data in the next table using a logarithmic

transformation of the data.

Solution:

The next figure (a) is a plot of the original data in its untransformed state.

Figure (b) shows the plot of the transformed data. A linear regression of

the log-transformed data yields the result

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

النــاسمالئكة كل تعد التكن وال أحالمك، فتنهارفتبكي عميـــاء بهم ثقتك

علىسذاجتك يومـــا

Page 18: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

log y=1.75 log x−0.300

Thus, the intercept, log α 2, equals -0.300, and therefore, by taking the

antilogarithm, α 2=10−0.3=0.5. The slope is β2=1.75.

Consequently, the power equation is

y=0.5 x1.75

This curve, as plotted in the next figure (a),

indicates a good fit.

17.1.6 General Comments on Linear Regression

We have focused on the simple derivation and practical use of

equations to fit data.

Some statistical assumptions that are inherent in the linear least-square

procedures are

1. Each x has a fixed value; it is not random and is known without

error.

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

النصيحة قبول اليوم يأبى منفسوف شيئا التكلفه التي

األسف إلىشراء غدا يضطر

Page 19: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

2. The y values are independent random variables and all have the

same variance.

3. The y values for a given x must be normally distributed.

Such assumptions are relevant to the proper derivation and use of

regression. For example, the first assumption means that (1) the x values

must be error-free and (2) the regression of y versus x is not the same as x

versus y.

17.2 Polynomial Regression

Some engineering data, although representing a marked pattern, is

poorly represented by a straight line. For these cases, a curve would be

better suited to fit the data. One method to accomplish this objective is to

use transformations. Another alternative is to fit polynomials to the data

using polynomial regression.

The least-squares procedure can be readily extended to fit the data to

a higher-order polynomial. For example, suppose that we fit a

second-order polynomial or quadratic:

y=a0+a1 x+a2 x2+e

for this case the sum of the squares of the residuals is

Sr=∑i=1

n

( y i−a0−a1 xi−a2 xi2)2

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

فهو فىخطأ يقع منعليه يصر ومن إنسان

شيطان فهو

Page 20: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

Following the procedure of the previous section, we take the derivative of

the previous equation with respect to each of the unknown coefficients of

the polynomial, as in∂Sr∂a0

=−2∑( y i−a0−a1 xi−a2 xi2)

∂Sr∂a1

=−2∑ x i( y i−a0−a1 x i−a2 xi2)

∂Sr∂a2

=−2∑ xi2( y i−a0−a1 x i−a2 xi

2)

These equations can be set equal to zero and rearranged to develop the

following set of normal equations:

(n ) a0+(∑ xi )a1+(∑ x i2 )a2=∑ y i

(∑ x i )a0+ (∑ x i2 )a1+(∑ xi

3 ) a2=∑ x i y i

(∑ xi2 )a0+ (∑ x i

3 )a1+(∑ xi4 )a2=∑ xi

2 y i

where all summations are from i = 1 through n. Note that the above three

equations are linear and have three unknowns: a0 , a1 ,and a2.

The coefficients of the unknowns can be calculated directly from the

observed data.

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

يتساوىفيشدة يطلب ال من الحمق

مع أبدا الغير نصيحة

Page 21: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

For this case, we see that the problem of determining a least-squares

second-order polynomial is equivalent to solving a system of three

simultaneous linear equations.

Example Polynomial Regression

Problem Statement:

Fit a second-order polynomial to the data in the first two columns of the

next table.

Solution:

From the given data,

m = 2 ∑ x i=15 ∑ x i4=979

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

من االستفادة أحسن كلخمسدقائق

خفيففي بشيءأو أذكار أو جيبك

Page 22: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

n = 6 ∑ yi=152.6 ∑ x i y i=585.6

x=2.5 ∑ x i2=55 ∑ x i

2 y i=2488.8

y=25.433 ∑ x i3=225

Therefore, the simultaneous linear equations are

[ 6 15 5515 55 22555 225 979]{a0

a1

a2}={ 152.6

585.62488.8}

Solving these equations through a technique such as Gauss elimination

gives a0 = 2.47857, a1 = 2.35929, and a2 = 1.86071.

Continue:

Therefore, the least-squares quadratic equations for this case is

y = 2.47857 + 2.35929x + 1.86071x2

The standard error of the estimate based on the regression polynomial is

Sy / x=√ Srn−(m+1)

=√ 3.746576−3

=1.12

The coefficient of determination is

r2=St−SrS t

= 2513.39−3.746572513.39

=0.99851

and the correlation coefficient is r = 0.99925.

These results indicate that 99.851 percent of the original uncertainty

has been explained by the model. This result supports the conclusion that

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

تتجه، أي إلى تعرف ال إذاالمطافعلى بك فسينتهي

Page 23: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

the quadratic equation represents an excellent fit, as is also evident from

the next figure.

17.3 Multiple Linear Regression

A useful extension of linear regression is the case where y is a linear

function of two or more independent variables. For example, y might be a

linear function of x1 and x2, as iny=a0+a1 x1+a2 x2+e

Such an equation is particularly useful when fitting experimental data

where the variable being studied is often a function of two other variables.

For this two-dimensional case, the regression “line” becomes a “plane”

(next figure).

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

اليوم يعيشاإلنسانقرارات فيظل

فأين سابقةاتخذهابعد تكون أن تحب

Page 24: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

As with the previous cases, the “best” values of the coefficients are

determined by setting up the sum of the squares of the residuals,

Sr=∑i=1

n

( y i−a0−a1 x1 i−a2 x2 i)2

and differentiating with respect to each of the unknown coefficients.

∂S r∂a0

=−2∑( y i−a0−a1 x1 i−a2 x2 i)

∂S r∂a1

=−2∑ x1i( y i−a0−a1 x1 i−a2 x2 i)

∂S r∂a2

=−2∑ x2i( y i−a0−a1 x1 i−a2 x2 i)

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

تعيشعلى أن تود هليعيش كما هامشالحياة

لديك أن الناسأم أغلبوللعالم إسهاماتألمتك

Page 25: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

The coefficients yielding the minimum sum of the squares of the residuals

are obtained by setting the partial derivatives equal to zero and expressing

the result in matrix form as

[ n ∑ x1 i ∑ x2 i

∑ x1i ∑ x1 i2 ∑ x1 i∑ x2i

∑ x2i ∑ x1 i∑ x2i ∑ x2 i2 ]{a0

a1

a2}={ ∑ y i

∑ x1 i y i∑ x2 i y i

}

Example 17.6 Multiple Linear Regression

Problem Statement:

The following data was calculated from the

equations y = 5 + 4x1 – 3x2:

Use multiple linear regression to fit this data.

Solution:فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالة

اإللكتروني أو 9 4444 260 بالبريد Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economy

شعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرحhs. com , eng-hs. net

تعمل أن هو الجنونبنفس نفساألعمال

نتائج وتتوقع الطريقة) أنشطين ) مختلفة

Page 26: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

The summations required to develop the previous equation are:

The result is

[ 6 16.5 1416.5 76.25 4814 48 54]{a0

a1

a2}={ 54

243.5100 }

Which can be solved using a method such as Gauss elimination for

a0 = 5 ai = 4 a2= -3

which is consistent with the original equation from which the data was

derived.

The foregoing two-dimensional case can be easily extended to m

dimensions, as in

y = a0 + a1x1 + a2x2 + … + amxm + e

where the standard error is formulated as

Sy / x=√ Srn−(m+1)

and the coefficient of determination is computed as in Eq (17.10).

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

: تعبعاجل ثمنين أحد ستدفععاجلة متعة أو نجاح، عاقبتهمؤلم فشل ثمنها مؤقتة

Page 27: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

Although there may be certain cases where a variable is linearly

related to two or more other variables, multiple linear regression has

additional utility in the derivation of power equations of the general form

y=a0 x1a1 x2

a2…. xmam

Such equations are extremely useful when fitting experimental data.

To use multiple linear regression, the equation is transformed by taking its

logarithm to yield.log y=log a0+a1log x1+¿a2 log x2+…+am log xm¿

This transformation is similar in spirit to the one used to fit a power

equations when y is a function of a single variable x.

Problem 17.5

Use least-squares regression to fit a straight line to

x 6 7 11 15 17 21 23 29 29 37 39

y 29 21 29 14 21 15 7 7 13 0 3

Compute the standard error of the estimate and the correlation

coefficient. Plot the data and the regression line. If someone made an

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

الكرسي مثل القلقسيجعلك الهزاز

ولكنه دائما تتحركأي الى يوصلك لن

Page 28: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

05

101520253035

0 10 20 30 40

additional measurement of x = 10, y = 10, would you suspect, that the

measurement was valid or faulty? Justify your conclusion.

Solution:

The results can be summarized as

y=31 . 0589−0 . 78055 x (s y / x=4 . 476306 ;r=0. 901489 )

At x = 10, the best fit equation gives 23.2543. The line and data can be

plotted along with the point (10, 10).

The value of 10 is nearly 3 times the standard error away from the line,

23.2543 – 10 = 13.2543 ≈ 34.476

Thus, we can conclude that the value is probably erroneous.

Problem 17.13

An investigator has reported the data tabulated below for an experiment

to determine the growth rate of bacteria k (per d), as a function of oxygen

concentration c (mg/L). It is known that such data can be modeled by the

following equation:

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

سعادة أكثر ستصبحالحياة أن تشعر حينترغبفي نفسها

ومساندتك دعمك

Page 29: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

k=kmaxc

2

cs+c2

Where cs and kmax are parameters. Use a transformation to linearize this

equation. Then use linear regression to estimate csand kmax and predict the

growth rate at c = 2 mg/L.

C 0.5 0.8 1.5 2.5 4

K 1.1 2.4 5.3 7.6 8.9

Solution:

The equation can be linearized by inverting it to yield

1k=cskmax

1c2 + 1

kmax

Consequently, a plot of 1/k versus 1/c should yield a straight line with an

intercept of 1/kmax and a slope of cs/kmax

c, mg/L k, /d 1/c2 1/k 1/c21/k (1/c2)2

0.5 1.1 4.000000 0.909091 3.636364 16.0000000.8 2.4 1.562500 0.416667 0.651042 2.4414061.5 5.3 0.444444 0.188679 0.083857 0.1975312.5 7.6 0.160000 0.131579 0.021053 0.0256004 8.9 0.062500 0.112360 0.007022 0.003906

Sum 6.229444 1.758375 4.399338 18.66844

Continue:

The slope and the intercept can be computed as

a1=5(4 .399338 )−6 . 229444(1 .758375 )

5(18 .66844 )−(6 . 229444)2 =0 .202489

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

ال قاموسالواثقين

إذا ) . . لكن على يحتوي

Page 30: CHAPTER 5 - موقع المهندس حماده شعبان CH17.docx · Web viewFor these cases, a curve would be better suited to fit the data. One method to accomplish this objective

July 2013

a0=1 .758375

5−0 .202489 6 . 229444

5=0 . 099396

Therefore, kmax = 1/0.099396 = 10.06074 and cs = 10.06074(0.202489) =

2.037189, and the fit is

k=10 .06074 c2

2. 037189+c2

This equation can be plotted together with the data:

024

68

10

0 1 2 3 4 5

The equation can be used to compute

k=10 .06074 (2)2

2. 037189+(2 )2=6 . 666

فيرجى العام للنفع مجانية عن المساهمةالنوتات خطأ باإلبالغ ضرورية مالحظات أوأي نصية تراها برسالةاإللكتروني أو 9 4444 260 بالبريد

Physics I/II, English 123, Statics, Dynamics, Strength, Structure I/II, C++, Java, Data, Algorithms, Numerical, Economyشعبان. حمادة محلولة info@ eng-hs.com 260 4444 9م ومسائل -eng بالموقعين مجاناًشرح

hs. com , eng-hs. net

الذيفي الظالم كلأن يستطيع ال الدنيا

شمعتك يخفيضوء