curve fitting student notes

CURVE FITTINGStudent NotesENGR 351 Numerical Methods for EngineersSouthern Illinois University CarbondaleCollege of EngineeringInstructor: L.R. Chevalier, Ph.D., P.E.

Applications

Food Available, S

Need to determine parameters for saturation-growth rate model to characterize microbial kinetics

s maxmm

Applications

0 10 20 30

T (o C)

Epilimnion

Thermocline

Hypolimnion

0 10 20 30

Applications

Interpolation of dataWhat is kinematic viscosity at 7.5º C?T (oC) v, 10-2 (cm2/s)

0 1.79234 1.56158 1.387412 1.239616 1.116820 1.010524 0.9186

We want to find the best “fit” of a curve through the data.Here we see :

a) Least squares fit b) Linear interpolation

Can you suggest another?

Material to be Covered in Curve Fitting Linear Regression

Polynomial Regression Multiple Regression General linear least squares Nonlinear regression

Interpolation Lagrange polynomial Coefficients of polynomials (Collocation-

Polynomial Fit) Splines

Specific Study Objectives Understand the fundamental difference

between regression and interpolation and realize why confusing the two could lead to serious problems Understand the derivation of linear least

squares regression and be able to assess the reliability of the fit using graphical and quantitative assessments.

Specific Study Objectives Know how to linearize data by

transformation Understand situations where

polynomial, multiple and nonlinear regression are appropriate Understand the general matrix

formulation of linear least squares Understand that there is one and only

one polynomial of degree n or less that passes exactly through n+1 points

Specific Study Objectives Realize that more accurate results are

obtained if data used for interpolation is centered around and close to the unknown point Recognize the liabilities and risks

associated with extrapolation Understand why spline functions have

utility for data with local areas of abrupt change

Least Squares Regression Simplest is fitting a straight line to a set

of paired observations (x1,y1), (x2, y2).....(xn, yn)

The resulting mathematical expression is y = ao + a1x + e

We will consider the error introduced at each data point to develop a strategy for determining the “best fit” equations

yxyxna

Let’s consider where this comes from.

Determining the Coefficients

x y0 91 72 53 44 35 16 0

012345678910

0 2 4 6 8

S e y a a xri

i o ii

Sum of the Residual Error, Sr

S e y a a xri

i o ii

Note:In this equation yi is the raw data point (dependent data) associated with xi (independent data)

S e y a a xri

i o ii

A line that models this data is:

y = ao + a1x

S e y a a xri

i o ii

S e y a a xri

i o ii

y a a xi o i 1

To determine the values for ao and a1, differentiatewith respect to each coefficient

y a a x

y a a x x

oi o i

ri o i i

Note: we have simplified the summation symbols.What mathematics technique will minimize Sr?

y a a x

y a a x x

oi o i

ri o i i

Setting the derivative equal to zero will minimizing Sr.If this is done, the equations can be expressed as:

y a a x

y x a x a xi o i

i i o i i

y a a x

y x a x a xi o i

i i o i i

We have two simultaneous equations, with two unknowns, ao and a1.

What are these equations? (hint: only place terms with ao and a1 on the LHS of the equations)

What are the final equations for ao and a1?

a nao o

na x a y

x a x a x y

an x y x y

a y a x

i o i i i

i i i i

These first twoequations are calledthe normal equations

ExampleDetermine the linear equation for the following datax y0 91 72 53 44 35 16 0

012345678910

0 2 4 6 8

Strategy

StrategySet up a table

x y xy x2

Sx Sy Sxy Sx2

From these values determine the average values of x and y (x- and y-bar)Calculate a0 and a1

Error f(x)

The most common measure of the “spread” of a sample is the standard deviation about the mean:

Coefficient of determination r2:

r is the correlation coefficient

The following signifies that the line explains 100 percent of the variability of the data:

Sr = 0 r = r2 = 1

If r = r2 = 0, then Sr = St and the fit is invalid.

ExampleDetermine the R2 value for the following datax y0 91 72 53 44 35 16 0

012345678910

0 2 4 6 8

Strategy

StrategyComplete table

x y xy x2 ymodel e2 (y-yavg)2

Sx Sy Sxy Sx2 SSr SSt

Calculate Sr, StDetermine R2

Data 1 Data 2 Data 3 Data 410 8.04 10 9.14 10 7.46 8 6.588 6.95 8 8.14 8 6.77 8 5.7613 7.58 13 8.74 13 12.74 8 7.719 8.81 9 8.77 9 7.11 8 8.8411 8.33 11 9.26 11 7.81 8 8.4714 9.96 14 8.10 14 8.84 8 7.046 7.24 6 6.13 6 6.08 8 5.254 4.26 4 3.10 4 5.39 19 12.5012 10.84 12 9.13 12 8.15 8 5.567 4.82 7 7.26 7 6.42 8 7.915 5.68 5 4.74 5 5.73 8 6.89

Consider the following four sets of data

y = 0.5001x + 3.0001R2 = 0.6665

0 5 10 15

y = 0.5x + 3.0009R2 = 0.6662

0 5 10 15

y = 0.4999x + 3.0017R2 = 0.6667

0 5 10 15 20

y = 0.4997x + 3.0025R2 = 0.6663

0 5 10 15

Linearization of non-linear relationships

Some data is simply ill-suited for linear least squares regression....

or so it appears.

slope = r

intercept = ln P0

P P eort

EXPONENTIALEQUATIONS

slope = rintercept = ln Po

Can you see the similaritywith the equation for a line:

y = ao + a1x

slope = rintercept = ln P0

After taking the natural logof the y-data, perform linearregression.From this regression:

The value of ao will give usln (P0). Hence, P0 = eao

The value of a1 will give us rdirectly.

POWER EQUATIONS

Here we linearizethe equation bytaking the log ofH and Q data.What is the resultingintercept and slope?

(Flow over a weir)

c Hc a H

log log

log loglog log

slope = a

intercept = log c

So how do we getc and a fromperforming regressionon the log H vs log Qdata?From : y = ao + a1x

ao = log c

c = 10ao

a1 = a

slope = a

intercept = log c

c Hc a H

log log

log loglog log

SATURATION-GROWTHRATE EQUATION

s maxmm

slope = Ks/mmax

intercept = 1/mmax

Here, m is the growth rate of a microbial population,mmax is the maximum growth rate, S is the substrate or food concentration, Ks is the substrate concentration at a value of m = mmax/2

ExampleGiven the data below, determine

the coefficients a and b for the equation y=axb

x y1 3.12 263 1104 250

0.5 1 1.5 2 2.5 3 3.5 4 4.50

Raw D...

Strategy

StrategyStart a table. For y=axb you need

a log-log table and graph.x y log x log y

Perform linear regression on the log-log dataBased on y = a0 + a1x, calculate log a = a0 therefore a = 10a0

Residual ErrorLinear: y = ao + a1x

22ioimodelir xaayyyeS

Power: y = axb

imodelir iaxyyyeS

Exponential: y=aexb

222 bx

imodeliriaeyyyeS

ExampleX Y Ymodel

1 0.7 0.82 1.7 1.63 3.3 3.14 7.3 6.25 10.9 12.16 22.7 23.9

y = 0.407e 0.679x

Given the following results, determine Sr.Strategy

StrategyCalculate e2 = (yi – ymodel)2 for each

(x,y) pairDetermine Se2 = Sr

Demonstration with ExcelX Y1 0.72 1.73 3.34 7.35 10.96 22.7

0 2 4 6 8

See: 2012-L4-NonlinearRegressionExample.xlsx

General Comments of Linear RegressionYou should be cognizant of the fact

that there are theoretical aspects of regression that are of practical importance but are beyond the scope of this bookStatistical assumptions are

inherent in the linear least squares procedure

x has a fixed value; it is not random and is measured without errorThe y values are independent

random variable and all have the same varianceThe y values for a given x must be

normally distributed

General Comments of Linear Regression

The regression of y versus x is not the same as x versus yThe error of y versus x is not the

same as x versus y

The regression of y versus x is not the same as x versus yThe error of y versus x is not the

same as x versus yf(x)

x-direction

Polynomial Regression

One of the reasons you were presented with the theory behind linear regression was to allow you the insight behind similar procedures for higher order polynomials y = a0 + a1xmth - degree polynomial

y = a0 + a1x + a2x2 +....amxm + e

S y a a x a x a xr i o i i m im 1 2

2 2......

Based on the sum of the squares of the residuals

S y a a x a x a xr i o i i m im 1 2

2 2......

1. Take the derivative of the above equation with respect to each of the unknown coefficients: i.e. the partial with respect to a2

x y a a x a x a xri i o i i m i

22 .....

2. These equations are set to zero to minimize Sr, i.e. minimize the error.

3. Set all unknowns values on the LHS of the equation.

a x a x a x a x x yo i i i m im

24 2 2 .....

4. This set of normal equations result in m+1 simultaneous equations which can be solved using matrix methods to determine a0, a1, a2......am

ii23i1

i22i1io

yxaxaxax

yaxaxan

Multiple Linear RegressionA useful extension of linear

regression is the case where y is a linear function of two or more variables y = ao + a1x1 + a2x2

We follow the same procedure y = ao + a1x1 + a2x2 + e

Multiple Linear Regression

For two variables, we would solve a 3 x 3 matrixin the following form:

n x xx x x xx x x x

yx yx y

i i i i

2 1 22

[A] and {c}are clearly based on data given for x1, x2 and y to solve for the unknowns in {x}.

InterpolationGeneral formula for an n-th order

polynomial y = a0 + a1x + a2x2 +....amxm

For m+1 data points, there is one, and only one polynomial of order m or less that passes through all pointsExample: y = a0 + a1x

fits between 2 points 1st order

Linear InterpolationTemperature, C Density, kg/ m3

0 999.95 1000.010 999.715 999.120 998.2

How would you approach estimating the density at 17 C?

???999.1 > r > 998.2r

T15 20

Temperature, C Density, kg/ m3

0 999.95 1000.010 999.715 999.120 998.2

Assume a straight line between the known data.Then calculate the slope.

T15201.9992.998r

T15 20

Assuming this linear relationship is constant,the slope is the same between the unknown point and a known point.

T15171.999r

T15 20

Therefore, the slope of one interval will equal the slope of the other interval.

T15171.999

15201.9992.998

Solve for r

T15 20

f x f xf x f x

x xx xo1

998 2 999 120 15

998 220 17

Alternate interpretation

0101 xxaaxf

the intercept is f(x0) the slope is a finite difference approx. of dy/dx

true solution

smaller intervalsprovide a better estimate

true solution1

Alternative approach would be to include a third point and estimate f(x) from a 2nd order polynomial.

true solution

Alternative approach would be to include a third point and estimate f(x) from a 2nd order polynomial.

Lagrange Interpolating Polynomial

f x L x f x

L xx xx x

n i ii

i jjj i

where P designates the “product of”The linear version of this expression is at n=1

f x L x f x

L xx xx x

f x xx x

f x x xx x

n i ii

i jjj i

Linear version: n=1

Your text shows you how to do n=2 (second order).What would third order be?

f x L x f x

L xx xx x

fx x x x x x

x x x x x xf x

n i ii

i jjj i

31 2 3

0 1 0 2 0 30

.......

Note:x0 isnot being subtractedfrom the constantterm x

f x L x f x

L xx xx x

fx x x x x x

x x x x x xf x

n i ii

i jjj i

31 2 3

0 1 0 2 0 30

.......

Note:x0 isnot being subtractedfrom the constantterm x or xi = x0 inthe numeratoror the denominatorj= 0

f x L x f x

L xx xx x

fx x x x x x

x x x x x xf x

x x x x x xx x x x x x

n i ii

i jjj i

31 2 3

0 1 0 2 0 30

1 0 1 2 1 31

.......

f x L x f x

L xx xx x

fx x x x x x

x x x x x xf x

n i ii

i jjj i

31 2 3

0 1 0 2 0 30

1 0 1 2 1 31

.......

f x L x f x

L xx xx x

fx x x x x x

x x x x x xf x

n i ii

i jjj i

31 2 3

0 1 0 2 0 30

1 0 1 2 1 31

2 0 2 1 2 32

......

f x L x f x

L xx xx x

fx x x x x x

x x x x x xf x

n i ii

i jjj i

31 2 3

0 1 0 2 0 30

1 0 1 2 1 31

2 0 2 1 2 32

3 0 3 1 3 23

Example

0 999.95 1000.010 999.715 999.120 998.2

Determine the densityat 17o C using a 2nd order Lagrange Interpolating Polynomial

1000.5

0 5 10 15 20 25

Strategy

StrategyDetermine the number of data

points needed 2nd Order – 3 points 3rd Order – 4 points

Chose the data points for the formulaWrite out the formula without the

dataAdd the dataSolve

Coefficients of an Interpolating Polynomial

y = a0 + a1x + a2x2 +....amxm

HOW CAN WE BE MORE STRAIGHT FORWARD IN GETTING VALUES?

222102

2121101

2020100

xaxaaxf

This is a 2nd order polynomial.

We need three data points.

Plug the value of xi and f(xi)directly into equations.

This gives three simultaneous equationsto solve for a0 , a1 , and a2

ExampleDetermine the densityat 17o C using the method of “Coefficient of Interpolating Polynominals”. Base your solution on a 2nd order polynomial.

1000.5

0 10 20 30

0 999.95 1000.010 999.715 999.120 998.2

Strategy

Strategy Recognize that you are solving for a0, a1 and a2

for the formula y = a0 + a1x + a2x2

Determine the three points you need for the problem

Recognize that the x-value associate with a0 is x0 = 1

Set up the following matrix

xxxxxx

Spline InterpolationOur previous approach was to

derive an nth order polynomial for n+1 data points.An alternative approach is to apply

polynomials to subset of data pointsSuch connecting polynomials are

called spline functionsAdaptation of drafting techniques

Spline interpolation is an adaptation of the drafting technique of using a spline to draw smooth curvesthrough a series of points

Linear Splines

f x f x m x x x x xf x f x m x x x x x

f x f x m x x x x xwhere

mf x f x

n n n n n

0 0 0 0 1

1 1 1 1 2

1 1 1 1

1 cxbxa

Quadratic Spline

2 cxbxa

Quadratic Spline

3 cxbxa

Quadratic Spline

4 cxbxa

Quadratic Spline

1 cxbxa

2 cxbxa

3 cxbxa

4 cxbxa

Quadratic Spline

APPROACHSHOW EXAMPLE

PRESENT THEORY

ExampleA well pumping at 250 gallons per minute has observation wells located at 15, 42, 128, 317 and 433 ft away along a straight line from the well.After three hours of pumping, the following drawdowns in the five wells were observed: 14.6, 10.7, 4.8,1.7 and 0.3 ft respectively. Derive equations of each quadratic spline.

10121416

0 100 200 300 400 500

Distance from well (ft)

Strategy

iiiiii

xfcxbxa

15 14.642 10.7

128 4.8317 1.7433 0.3

10121416

0 100 200 300 400 500

Distance from well (ft)Dr

Strategy Determine how many

segments and equations you have

Determine the two equations that go to each internal point

Determine the equation that goes to each end point

Determine the equations for the first derivatives at each internal point

What are the equations?

10121416

0 100 200 300 400 500

a1x2 + b1x + c1 = ya2x2 + b2x + c2 = ya3x2 + b3x + c3 = y

a4x2 + b4x + c4 = y

10121416

0 100 200 300 400 500

a x b x c f x

a x b x c f xi i i i i i

i i i i i i

21 1 1 1

(42)2 a1 + 42b1 + c1 = 10.7(42)2 a2 + 42 b2 + c2 = 10.7

15 14.642 10.7

128 4.8317 1.7433 0.3

Solution

10121416

0 100 200 300 400 500

16,384a2 + 128b2 + c2 = 4.816,384 a3 + 128 b3 + c3 = 4.8

a x b x c f x

i i i i i i

21 1 1 1

15 14.642 10.7

128 4.8317 1.7433 0.3

Solution

10121416

0 100 200 300 400 500

) 100,489a3 + 317b3 + c3 = 1.7100,489a4 + 317b4 + c4 = 1.7

a x b x c f x

i i i i i i

21 1 1 1

15 14.642 10.7

128 4.8317 1.7433 0.3

Solution

(15)2a1 + 15 b1 + c1 = 14.6 187,489a4 + 433b4 + c4 = 0.3Similarly,the equationsincludethe end points

10121416

0 100 200 300 400 500

15 14.642 10.7

128 4.8317 1.7433 0.3

Solution

2 21 1 1 1a x b a x bi i i i i

The first derivative at the interior knots must be equal.

2a1 (42) + b1 = 2a2 (42) + b2

2a2 (128) + b2 = 2a3 (128) + b3

2a3 (317) + b3 = 2a4 (317) + b4

10121416

0 100 200 300 400 500

15 14.642 10.7

128 4.8317 1.7433 0.3

Solution

Addition the last condition a1 = 0

You should be able to set these equations into a matrix to solve for ai , bi, and ci for i = 1,3

....end of problem

15 14.642 10.7

128 4.8317 1.7433 0.3

Solution

Splines To ensure that the mth derivatives are

continuous at the “knots”, a spline of at least m+1 order must be used 3rd order polynomials or cubic splines

that ensure continuous first and second derivatives are most frequently used in practice Although third and higher derivatives

may be discontinuous when using cubic splines, they usually cannot be detected visually and consequently are ignored.

Splines The derivation of cubic splines is

somewhat involved First illustrate the concepts of spline

interpolation using second order polynomials. These “quadratic splines” have

continuous first derivatives at the “knots” Note: This does not ensure equal

second derivatives at the “knots”

Quadratic Spline

1.The function must be equal at the interior knots. This condition can be represented as:

a x b x c f x

i i i i i i

1 1 1 1

note: we are referencing the same x and f(x)

a x b x c f x

i i i i i i

21 1 1 1

This occurs between i = 2, n

Using the interior knots (n-1) this will provide2n -2 equations.

Quadratic Spline

2. The first and last functions must pass through the end points. This will add two more equations.

a x b x c f x

a x b x c f xn n n n n n

1 0 1 0

We now have 2n - 2 +2 = 2n equations.

How many do we need?

Quadratic Spline

3. The first derivative at the interior knots must be equal.

This provides another n-1 equations for 2n + n-1 =3n -1.

We need 3n

2 21 1 1 1a x b a x bi i i i i

Quadratic Spline

4. Unless we have some additional information regarding the functions or their derivatives, we must make an arbitrary choice in order to successfully compute the constants.5. Assume the second derivative is zero at the first point. The visual interpretation of this condition is that the first two points will be connected by a straight line. a1 = 0

Quadratic Spline

Cubic Splines

Third order polynomialNeed n+1 = 3+1 = 4 intervalsConsequently there are 4n

unknown constants to evaluateWhat are these equations?

f x a x b x c x di i i i i 3 2

Cubic Splines The function values must be equal at

the interior knots (2n -2) The first and last functions must pass

through the end points (2) The first derivatives at the interior knots

must be equal (n-1) The second derivatives at the interior

knots must be equal (n-1) The second derivative at the end knots

are zero (2)

Cubic Splines The function values must be equal at

the interior knots (2n -2) The first and last functions must pass

through the end points (2) The first derivatives at the interior knots

must be equal (n-1) The second derivatives at the interior

knots must be equal (n-1) The second derivative at the end knots

are zero (2)

Previous Exam QuestionGiven the following data, develop the simultaneous equations for a quadratic spline. Express your final answers in matrix form.

x f(x)1 0.504 4.606 1.507 3.00

0 1 2 3 4 5 6 7 8

Interior knots:16a1 + 4b1 + c1 = 4.616a2 + 4b2 + c2 = 4.6

36a2 + 6b2 + c2 = 1.536a3 + 6b3 + c3 = 1.5

End conditionsa1 + b1 + c1 = 0.549a3 + 7b3 + c3 = 3.0

First derivative continuous at interior knots8a1 + b1 = 8a2 + b2

12a2 + b2 = 12a3 + b3

Extra equationa1 =0

x f(x)1 0.504 4.606 1.507 3.00

(4, 4.6)

Interior knots:16a1 + 4b1 + c1 = 4.6 16a2 + 4b2 + c2 = 4.636a2 + 6b2 + c2 = 1.5 36a3 + 6b3 + c3 = 1.5

4 1 0 0 0 0 0 00 0 16 4 1 0 0 00 0 36 6 1 0 0 00 0 0 0 0 36 6 11 1 0 0 0 0 0 00 0 0 0 0 49 7 11 0 8 1 0 0 0 00 0 12 1 0 12 1 0

4.64.61515053 000

bcabcabc

End conditionsa1 + b1 + c1 = 0.5 49a3 + 7b3 + c3 = 3.0First derivative cont. at interior knots8a1 + b1 = 8a2 + b2 12a2 + b2 = 12a3 + b3

Extra equationa1 =0

SPECIAL NOTE

On the surface it may appear that a third order approximation using splines would be inferior to higher order polynomials.

Consider a situation where a spline may perform better:

A generally smooth function undergoes an abrupt change in a region of interest.

The abrupt changeinduces oscillationsin interpolating polynomials.

In contrast,the cubic splineprovides amuch moreacceptableapproximation

Specific Study Objectives

Understand the fundamental difference between regression and interpolation and realize why confusing the two could lead to serious problems Understand the derivation of linear least

squares regression and be able to assess the reliability of the fit using graphical and quantitative assessments.

Specific Study Objectives Know how to linearize data by

transformation Understand situations where

polynomial, multiple and nonlinear regression are appropriate Understand the general matrix

formulation of linear least squares Understand that there is one and only

one polynomial of degree n or less that passes exactly through n+1 points

Specific Study ObjectivesRealize that more accurate results

are obtained if data used for interpolation is centered around and close to the unknown pointRecognize the liabilities and risks

associated with extrapolationUnderstand why spline functions

have utility for data with local areas of abrupt change

curve fitting student notes

Documents

graphing and curve fitting

optimization without curve fitting

the curve-fitting problem

l14 curve fitting

curves and curve fitting

1.0 curve fitting

ch11 curve fitting

curve fitting features in stable32 -...

overview curve fitting - university of...

curve fitting in excel

engineering computation curve fitting: interpolation 1 curve...

curve fitting – general - ncss · curve fitting –...

curve fitting

chapter 16: curve fitting curve fitting

chapter 16: curve fitting curve fitting - origin · pdf...

curve fitting in matlab

polynomial curve fitting - ng

chapter 5 curve fitting

polynomial curve fitting

curve fitting guide - kaleidagraph - scientific graphing,...