Chapter 5

Curve fitting

Assignment (please use excell)

Given the data below, use least squares regression to fit a) a straight line b) a power equation c) a saturation-growth-rate equation and d) a parabola. Find the r2 value and justify which is the best equation to represent the data.

Curve fitting

2

x 5 10 15 20 25 30 35 40 45 50

y 17 24 31 33 37 37 40 40 42 51

Interpolation (pg 488) • Polynomial Interpolation is a common method to determine

intermediate values between data points.

• General equation for nth order polynomial is:

• Polynomial interpolation consists of determining the unique nth-order

polynomial that fits n+1 data point.

• For n+1 data points, there is only one polynomial of order n that

passes through all the points.

• For example, there is only one straight line (first-order polynomial)

that connects two points (Fig. 18.1a) and only one parabola connects

a set of three points (Fig.18.1b).

n

nxaxaxaaxf .....)( 2

210

3

Curve fitting

----- 18.1

• Two popular alternative mathematical formats used to express an interpolating polynomial:

a. Newton polynomial

b. Lagrange polynomial

18.1 Newton’s Divided-Difference Interpolating Polynomials

• The most popular and useful in polynomial forms.

• Consists of the first- and second-order versions.

18.1.1 Linear Interpolation

• The simplest form of interpolation is to connect two data points with a straight line.

Curve fitting

4

Figure 18.2: graphical depiction of linear interpolation.

Curve fitting

5

• This linear interpolation technique can be depicted graphically as shown in

fig 18.2, in which, the similar triangles can be rearranged to yield a linear-

interpolation formula;

- f1(x) is refer to first order interpolation polynomial

-The term is a finite-divided-difference approximation of the

first derivative.

• In general, the smaller the interval between the data points, the better the

approximation.

)()()(

)()( 0

01

0101 xx

xx

xfxfxfxf

)()(

01

01

xx

xfxf

6

Curve fitting

----- 18.2

Example 7

Estimate the natural logarithm of 2 using linear interpolation. First, perform the computation by interpolating between ln 1 = 0 and ln 6 = 1.791759. Then, repeat the procedure, but use a smaller interval from ln 1 to ln 4 (1.386294). Note that the true value of ln 2 is 0.6931472.

Solution:

• By using equation (18.2), a linear interpolation for ln 2 from xo = 1 to x1 = 6 to give;

Curve fitting

7

)()()(

)()( 0

01

0101 xx

xx

xfxfxfxf

3583519.0)12(16

0791759.10)2(1

f

x f(x)

x0=1 f(x0) =0

x=2 f1(x)=?

x1=6 f(x1)=1.791759

%3.48100*6931472.0

3583519.06931472.0

t

Curve fitting

8

• Then, using the smaller interval from xo = 1 to x1 = 4 yields;

• Thus, using the shorter interval reduces the percent relative error to t = 33.3%.

• Both interpolations are shown in Fig.18.3, along with true function.

x f(x)

x0=1 f(x0) =0

x=2 f1(x)=?

x1=4 f(x1)=1.386294

%3.33100*6931472.0

4620981.06931472.0

4620981.0)12(14

0386294.10)2(1

t

f

Figure 18.3: Comparison of two linear interpolations with different intervals.

Curve fitting

9

Curve fitting

10

QUIZ 4

Estimate the logarithm of 5 to the base 10 (log5) using linear

interpolation.

a) Interpolate between log 4=0.60206 and log6=0.7781513

b) Interpolate between log4.5=0.6532125 and log

5.5=0.7403627

For each of the interpolations, compute the percent relative error

based on the true value

18.1.2 Quadratic Interpolation

• The error in example 18.1 (linear interpolation) resulted from approximation a curve with a straight line.

• With 3 data points, the estimation can be improved with a second-Order Polynomial (quadratic polynomial or parabola). Thus;

----- (18.3)

• Although equation (18.3) seem to differ from the general polynomial (equation 18.1), the two equations are equivalent, by multiplying the terms in equation (18.3) to yield;

f2(x) = bo + b1x – b1xo + b2x2 + b2xox1 – b2xxo – b2xx1

Curve fitting

11

))(()()( 1020102 xxxxbxxbbxf

or in collecting terms,

f2(x) = ao + a1x + a2x2

where;

ao = bo – b1xo + b2xox1

a1 = b1 – b2xo – b2x1

a2 = b2

• Thus, equations 18.1 and 18.3 are alternative, equivalent formulations of the unique second-order polynomial joining three points.

• To determine the values of coefficient (bo, b1, and, b2), rearrange and use Eq 18.3, substitute Eq 18.4 into 18.3, and substitute Eq 18.4 and 18.5 into Eq 18.3 to yield Eq 18.6.

Curve fitting

12

02

01

01

12

12

2

0

01

1

00

)()()()(

)()(

0)(

xx

xx

xfxf

xx

xfxf

b

xx

xfxfb

xfb

----- 18.4

----- 18.5

----- 18.6

Example 8 Fit a second-order polynomial to the three points used

in linear interpolation example

xo = 1 f(x0) = 0

x1 = 4 f(x1) =1.386294

x2 = 6 f(x2) =1.791759

Use the polynomial to evaluate ln 2

Solution :

0518731.016

4620981.046

386294.1791759.1)()()()(

4620981.014

0386294.1)()(

0)(

02

01

01

12

12

2

0

01

1

00

xx

xx

xfxf

xx

xfxf

b

xx

xfxfb

xfb

13

Curve fitting

14

Curve fitting

Substituting these values into equation 18.3 yields the quadratic

formula:

f2(x) = bo + b1(x – xo) + b2(x – xo)(x – x1)

f2(x) = 0 + 0.462098(x – 1) – 0.0518731(x – 1)(x – 4)

which can be evaluated at x = 2 for;

f2(2) = 0.5658444

which represents a relative error of t = 18.4%.

• Thus, the curvature introduced by the quadratic formula

(Fig.18.4) improves the interpolation compared with the result

obtained using straight lines in example 18.1, Fig.18.3.

2 2 2

Figure 18.4: The use of quadratic formula to estimate ln 2 improves the interpolation.

Curve fitting

15

Example 9

Given the data, calculate f(3.4) using Newton’s polynomials of order 1 to 2.

Solution

Curve fitting

16

x 1 2 2.5 3 4 5

f(x) 1 5 7 8 2 1

1)( 5

7)(5.2

2)(4

8)(3

22

22

11

00

xfx

or

xfx

xfx

xfx

3.4

Use equations 18.4-18.6 to find b0, b1 and b2.

1 st order need f [x0, x1] )

From eq 18.4;

From eq 18.5;

From equation 18.3,

Curve fitting

17

6.5)34.3)(6(8)4.3(

)3)(6(8)(

)()(

634

82)()(

8)(

1

1

0101

01

01

1

00

f

xxf

xxbbxf

xx

xfxfb

xfb

2nd order (need f [x0, x1, x2] )

From equation 18.6;

From eq 18.3 for 2nd order

Curve fitting

18

98.6)4.3(

)44.3)(34.3)(75.5()34.3)(6(8)4.3(

)4)(3)(75.5()3)(6(8)(

))(()()(

75.535.2

)6(45.2

27)()()()(

2

2

2

1020102

02

01

01

12

12

2

f

f

xxxxf

xxxxbxxbbxf

xx

xx

xfxf

xx

xfxf

b

Curve fitting

19

QUIZ 5

x 1 2 3 5 6

f(x) 4.75 4 5.25 19.75 36

Given the data, calculate f(4) using Newton’s

polynomial of order 1 to 2

18.1.3 General Form of Newton’s Interpolating Polynomials

• The nth order polynomial is:

fn(x) = bo + b1(x – xo) + b2(x – xo)(x – x1)+ . . .

+ bn(x – xo)(x – x1). . . (x – xn-1) ----- (18.7)

• For n’th-order polynomial, n + 1 data points are required:

[ xo, f(xo) ], [ x1,f(x1) ] . . . . [ xn, f(xn) ]

• Then, we used these data points and following equation’s to evaluate the

coefficients

20

Curve fitting

],,,,[

],,[

],[

)(

011

0122

011

00

xxxxfb

xxxfb

xxfb

xfb

nnn

----- 18.8

----- 18.9

----- 18.10

----- 18.11

• Where the bracketed [ ] function evaluations are finite divided differences;

• For example, the first finite divided difference is represented generally as;

• Similarly, the n’th finite divided difference is

Curve fitting

21

ki

kjji

kji

ji

ji

ji

xx

xxfxxfxxxf

xx

xfxfxxf

],[],[],,[

difference divided finite Second

)()(],[ ----- 18.12

----- 18.13

0

02111

011

],[],[],,[

xx

xxxfxxxfxxxxf

n

nnnn

nn

----- 18.14

• These differences can be used to evaluate the coefficients in

equations (18.8) through (18.11), which can then substituted into

equation (18.7) to yield the interpolating polynomial, called as

Newton’s divided-difference interpolating polynomial;

01110

012100100

,....,,)...)((....

,,))((,)()()(

xxxfxxxxxx

xxxfxxxxxxfxxxfxf

nnn

n

22

Curve fitting

----- 18.15

Example 10

From example 18.2, data points at x0=1, x1=4 and x2=6 were used to estimate ln 2 with a parabola. Now adding a fourth point (x3=5; f(x3)=1.609438), Estimate ln 2 with a third order Newton’s interpolating Polynomial.

Solution

The third-order polynomial with n = 3 is,

f3(x) = bo + b1(x – xo) + b2(x – xo)(x – x1) + b3(x – xo)(x – x1)(x-x2)

The first divided differences are (use eq 18.12):

Curve fitting

23

4620981.014

0386294.1)()(],[

01

0101

xx

xfxfxxf

x0=1 x1=4 x2=6 x3=5

f(x0)=0 f(x1)=1.386294 f(x2)=1.791759 f(x3)=1.609438

The second divided differences (use equation 18.13):

Curve fitting

24

2027326.046

386294.1791759.1)()(],[

12

1212

xx

xfxfxxf

1823216.065

386294.1609438.1)()(],[

23

2323

xx

xfxfxxf

02041100.045

2027326.01823216.0],[],[],,[

13

1223

123

xx

xxfxxfxxxf

05187311.016

4620981.02027326.0],[],[],,[

02

0112

012

xx

xxfxxfxxxf

The third divided differences:

Finally, using eqs 18.8-18.11:

Insert all values into eqs 18.7;

, which represents a relative error of t = 9.3%.

Curve fitting

25

007865529.0],,,[

15

)05187311.0(02041100.0],,[],,[],,,[

0123

03

012123

0123

xxxxf

xx

xxxfxxxfxxxxf

b0 f (x0) 0

b1 f x1,x0 0.4620981

b2 f x2,x1,x0 0.05187311

b3 f x3,x2,x1,x0 0.007865529

)6)(4)(1(007865529.0

)4)(1(05187311.0)1(4620981.00)(3

xxx

xxxxf

6287686.0)2(3 f

QUIZ 6

Calculate f (4) with a third and fourth order Newton’s interpolating polynomial.

Curve fitting

26

x 1 2 3 5 6

f(x) 4.75 4 5.25 19.75 36

Solution

1st FDD;

Curve fitting

27

25.735

25.575.19)()(],[

01

01

01

xx

xfxfxxf

25.552

75.194)()(],[

12

12

12

xx

xfxfxxf

826

436)()(],[

23

23

23

xx

xfxfxxf

2nd FDD

3rd FDD

Curve fitting

28

232

25.725.5],[],[],,[

02

0112

012

xx

xxfxxfxxxf

75.256

25.58],[],[],,[

13

1223

123

xx

xxfxxfxxxf

25.0],,,[

36

275.2],,[],,[],,,[

0123

03

012123

0123

xxxxf

xx

xxxfxxxfxxxxf

Finally,

f3(x) = bo + b1(x – xo) + b2(x – xo)(x – x1) + b3(x – xo)(x – x1)(x-x2)

Curve fitting

29

25.0,,

2,,

25.7,

25.5)(

012,33

0122

011

00

xxxxfb

xxxfb

xxfb

xfb

)2)(5)(3(25.0)5)(3(2)3(25.725.5)(3

xxxxxxxf

10)4(3

f

Chapter 5

Curve fitting

Linear regression (exponential model,

power equation and saturation growth

rate equation)

Polynomial Regression

Polynomial Interpolation (Linear

interpolation, Quadratic

Interpolation, Newton DD)

Lagrange Interpolation

Spline Interpolation

18.2 Lagrange Interpolating Polynomials

• The Lagrange interpolating polynomial is simply a reformulation of

the Newton’s polynomial that avoids the computation of divided

differences:

• Where designates the “product of”. For example the linear version

(n =1) is

n

ijj

ji

j

i

n

iiin

xx

xxxL

xfxLxf

0

0

)( where

)()()(

)()()( 1

01

00

10

11 xf

xx

xxxf

xx

xxxf

31

Curve fitting

----- 18.20

----- 18.21

----- 18.22

• And the second-order version (n=2) is

• For n=3

Curve fitting

32

)(

)()()(

2

1202

10

1

2101

200

2010

212

xfxxxx

xxxx

xfxxxx

xxxxxf

xxxx

xxxxxf

----- 18.23

)()(

)()()(

3

231303

2102

321202

310

1

312101

3200

302010

3212

xfxxxxxx

xxxxxxxf

xxxxxx

xxxxxx

xfxxxxxx

xxxxxxxf

xxxxxx

xxxxxxxf

33

Curve fitting

Example 11

Use a Lagrange interpolating polynomial of the first and second order to evaluate ln 2 based on the data given.

x0 = 1 f(x0) = 0

x1 = 4 f(x1) = 1.386294

x2 = 6 f(x2) = 1.791760

Solution:

First-order polynomial at x = 2, use eq. 18.22;

Second-order polynomial at x = 2, use eq 18.23;

4620981.0)386294.1(14

12)0(

41

42)2(1

f

5658444.0)791760.1()46)(16(

)42)(12(

)386294.1()64)(14(

)62)(12()0(

)61)(41(

)62)(42()(2

xf

Quiz

Quiz 7

Given the data, calculate f(4) using the Lagrange

polynomials of order 1 to 2.

x 1 2 3 5 6

f(x) 4.75 4 5.25 19.75 36

18.6 Spline Interpolation

• In the previous sections, the nth order polynomials were used to interpolate between n+1 data points. For example we can derive a perfect seventh-order polynomial for eight points (Figure 18.14 a,b,c)

• However, there are cases where these functions can lead to erroneous results. An alternative approach is to apply lower-order polynomials to subsets of data points. Such connecting polynomials are called spline functions.

• There are three types of spline interpolations;

1. Linear spline/1st order spline

2. Quadratic spline

3. Cubic spline

Curve fitting

35

Figure 18.14: Illustration shows the splines are superior to higher order interpolating polynomials.

Curve fitting

36

18.6.1 Linear Splines

• The simplest connection between two points is a straight line.

• The first order splines for a group of ordered data points can be

defined as a set of linear functions

• This equations can be used to evaluated the function at any point

between x0 and xn by first locating the interval within which the

point lies.

• Then the appropriate equation is used to determine the function

value within the interval. The method identical to linear

interpolation.

ii

ii

i

nnnnn

xx

xfxfm

xxxxxmxfxf

xxxxxmxfxf

xxxxxmxfxf

1

1

1111

21111

10000

)()( where

)( where)()()(

)( where)()()(

)( where)()()(

37

Curve fitting

----- 18.27

Disadvantages of 1st Order Splines (see figure 18.16)

1. Not smooth

2. At the data points where two splines meet (called a knot) the slope changes abruptly

These disadvantages can be overcome by using higher-order polynomial splines that ensure smoothness at the knot.

Curve fitting

38

Example 12a: First-order Splines

Fit the data in the table below with first-order splines. Evaluate the function at x = 5.

Solution:

Determine the slopes between points for every intervals. For interval x = 4.5 to x = 7, using eq 18.27:

Curve fitting

39

x f(x)

x0 = 3.0 2.5

x1 = 4.5 1.0

x2 = 7.0 2.5

x3 = 9.0 0.5

5

ii

ii

ixx

xfxfm

1

1)()(

Therefore, at x = 5;

Curve fitting

40

60.05.47

15.2

m

12

12

1

)()(

xx

xfxfm

3.1)5.45(6.00.1)5(

)5.4(6.00.1)()()(111

f

xxxmxfxf

18.6.2 Quadratic splines

Curve fitting

41

• Quadratic splines is use to derive a 2nd order polynomial for each interval between data points.

• General polynomial for each interval can be represented as;

• For n+1 data points (i=0,1,2…,n) there are n intervals and 3n unknown constants (the a’s, b’s and c’s), hence 3 equations are required.

• Figure 18.17 shows the notation used to derive quadratic splines.

iiii cxbxaxf 2)( ----- 18.28

Figure 18.17 : 3 intervals with 4 data points.

Curve fitting

42

• The 3n equations/conditions are required to evaluate the unknowns

1. The function values of adjacent polynomials must be equal at the interior knots.

for i = 2 to n. Therefore, the total is 2n – 2 conditions.

2. The first and the last functions must pass through end points. This adds two additional equations and total of 2n – 2 + 2 = 2n conditions.

Curve fitting

43

)(

)(

11

2

1

1111

2

11

iiiiii

iiiiii

xfcxbxa

xfcxbxa ----- 18.29

----- 18.30

)(

)(

2

0101

2

01

nnnnnn xfcxbxa

xfcxbxa

----- 18.31

----- 18.32

3. The first derivatives at the interior knots must be equal.

for i = 2 to n. This provides another n-1 for a total of 2n + n – 1 = 3n – 1 conditions.

4. Assume the second derivative is zero at the first point.

a1 = 0

Curve fitting

44

iiiiii bxabxa

baxxf

1111 22

2)('

iaxf 2)( ----- 18.34

----- 18.33

Example 12b: Quadratic splines

Fit quadratic splines to the same data used in example 12a. Use the results to estimate the value at x = 5.

4 data points n = 3 intervals 3n = 9 unknowns i.e.

(a1, b1, c1)

(a2, b2, c2)

(a3, b3 , c3)

Curve fitting

45

x f(x)

3.0 2.5

4.5 1.0

7.0 2.5

9.0 0.5

Solutions

a. 2n – 2 = 2(3) – 2 = 4 equations

Using eqs 18.29 and 18.30;

for i = 2 to n;

Curve fitting

46

)(

)(

11

2

1

1111

2

11

iiiiii

iiiiii

xfcxbxa

xfcxbxa

)(

)(

'3for

)(

)(

,2for

2323

2

23

2222

2

22

1212

2

12

1111

2

11

xfcxbxa

xfcxbxa

i

xfcxbxa

xfcxbxa

i

Therefore;

b. Passing the first and last functions 2n – 2 + 2 = 2n conditions i.e. 2(3)=6 equations.

Using eqs 18.31 and 18.32

For x0 = 3 and xn = x3 = 9;

Curve fitting

47

5.2)7()7(

5.2)7()7(

0.1)5.4()5.4(

0.1)5.4()5.4(

33

2

3

22

2

2

22

2

2

11

2

1

cba

cba

cba

cba

)(

)(

2

0101

2

01

nnnnnn xfcxbxa

xfcxbxa

5.0)9()9(

5.2)3()3(

33

2

3

11

2

1

cba

cba

----- 3

----- 2

----- 1

----- 4

Therefore;

c. The first derivatives at the interior knots must be equal.

Using eq. 18.33

Curve fitting

48

5.0981

5.239

333

111

cba

cba

----- 6

----- 5

323222

212111

1111

22

3For

22

2For

22

2)('

bxabxa

i

bxabxa

i

bxabxa

baxxf

iiiiii

Therefore;

d. Second derivative is zero at the first point i.e. x1.

Using eq. 18.34;

a1 = 0

Because this equation specifies a1, we have 8 simultaneous equations.

Curve fitting

49

3322

2211

1414

99

baba

baba

----- 7

----- 8

02)( i

axf

----- 9

In matrix form:

Curve fitting

50

0

0

5.0

5.2

5.2

5.2

1

1

0114011400

00001901

198100000

00000013

174900000

000174900

00015.425.2000

00000015.4

3

3

3

2

2

2

1

1

c

b

a

c

b

a

c

b

These equations are solved with the results:

a1 = 0 b1 = -1 c1 = 5.5

a2 = 0.64 b2 = -6.76 c2 = 18.46

a1 = -1.6 b1 = 24.6 c1 = -91.3

Substituted into the original quadratic equations:

f1(x) = -x + 5.5 3.0 < x < 4.5

f2(x) = 0.64x2 – 6.76x + 18.46 4.5 < x < 7.0

f3(x) = -1.6x2 + 24.6x – 91.3 7.0 < x < 9.0

Therefore f2 at x = 5 is:

f2(5) = 0.64(5)2 – 6.76(5) + 18.46 = 0.66

Curve fitting

51

iiiicxbxaxf 2)(

Chapter 5

Curve fitting

Assignment (please use excell)

Given the data below, use least squares regression to fit a) a straight line b) a power equation c) a saturation-growth-rate equation and d) a parabola. Find the r2 value and justify which is the best equation to represent the data.

Curve fitting

53

x 5 10 15 20 25 30 35 40 45 50

y 17 24 31 33 37 37 40 40 42 51

Solution 17.14 a) The regression of y versus x to give

17.14 b) The regress ion of log10y versus log10x to give

• Therefore, 2 = 100.99795 = 9.952915 and 2 = 0.385077, and the power model is

Curve fitting

54

xy 494545.06.20

y = 0.4945x + 20.6

R2 = 0.8385

0

10

20

30

40

50

0 10 20 30 40 50 60

xy 1010 log385077.099795.0log

y = 9.9529x0.3851

R2 = 0.9553

0

10

20

30

40

50

0 10 20 30 40 50 60

17.14 c) The regression of 1/y versus 1/x to give

• Therefore, 3 = 1/0.01996322 = 50.09212 and 3 = 0.19746357(50.09212) = 9.89137, and the saturation-growth-rate model is

Curve fitting

55

xy

1197464.0019963.0

1

x

xy

9.8913709212.50

0

10

20

30

40

50

0 10 20 30 40 50 60

y = 50.092x

x + 9.891369

R2 = 0.98919

17.14 d) The polynomial regression to fit a parabola

The model and the data can be plotted as

• Comparison of fits: The linear fit is obviously inadequate. Although the power fit follows the

general trend of the data, it is also inadequate because (1) the residuals do not appear to be randomly distributed around the best fit line and (2) it has a lower r2 than the saturation and parabolic models.

• The best fits are for the saturation-growth-rate and the parabolic models. They both have randomly distributed residuals and they have similar high coefficients of determination. The saturation model has a slightly higher r2. Although the difference is probably not statistically significant, in the absence of additional information, we can conclude that the saturation model represents the best fit.

Curve fitting

56

76667.11377879.101606.0 2 xxy

y = -0.0161x2 + 1.3779x + 11.767

R2 = 0.98

0

10

20

30

40

50

0 10 20 30 40 50 60

y = 0.4945x + 20.6

R2 = 0.8385

0

10

20

30

40

50

0 10 20 30 40 50 60

y = 9.9529x0.3851

R2 = 0.9553

0

10

20

30

40

50

0 10 20 30 40 50 60

0

10

20

30

40

50

0 10 20 30 40 50 60

y = 50.092x

x + 9.891369

R2 = 0.98919

y = -0.0161x2 + 1.3779x + 11.767

R2 = 0.98

0

10

20

30

40

50

0 10 20 30 40 50 60