selected from presentations by jim ramsay, mcgill university, hongliang fei, and brian quanz basis...

Selected from presentations byJim Ramsay, McGill University,Hongliang Fei, and Brian Quanz

Basis Basics

1. Introduction Basis: In Linear Algebra, a basis is

a set of vectors satisfying: Linear combination of the basis

can represent every vector in a given vector space;

No element of the set can be represented as a linear combination of the others.

In Function Space, Basis is degenerated to a set of basis functions;

Each function in the function space can be represented as a linear combination of the basis functions.

Example: Quadratic Polynomial bases {1,t,t^2}

What are basis functions?

We need flexible method for constructing a function f(t) that can track local curvature.

We pick a system of K basis functions φk(t), and call this the basis for f(t).

We express f(t) as a weighted sum of these basis functions:

f(t) = a1φ1(t) + a2φ2(t) + … + aKφK(t)

The coefficients a1, … , aK determine the shape of the function.

What do we want from basis functions?

Fast computation of individual basis functions. Flexible: can exhibit the required curvature

where needed, but also be nearly linear when appropriate.

Fast computation of coefficients ak: possible if matrices of values are diagonal, banded or sparse.

Differentiable as required: We make lots of use of derivatives in functional data analysis.

Constrained as required, such as periodicity, positivity, monotonicity, asymptotes and etc.

What are some commonly used basis functions?

Powers: 1, t, t2, and so on. They are the basis functions for polynomials. These are not very flexible, and are used only for simple problems.

Fourier series: 1, sin(ωt), cos(ωt), sin(2ωt), cos(2ωt), and so on for a fixed known frequency ω. These are used for periodic functions.

Spline functions: These have now more or less replaced polynomials for non-periodic problems. More explanation follows.

What is Basis Expansion? Given data X and transformation Then we model

as a linear basis expansion in X, where

is a basis function.

( ) : , 1,..., .pmh X m M

( )mh X

Why Basis Expansion? In regression problems, f(X) will

typically nonlinear in X; Linear model is convenient and

easy to interpret; When sample size is very small but

attribute size is very large, linear model is all what we can do to avoid over fitting.

2. Piecewise Polynomials and Splines

Spline: In Mathematics, a spline is a special function

defined piecewise by polynomials; In Computer Science, the term spline more

frequently refers to a piecewise polynomial (parametric) curve.

Simple construction, ease and accuracy of evaluation, capacity to approximate complex shapes through curve fitting and interactive curve design.

Assume four knots spline (two boundary knots and two interior knots), also X is one dimensional.

Piecewise constant basis:

Piecewise Linear Basis:

Basis functions:

Six functions corresponding to a six-dimensional linear space.

Piecewise Cubic Polynomial

Piecewise Cubic Polynomial

http://numericalmethods.eng.usf.edu 14

Spline Interpolation Method

Slides taken from the lecture by

Authors: Autar Kaw, Jai Paul

http://numericalmethods.eng.usf.edu15

What is Interpolation ?

Given (x0,y0), (x1,y1), …… (xn,yn), find the value of ‘y’ at a value of ‘x’ that is not given.


Interpolants

Polynomials are the most common choice of interpolants because they are easy to:

EvaluateDifferentiate, and Integrate.


Why Splines ?2251

1)(

xxf

Table : Six equidistantly spaced points in [-1, 1]

Figure : 5th order polynomial vs. exact function

x 2251

1

xy

-1.0 0.038461

-0.6 0.1

-0.2 0.5

0.2 0.5

0.6 0.1

1.0 0.038461


Why Splines ?

Figure : Higher order polynomial interpolation is a bad idea

-0.8

-0.4

0

0.4

0.8

1.2

-1 -0.5 0 0.5 1

x

y

19th Order Polynomial f (x) 5th Order Polynomial


Linear InterpolationGiven nnnn yxyxyxyx ,,,......,,,, 111100 , fit linear splines to the data. This simply involves

forming the consecutive data through straight lines. So if the above data is given in an ascending

order, the linear splines are given by )( ii xfy

Figure : Linear splines


Linear Interpolation (contd)

),()()(

)()( 001

010 xx

xx

xfxfxfxf

10 xxx

),()()(

)( 112

121 xx

xx

xfxfxf

21 xxx

.

.

.

),()()(

)( 11

11

nnn

nnn xx

xx

xfxfxf nn xxx 1

Note the terms of

1

1)()(

ii

ii

xx

xfxf

in the above function are simply slopes between 1ix and ix .


Example The upward velocity of a rocket is given as a

function of time in Table 1. Find the velocity at t=16 seconds using linear splines.

Table Velocity as a function of time

Figure. Velocity vs. time data for the rocket example

(s) (m/s)0 0

10 227.0415 362.7820 517.35

22.5 602.9730 901.67

t )(tv


Linear Interpolation

10 12 14 16 18 20 22 24350

400

450

500

550517.35

362.78

y s

f range( )

f x desired

x s1

10x s0

10 x s range x desired

,150 t 78.362)( 0 tv

,201 t 35.517)( 1 tv

)()()(

)()( 001

010 tt

tt

tvtvtvtv

)15(1520

78.36235.51778.362

t

)15(913.3078.362)( ttv

At ,16t

)1516(913.3078.362)16( v

7.393 m/s


Quadratic InterpolationGiven nnnn yxyxyxyx ,,,,......,,,, 111100 , fit quadratic splines through the data. The splines

are given by

,)( 112

1 cxbxaxf 10 xxx

,222

2 cxbxa 21 xxx

.

.

.

,2nnn cxbxa nn xxx 1

Find ,ia ,ib ,ic i 1, 2, …, n


Quadratic Interpolation (contd)

Each quadratic spline goes through two consecutive data points

)( 0101

2

01 xfcxbxa

)( 11112

11 xfcxbxa .

.

.

)( 11

2

1 iiiiii xfcxbxa

)(2

iiiiii xfcxbxa .

.

.

)( 11

2

1 nnnnnn xfcxbxa

)(2

nnnnnn xfcxbxa

This condition gives 2n equations


Quadratic Splines (contd)The first derivatives of two quadratic splines are continuous at the interior points.

For example, the derivative of the first spline

112

1 cxbxa is 112 bxa

The derivative of the second spline

222

2 cxbxa is 222 bxa

and the two are equal at 1xx giving

212111 22 bxabxa

022 212111 bxabxa


Quadratic Splines (contd)Similarly at the other interior points,

022 323222 bxabxa

.

.

.

022 11 iiiiii bxabxa

.

.

.

022 1111 nnnnnn bxabxa

We have (n-1) such equations. The total number of equations is )13()1()2( nnn .

We can assume that the first spline is linear, that is 01 a


Quadratic Splines (contd)This gives us ‘3n’ equations and ‘3n’ unknowns. Once we find the ‘3n’ constants,

we can find the function at any value of ‘x’ using the splines,

,)( 112

1 cxbxaxf 10 xxx

,222

2 cxbxa 21 xxx

.

.

.

,2nnn cxbxa nn xxx 1


Quadratic Spline ExampleThe upward velocity of a rocket is given as a function of time. Using quadratic splinesa) Find the velocity at t=16 secondsb) Find the acceleration at t=16 secondsc) Find the distance covered between t=11 and t=16 secondsTable Velocity as a

function of time

Figure. Velocity vs. time data for the rocket example

(s) (m/s)0 0

10 227.0415 362.7820 517.35

22.5 602.9730 901.67

t )(tv


Solution,)( 11

21 ctbtatv 100 t

,222

2 ctbta 1510 t

,332

3 ctbta 2015 t,44

24 ctbta 5.2220 t

,552

5 ctbta 305.22 t

Let us set up the equations


Each Spline Goes Through Two Consecutive Data

Points,)( 11

21 ctbtatv 100 t

0)0()0( 112

1 cba

04.227)10()10( 112

1 cba


t v(t)

s m/s

0 0

10 227.04

15 362.78

20 517.35

22.5 602.97

30 901.67

Each Spline Goes Through Two Consecutive Data

Points04.227)10()10( 22

22 cba

78.362)15()15( 222

2 cba

78.362)15()15( 332

3 cba

35.517)20()20( 332

3 cba

67.901)30()30( 552

5 cba

35.517)20()20( 442

4 cba

97.602)5.22()5.22( 442

4 cba

97.602)5.22()5.22( 552

5 cba


Derivatives are Continuous at Interior Data Points

,)( 112

1 ctbtatv 100 t

,222

2 ctbta 1510 t

10

222

210

112

1

tt

ctbtadt

dctbta

dt

d

10221011 22

ttbtabta

2211 102102 baba

02020 2211 baba


Derivatives are continuous at Interior Data Points

0)10(2)10(2 2211 baba

0)15(2)15(2 3322 baba

0)20(2)20(2 4433 baba

0)5.22(2)5.22(2 5544 baba

At t=10

At t=15

At t=20

At t=22.5


Last Equation

01 a


Final Set of Equations

0

0

0

0

0

67.901

97.602

97.602

35.517

35.517

78.362

78.362

04.227

04.227

0

000000000000001

01450145000000000

00001400140000000

00000001300130000

00000000001200120

130900000000000000

15.2225.506000000000000

00015.2225.506000000000

000120400000000000

000000120400000000

000000115225000000

000000000115225000

000000000110100000

000000000000110100

000000000000100

5

5

5

4

4

4

3

3

3

2

2

2

1

1

1

c

b

a

c

b

a

c

b

a

c

b

a

c

b

a


Coefficients of Splinei ai bi ci1 0 22.704 0

2 0.8888 4.928 88.88

3 −0.1356 35.66 −141.61

4 1.6048 −33.956

554.55

5 0.20889 28.86 −152.13


Quadratic Spline InterpolationPart 2 of 2

http://numericalmethods.eng.usf.edu

http://numericalmethods.eng.usf.edu/


Final Solution,704.22)( ttv 100 t

,88.88928.48888.0 2 tt 1510 t,61.14166.351356.0 2 tt 2015 t,55.554956.336048.1 2 tt 5.2220 t,13.15286.2820889.0 2 tt 305.22 t


Velocity at a Particular Pointa) Velocity at t=16

,704.22)( ttv 100 t,88.88928.48888.0 2 tt 1510 t

,61.14166.351356.0 2 tt 2015 t,55.554956.336048.1 2 tt 5.2220 t,13.15286.2820889.0 2 tt 305.22 t

m/s24.394

61.1411666.35161356.016 2

v

Quadratic Spline Graph

t=a:2:b;

Quadratic Spline Graph

t=a:0.5:b;

Natural Cubic Spline Interpolation

The domain of S is an interval [a,b]. S, S’, S’’ are all continuous functions on [a,b]. There are points ti (the knots of S) such that a = t0 < t1 < .. tn =

b and such that S is a polynomial of degree at most k on each subinterval [ti, ti+1].

SPLINE OF DEGREE k = 3

yn…y1y0y

tn…t1t0x

ti are knots


Si(x) is a cubic polynomial that will be used on the subinterval [ xi, xi+1 ].


Si(x) = aix3 + bix2 + cix + di• 4 Coefficients with n subintervals = 4n

equations• There are 4n-2 conditions

• Interpolation conditions• Continuity conditions

• Natural Conditions

• S’’(x0) = 0

• S’’(xn) = 0

selected from presentations by jim ramsay, mcgill university, hongliang fei, and brian quanz basis...

Documents

piecewise linear basis

used basis functions

set of basis

linear basis expansion

introduction basis

piecewise constant basis

spline functions

periodic functions