the topic: least squares method numeriska beräkningar i naturvetenskap och teknik
TRANSCRIPT
The topic:
Least squares method
Numeriska beräkningar i Naturvetenskap och Teknik
An exemple
80
70
60
50
40
30
20
T
83.54
48.53
76.51
71.50
60.50
19.50
56.48
)(TfL
T
L
Model
TccTfL 10)(
20 40 60 80
48
50
52
54
Why do the measured values deviate from the model if the measurement is correct?
Numeriska beräkningar i Naturvetenskap och Teknik
How determine the ‘best’ straight line?
T
L
Model bTaTfL )(
20 40 60 80
48
50
52
54
Numeriska beräkningar i Naturvetenskap och Teknik
Distance between line and measurements points...
T
L
20 40 60 80
48
50
52
54
1d
2d3d
4d5d
6d
7d
Numeriska beräkningar i Naturvetenskap och Teknik
How to define the distance between the line and the measurement points?
id Largest deviation at minimum Approximation in maximum norm
2id
Sum of deviations squared as small as possible Approximation in Euclidian norm
Easier to calculate!
Norm
Numeriska beräkningar i Naturvetenskap och Teknik
Matrix formulation: An example
3
2
0
1
3
x
9.0
6.0
6.0
9.0
1.2
)(
xf
x
f
xccxf 10)(
9.03
6.02
6.0
9.0
1.23
10
10
0
10
10
cc
cc
c
cc
cc
9.0
6.0
6.0
9.0
1.2
1
1
1
1
1
3
2
0
1
3
1
0
c
c
fAc with
More equations than unknowns!
Numeriska beräkningar i Naturvetenskap och Teknik
Matrix formulation: An example
9.0
6.0
6.0
9.0
1.2
1
1
1
1
1
3
2
0
1
3
1
0
c
c
fAAcA TT
3
1
1
1
0
1
2
1
3
1
1
1
1
1
1
3
2
0
1
3
1
0
c
c
3
1
1
1
0
1
2
1
3
1
9.0
6.0
6.0
9.0
1.2
Numeriska beräkningar i Naturvetenskap och Teknik
Matrix formulation: An example
fAAcA TT
23 1
1 5
1
0
c
c
11.1
1.2
1
0
c
c
0.505
521.0
xxf 505.0521.0)(
Numeriska beräkningar i Naturvetenskap och Teknik
Matrix formulation: An example
xxf 505.0521.0)(
Numeriska beräkningar i Naturvetenskap och Teknik
General Statement of the Problem:
Depending on the model, the measurement data can of course be described by other expressions than the straight line.
In general terms one seeks a function f* that approximates f’s given values as well as possible in euclidian norm.
Specifically, above we looked for a solution expressed as
xccxf 10)(
but we could as well have looked for a solution given by another function (possibly then for different data)
)cos()sin()( 10 xcxcxf
etc...
Numeriska beräkningar i Naturvetenskap och Teknik
Generally one can thus write:
nncccxf ...)( 1100
f(x) is in other words a linear combination of given functions
n ,...,, 10 Where the coefficients
nccc ,...,, 10 are sought
One can in accordande with a vector space look at it so that
n ,...,, 10
Spans a function space (a space of this kind which fulfills certain conditions is called a Hilbert space, cmp. quant. mech)
Numeriska beräkningar i Naturvetenskap och Teknik
In the case of the straight line we have
x
1
0 1
In a geometrical comparision these two functions, which can be seen as two vectors in the function space, span a plane U:
10
x1
*f
”vector” 0
”vector” 1
Approximating function
f sought function
The smallest distance from the plane is given by a normal. The Smallest deviation between f* och f is for f*-f orthogonal to the plane U!
Numeriska beräkningar i Naturvetenskap och Teknik
Normal equations
Since we are interested in fitting m measured values weleave the picture of the continuous function space and view f(x) as an m-dimensional vector with values:
)(
...
)(
)(
2
1
mxf
xf
xf
)(0
)2(0
)1(0
...m
)(1
)2(1
)1(1
...m
That should be expressed by
and0 1f
For the straight line:
1
...
1
1
0
mx
x
x
...1
1
1
Numeriska beräkningar i Naturvetenskap och Teknik
The orthogonality condition now gives the equations:
0),*(
0),*(
1
0
ff
ff1100* ccf where
0),(
0),(
11100
01100
fcc
fcc
€
c0(ϕ 0,ϕ 0) + c1(1ϕ1,ϕ 0) − ( f ,ϕ 0) = 0
c0(ϕ 0,ϕ1) + c1(ϕ1,ϕ1) − ( f ,ϕ1) = 0
€
c0(ϕ 0,ϕ 0) + c1(1ϕ1,ϕ 0) = ( f ,ϕ 0)
c0(ϕ 0,ϕ1) + c1(ϕ1,ϕ1) = ( f ,ϕ1)
the equations for the normal:
Which gives
Numeriska beräkningar i Naturvetenskap och Teknik
),(),(),(
),(),(),(
1111100
0011000
fcc
fcc
The equations for the normal :
),( ),(
),( ),(
1110
0100
1
0
c
c
),(
),(
1
0
f
f
1
0
. .
. .
. .
. .
. .
. .
10 1
0
c
c
1
0
.
.
.
.
.
.
f
Numeriska beräkningar i Naturvetenskap och Teknik
Back to the exemple:
3
2
0
1
3
x
9.0
6.0
6.0
9.0
1.2
)(
xf
xccxf 10)(
1
1
1
1
10
3
2
0
1
3
)(1
x
Model:
3
2
0
1
3
3 2 0 1- 3 1 1 1 1 1
1
1
1
1
1
1
0
c
c
9.0
6.0
6.0
9.0
1.2
1 1 1 1 1 3 2 0 1- 3
TA A TAc f
Data:
Numeriska beräkningar i Naturvetenskap och Teknik
Conclusion:
2||*|| ff
the minimum of
ff * is orthogonal to the basis vectors
n ...,, 3,210
Assuming the model
nncccccf ...* 33221100
Given data f
is obtained when
The coefficienterna c1, c2, c3, cn are determined from
Numeriska beräkningar i Naturvetenskap och Teknik
The equations
),(),(...),(),(
...
),( ),(...),(),(
),(),(...),(),(
1100
11111100
00011000
nnnnnn
nn
nn
fccc
fccc
fccc
or
fAAcA TT
Where the colomuns in A are: n ...,, 3,210
Numeriska beräkningar i Naturvetenskap och Teknik
Note 1:
The func’s Have to be linearly independent
n ...,, 3,210
(cmp vectors in a vector space)
Note 2:
993
994
996
997
999
x
9.0
6.0
6.0
9.0
1.2
)(
xf
Assume our problem would have been (x koord -996)
4958111 4979
4979- 5
1
0
c
c
2102.7
1.2
23 1
1 5
1
0
c
c
11.1
1.2
cmp to
Numeriska beräkningar i Naturvetenskap och Teknik