CHAPTER 3CHAPTER 3

RECURSIVE ESTIMATION FOR LINEAR MODELS

•Organization of chapter in ISSO–Linear models

•Relationship between least-squares and mean-square

–LMS and RLS estimation •Applications in adaptive control

–LMS, RLS, and Kalman filter for time-varying solution

–Case study: Oboe reed data

Slides for Introduction to Stochastic Search and Optimization (ISSO) by J. C. Spall

3-2

Basic Linear ModelBasic Linear Model

•Consider estimation of vector in model that is linear in

•Model has classical linear form

where zk is kth measurement, hk is corresponding “design vector,”

and vk is unknown noise value

•Model used extensively in control, statistics, signal processing,

etc.

•Many estimation/optimization criteria based on “squared-error”-

type loss functions

– Leads to criteria that are quadratic in

– Unique (global) estimate

,Tk k kz vh

3-3

Least-Squares EstimationLeast-Squares Estimation

•Most common method for estimating in linear model is by method

of least squares

•Criterion (loss function) has form

where Zn = [z1, z2 ,…, zn]T and Hn is n p concatenated matrix of hk

T

row vectors

•Classical batchbatch least-squares estimate is

•Popular recursiverecursive estimates (LMS, RLS, Kalman filter) may be

derived from batch estimate

2

1

1 1

2 2( ) ( )( )

nT T

k k n n n nk

zn n

h Z H Z H

( ) 1ˆ ( )n T Tn n n nH H H Z

3-4

Geometric Interpretation of Least-Squares Geometric Interpretation of Least-Squares Estimate when Estimate when pp = 2 and = 2 and nn = 3 = 3

3-5

Recursive EstimationRecursive Estimation

•Batch form not convenient in many applications– E.g., data arrive over time and want “easy” way to update

estimate at time k to estimate at time k+1

•Least-mean-squares (LMS) method is very popular recursive method

– Stochastic analogue of steepest descent algorithm

•LMS recursion:

•Convergence theory based on stochastic approximation (e.g., Ljung, et al., 1992; Gerencsér, 1995)

– Less rigorous theory based on connections to steepest descent (ignores noise) (Widrow and Stearns, 1985; Haykin, 1996)

1 1 1 1ˆ ˆ ˆ , 0( )T

k k k k k ka z ah h

3-6

LMS in Closed-Loop ControlLMS in Closed-Loop Control•Suppose process is modeled according to autoregressive (AR) form:

where xk represents state, and i are unknown parameters, uk is control, and wk is noise

•Let target (“desired”) value for xk be dk •Optimal control law known (minimizes mean-square tracking error):

•Certainty equivalence principle justifies substitution of parameter estimates estimates for unknown true parameters

– LMS used to estimate and i in closed-loop mode

1 0 1 1 ,k k k m k m k kx x x x u w

1 0 1 1k k k m k m

kd x x x

u

3-7

LMS in Closed-Loop Control for LMS in Closed-Loop Control for First-Order AR ModelFirst-Order AR Model

3-8

Recursive Least Squares (RLS)Recursive Least Squares (RLS)

•Alternative to LMS is RLS – Recall LMS is stochastic analogue of steepest descent (“first

order” method)

– RLS is stochastic analogue of Newton-Raphson (“second order” method) faster convergence than LMS in practice

•RLS algorithm (2 recursions):

•Need P0 and to initialize RLS recursions

1 11

1 1

1 1 1 1 1

1

( )

Tk k k k

k k Tk k k

Tk k k k k k k

ˆ ˆ ˆ z

P h h PP P

h P h

P h h

0̂

3-9

Recursive Methods for Estimation of Time-Recursive Methods for Estimation of Time-Varying ParametersVarying Parameters

•It is common to have the underlying true evolve in time (e.g., target tracking, adaptive control, sequential experimental design, etc.)

– Time-varying parameters implies replaced with k

•Consider modified linear model

•Prototype recursive form for estimating k is

where choice of Ak and k depends on specific algorithm

1 1 1 1( ),Tk k k k k k k k

ˆ ˆ ˆ z A h A

Tk k k kz v h

3-10

Three Important Algorithms for Estimation Three Important Algorithms for Estimation of Time-Varying Parametersof Time-Varying Parameters

• LMSLMS– Goal is to minimize instantaneous squared-error criteria across

iterations– General form for evolution of true parameters k

• RLSRLS– Goal is to minimize weighted sum of squared errors– Sum criterion creates “inertia” not present in LMS– General form for evolution of k

• Kalman filterKalman filter– Minimizes instantaneous squared-error criteria – Requires precise statistical description of evolution of k via

state-space model

• Details for above algorithms in terms of prototype algorithm (previous slide) are in Section 3.3 of ISSO

3-11

Case Study: LMS and RLS with Oboe Reed DataCase Study: LMS and RLS with Oboe Reed Data

……an ill wind that nobody blows good.an ill wind that nobody blows good.—Comedian Danny Kaye in speaking of the oboe in the “The Secret Life of Walter Mitty” (1947)

•Section 3.4 of ISSO reports on linear and curvilinear models for predicting quality of oboe reeds

– Linear model has 7 parameters; curvilinear has 4 parameters

•This study compares LMS and RLS with batch least-squares estimates

– 160 data points for fitting models (reeddata-fitreeddata-fit ); 80 (independent) data points for testing models (reeddata-reeddata-testtest)

– reeddata-fitreeddata-fit and reeddata-testreeddata-test data sets available from ISSO Web site

Oboe with Oboe with Attached ReedAttached Reed

3-13

Comparison of Fitting Results for Comparison of Fitting Results for reeddata-fitreeddata-fit and and reeddata-testreeddata-test

• To test similarity of fit and test data sets, performed model fitting using test data set

• This comparison is for checking consistency of the two data sets; not for checking accuracy of LMS or RLS estimates

• Compared model fits for parameters in

– Basic linear model (eqn. (3.25) in ISSO) (p = 7)

– Curvilinear model (eqn. (3.26) in ISSO) (p = 4)

• Results on next slide for basic linear model

3-14

Comparison of Batch Parameter Estimates for Comparison of Batch Parameter Estimates for Basic Linear Model. Approximate 95% Basic Linear Model. Approximate 95%

Confidence Intervals Shown in [·, ·]Confidence Intervals Shown in [·, ·] reeddata-fit reeddata-test Constant, const

0.156 [ 0.52, 0.21]

0.240 [ 0.75, 0.28]

Top close, T 0.102 [0.01, 0.19]

0.067 [ 0.12, 0.25]

Appearance, A

0.055 [ 0.08, 0.19]

0.178 [ 0.03, 0.39]

Ease of Gouge, E

0.175 [0.05, 0.30]

0.095 [ 0.15, 0.34]

Vascular, V 0.044 [ 0.08, 0.17]

0.125 [ 0.06, 0.31]

Shininess, S

0.056 [ 0.06, 0.17]

0.066 [ 0.13, 0.26]

First blow, F 0.579 [0.41, 0.74]

0.541 [0.24, 0.84]

3-15

Comparison of Batch and RLS with Comparison of Batch and RLS with Oboe Reed Data Oboe Reed Data

• Compared batch and RLS using 160 data points in reeddata-fit and 80 data points for testing models in reeddata-test

• Two slides to follow present results

– First slide compares parameter estimates in pure linear model

– Second slide compares prediction errors for linear and curvilinear models

3-16

Batch and RLS Parameter Estimates for Basic Batch and RLS Parameter Estimates for Basic Linear Model (Data from Linear Model (Data from reeddata-fitreeddata-fit ))

Batch Estimates

RLS Estimates

Constant,

const 0.156 0.079

Top close, T 0.102 0.101 Appearance,

A 0.055 0.046

Ease of Gouge, E

0.175 0.171

Vascular, V 0.044 0.043 Shininess,

S 0.056 0.056

First blow, F 0.579 0.540

3-17

Mean and Median Absolute Prediction Mean and Median Absolute Prediction Errors for the Linear and Curvilinear Models Errors for the Linear and Curvilinear Models (Model fits from (Model fits from reeddata-fit;reeddata-fit; Prediction Prediction

Errors from Errors from reeddata-testreeddata-test) )

Batch linear model

RLS linear model

Batch curvilinear

model

RLS curvilinear

model Mean 0.242 0.242 0.235 0.235

Median 0.243 0.250 0.227 0.224

Ran matched-pairs t-test on linear versus curvilinear models. Used one-sided test.

P-value for Batch/linear versus Batch/curvilinear is 0.077

P-value for RLS/linear vs. RLS/curvilinear is 0.10

Modest evidence for superiority of curvilinear model