![Page 1: Helsinki University of Technology Adaptive Informatics Research Centre Finland](https://reader036.vdocument.in/reader036/viewer/2022062314/56814876550346895db57f46/html5/thumbnails/1.jpg)
Helsinki University of TechnologyAdaptive Informatics Research CentreFinland
Variational Bayesian Approach for Nonlinear Identification and Control
Matti Tornio and Tapani Raiko
October 9, 2006
![Page 2: Helsinki University of Technology Adaptive Informatics Research Centre Finland](https://reader036.vdocument.in/reader036/viewer/2022062314/56814876550346895db57f46/html5/thumbnails/2.jpg)
Introduction
System identification and control in nonlinear state-space models
Continues the work by Rosenqvist and Karlström (Automatica 2005)
Our background is in machine learning Uncertainties taken explicitly into account
by using Variational Bayesian treatment
![Page 3: Helsinki University of Technology Adaptive Informatics Research Centre Finland](https://reader036.vdocument.in/reader036/viewer/2022062314/56814876550346895db57f46/html5/thumbnails/3.jpg)
Why nonlinear state-space models?
System identification using a hidden state has many benefits:
More resistant to noise Observations (without history) do not always
carry enough information about the system state
Finds a representation of the state that is more suitable for approximating the dynamics
![Page 4: Helsinki University of Technology Adaptive Informatics Research Centre Finland](https://reader036.vdocument.in/reader036/viewer/2022062314/56814876550346895db57f46/html5/thumbnails/4.jpg)
System identification in nonlinear state-space models
We use a state-of-the-art tool by Valpola and Karhunen (Neural Computation 2002)
Parameters, states, and observations are modelled with Gaussian distributions
Less prone to overfitting (than the prediction error method)
Can determine the dimensionality of the state space etc.
![Page 5: Helsinki University of Technology Adaptive Informatics Research Centre Finland](https://reader036.vdocument.in/reader036/viewer/2022062314/56814876550346895db57f46/html5/thumbnails/5.jpg)
Properties of the method
The model scales well to higher dimensions Can model very complex dynamics Natural conjugate gradient algorithm is used for
fast system identification
![Page 6: Helsinki University of Technology Adaptive Informatics Research Centre Finland](https://reader036.vdocument.in/reader036/viewer/2022062314/56814876550346895db57f46/html5/thumbnails/6.jpg)
Nonlinearities by MLP networks
f(x(t),θ)=B tanh[Ax(t)+a] + b + noise The parameters θ include the weight matrices,
bias vectors, noise variances etc. Note that the policy mapping does not fix the
control signal (because of the noise model)
![Page 7: Helsinki University of Technology Adaptive Informatics Research Centre Finland](https://reader036.vdocument.in/reader036/viewer/2022062314/56814876550346895db57f46/html5/thumbnails/7.jpg)
Variational Bayesian treatment
Posterior probability p(x,θ|y) is approximated by q(x,θ)
q is assumed to be Gaussian with limited dependencies
The fit of q to p is measured by a cost function Both identification and prediction can be done
by minimising the misfit by adjusting the parameters defining q (means, variances, covariances)
![Page 8: Helsinki University of Technology Adaptive Informatics Research Centre Finland](https://reader036.vdocument.in/reader036/viewer/2022062314/56814876550346895db57f46/html5/thumbnails/8.jpg)
Control
Current state is estimated with extended Kalman filter (EKF)
Control signals u(t) are selected to minimise the expected cost E{J} over the distribution q
Quasi-Newton algorithm for optimisation Compare to dual control:
estimation errors increase the expected cost
![Page 9: Helsinki University of Technology Adaptive Informatics Research Centre Finland](https://reader036.vdocument.in/reader036/viewer/2022062314/56814876550346895db57f46/html5/thumbnails/9.jpg)
Control (cont.)
Prediction with variances is ~5 times slower too slow for some applications, the method can still
be used for system identification Learning is done offline
online learning possible as well, leads to different exploration strategies
![Page 10: Helsinki University of Technology Adaptive Informatics Research Centre Finland](https://reader036.vdocument.in/reader036/viewer/2022062314/56814876550346895db57f46/html5/thumbnails/10.jpg)
Optimistic inference control
Alternative control scheme Observations at some point in the future are
fixed and the states leading to this desired future are inferred
Allows the direct use of inference algorithms Conceptually very simple, but not as versatile
as NMPC constraints hard to model
![Page 11: Helsinki University of Technology Adaptive Informatics Research Centre Finland](https://reader036.vdocument.in/reader036/viewer/2022062314/56814876550346895db57f46/html5/thumbnails/11.jpg)
Experiments
Assume the cart-pole system to be unknown Dynamics are identified from only 2500
samples 6-dimensional state space x(t) was used
![Page 12: Helsinki University of Technology Adaptive Informatics Research Centre Finland](https://reader036.vdocument.in/reader036/viewer/2022062314/56814876550346895db57f46/html5/thumbnails/12.jpg)
![Page 13: Helsinki University of Technology Adaptive Informatics Research Centre Finland](https://reader036.vdocument.in/reader036/viewer/2022062314/56814876550346895db57f46/html5/thumbnails/13.jpg)
![Page 14: Helsinki University of Technology Adaptive Informatics Research Centre Finland](https://reader036.vdocument.in/reader036/viewer/2022062314/56814876550346895db57f46/html5/thumbnails/14.jpg)
Results
Very high success rate was reached even under high noise
Partially observed system is hard to control
4 obs
2 obs
4 obs, f=I
2 obs, f=I
0 20 40 60 80 100 120
4 obs
2 obs
4 obs, f=I
2 obs, f=I
0 10 20 30 40 50 60 70 80 90 100
Low noise High noise
![Page 15: Helsinki University of Technology Adaptive Informatics Research Centre Finland](https://reader036.vdocument.in/reader036/viewer/2022062314/56814876550346895db57f46/html5/thumbnails/15.jpg)
Results (initialisation)
Good initialisations are important Local minima are the biggest problem
Internal forward model can provide reasonable initialisations without significant extra computation
![Page 16: Helsinki University of Technology Adaptive Informatics Research Centre Finland](https://reader036.vdocument.in/reader036/viewer/2022062314/56814876550346895db57f46/html5/thumbnails/16.jpg)
Conclusion
Learning nonlinear state-space models seems promising when observations about the system state are incomplete
or the dynamics of the system are not well known
Variational Bayesian treatment helps against overfitting to determine the model order