linear-quadratic-gaussian problem study guide for es205 yu-chi ho jonathan t. lee feb. 5, 2001
DESCRIPTION
3 Linear-Quadratic Problem subject to linear system dynamics given the initial state x(0) where x(i) is the state variables at time i u(i) is the control variable at time i A(i) and B(i) are the cost matrices at time i NTRANSCRIPT
![Page 1: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/1.jpg)
Linear-Quadratic-Gaussian Problem
Study Guide for ES205
Yu-Chi HoJonathan T. LeeFeb. 5, 2001
![Page 2: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/2.jpg)
2
Outline Linear-Quadratic Problem Calculus of Variation Approach Dynamic Programming Approach Kalman Filter Linear Feedback Control
![Page 3: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/3.jpg)
3
Linear-Quadratic Problem
subject to linear system dynamics
given the initial state x(0)where x(i) is the state variables at time iu(i) is the control variable at time iA(i) and B(i) are the cost matrices at time i
1
0 21
21
21min
N
i
TTt
T
iuiuiBiuixiAixNxNANxJ
f
iuiixiix 1
N
![Page 4: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/4.jpg)
4
Calculus of Variation Approach Let
N
1
0
11N
i
T ixiuiixiiJH
NxNANixiAiiixH T with ,10
10 1 iiiBiuuH T
0 with ,10 xiuiixiixH
![Page 5: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/5.jpg)
5
Dynamic Programming Approach Cost-to-go
withwhere
and N
121
21
min
01
0
iNxJ
iNuiNBiNu
iNxiNAiNx
iNxJ
i
T
T
iNui
1112110
1 iNxiNSiNxiNxJ Ti
11211 iNAiNiNMiNiNS T
21
1211
12221
iNSiN
iNiNSiNiNB
iNiNSiNSiNM
T
T
![Page 6: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/6.jpg)
6
DP Approach (cont.) Set We have
Let
andThen
1 1 1u N i B N i N i M N i N x N i
0
0
iNuiNxJ iN
iNAiNiNMiNiNS T 1
iNxiNSiNxiNxJ Ti
210
N
1
1
1111
iNSiN
iNiNSiNiNB
iNiNSiNSiNM
T
T
![Page 7: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/7.jpg)
7
DP Approach (cont.) By induction, we have the optimal
solution to be
whereand
with boundary conditionwith
1 1 1Tu i B i i M i N x i iAiiMiiS T 1
NANS
N
11
111111
iSiiiSiiB
iiSiSiMTT
0,...,1Ni
![Page 8: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/8.jpg)
8
Static State
subject to state transition
with initial conditionwhere
and is noise
ixix 1 00 x
ixiz
N
ix
xizJ0
2
ˆˆ
21min
N
![Page 9: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/9.jpg)
9
Static State (cont.) Set
Thus,
0ˆ0
N
i
xiz
0ˆ
xJ
N
i
izN
x01
1ˆ
N
![Page 10: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/10.jpg)
10
Static State (cont.)
New Estimate = Old Estimate + Confidence Factor Correction Term
11
111
1ˆ1
izi
jzii
iixi
j
11
1ˆ1
izi
ixii
Term Correction
Factor Confidence
ˆ11
1ˆ ixizi
ix
N
![Page 11: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/11.jpg)
11
Dynamic State
subject to state transition
with disturbance (i)where
and is noise
iixix 1
N
i
N
iix
ixizixJ0
21
0
22
21
210
21min
ixiz
N
![Page 12: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/12.jpg)
12
Dynamic State (cont.) Cost-to-go
Set
where We have
N
22
0
00 00
210
21min0ˆ xzxxJ
x
00
00
xJ
002100
210ˆ xzxzx
0410ˆ 20
0 zxJ
00 x
![Page 13: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/13.jpg)
13
Dynamic System (cont.) Cost-to-go
Substitute
04111
210
21min
0ˆ11210
21min1
222
1
00
22
1
01
zxz
xJxzxJ
x
x
0ˆ10 xx
041
1121
0ˆ121
min12
22
1
01
z
xzxxxJ
x
N
![Page 14: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/14.jpg)
14
Dynamic System (cont.) Set
Let Then, we have
01
01
xJ
0ˆ1210ˆ
20ˆ11ˆ xzxxzx
0ˆ1 xx
112111ˆ xzxx
N
![Page 15: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/15.jpg)
15
Dynamic System (cont.) Cost-to-go
Substitute
1ˆ
21
21min 0
1220 ixJixiziixJ iixi
1ˆ ixixi
1ˆ211ˆ
21
minˆ0
1
220
ixJ
ixizixixixJ
iixi
N
![Page 16: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/16.jpg)
16
Dynamic System (cont.) Set
Let and Then, we have
00
ixJ i
1ˆ211ˆ
21ˆˆ
ixizixixizix
1ˆ ixix
ixizixix 21ˆ
00 x
N
![Page 17: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/17.jpg)
17
With Control
subject to state transition
with disturbance (i)where
and is noise
iwiuixix 1
N
ixiz
N
i
N
iix
ixizixJ0
21
0
22
21
210
21min
![Page 18: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/18.jpg)
18
With Control (cont.) Cost-to-go
Set
We haveN
22
0
00 00
210
21min0ˆ xzxxJ
x
00
00
xJ
0210ˆ zx
0410ˆ 20
0 zxJ
![Page 19: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/19.jpg)
19
With Control (cont.) Cost-to-go
Substitute
04111
210
21min
0ˆ11210
21min1
222
1
00
22
1
01
zxz
xJxzxJ
x
x
00ˆ10 uxx
041
112100ˆ1
21
min12
22
1
01
z
xzuxxxJ
x
N
![Page 20: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/20.jpg)
20
With Control(cont.) Set
Let Then, we have
01
01
xJ
00ˆ12100ˆ
200ˆ11ˆ
uxzux
uxzx
00ˆ1 uxx
112111ˆ xzxx
N
![Page 21: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/21.jpg)
21
With Control(cont.) Cost-to-go
Substitute
1ˆ
21
21min 0
1220 ixJixiziixJ iixi
11ˆ iuixixi
1ˆ2111ˆ
21
minˆ0
1
220
ixJ
ixiziuixixixJ
iix
i
N
![Page 22: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/22.jpg)
22
With Control(cont.) Set
Let andThen, we have
00
ixJ i
11ˆ2111ˆ
211ˆˆ
iuixiziuix
iuixizix
11ˆ iuixix
ixizixix 21ˆ
N
00 x
![Page 23: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/23.jpg)
23
Linear Feedback Control The predicted state
with initial statebased on the estimator
112111ˆ ixHizixix
11ˆ iuixix
00 xx
N
![Page 24: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/24.jpg)
24
Linear Feedback Control (cont.) Optimal control based on the
estimated state:
whereand
with boundary conditionwith
N
ixiSNiMiiBiu T ˆ111
iAiiMiiS T 1
NANS
0,...,1Ni
11
111111
iSiiiSiiB
iiSiSiMTT
![Page 25: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/25.jpg)
25
Applications Apollo program: control of the space
craft. Airplane controllers: autopilot. Neighboring Optimal Control for
Non-Linear Systems Chemical process controller Guidance and control
N
![Page 26: Linear-Quadratic-Gaussian Problem Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Feb. 5, 2001](https://reader035.vdocument.in/reader035/viewer/2022070616/5a4d1bea7f8b9ab0599e393f/html5/thumbnails/26.jpg)
26
References:
• Bryson, Jr., A. E. and Y.-C. Ho, Applied Optimal Control: Optimization, Estimation, and Control, Taylor & Francis, 1975.
• Ho, Y.-C., Lecture Notes, Harvard University, 1997.