![Page 1: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/1.jpg)
Verifying Stochastic SystemsJacob Steinhardt and Russ Tedrake
Massachusetts Institute of Technology
![Page 2: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/2.jpg)
Motivation
• Robots are often subject to large uncertainties
• dynamical: wind gusts
• perceptual: stereo vision
• To maximize performance, want to plan against typical case (99%) rather than worst case (100%)
![Page 3: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/3.jpg)
Lyapunov equations
![Page 4: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/4.jpg)
• Sufficient condition for stability: non-negative function V such that
Lyapunov equations
V̇ (x) ≤ 0
![Page 5: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/5.jpg)
Martingale condition
![Page 6: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/6.jpg)
Martingale condition
E[V̇ (x)] ≤ 0
![Page 7: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/7.jpg)
Martingale condition
E[V̇ (x)]� �� � ≤ 0� �� �
lim∆t→0+
E[V (x(t+∆t)) | x(t)]− V (x(t))
∆t
![Page 8: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/8.jpg)
Martingale condition
![Page 9: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/9.jpg)
Martingale condition
• Too strong!
E[V̇ (x)] ≤ 0
![Page 10: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/10.jpg)
Martingale condition
• Too strong!
• Consider the system
E[V̇ (x)] ≤ 0
![Page 11: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/11.jpg)
Martingale condition
• Too strong!
• Consider the system
E[V̇ (x)] ≤ 0
![Page 12: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/12.jpg)
Martingale condition
• Too strong!
• Consider the system
• What is ?
E[V̇ (x)] ≤ 0
dx(t) = −xdt+ dw(t)
E[V̇ (0)]
![Page 13: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/13.jpg)
Martingale condition
![Page 14: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/14.jpg)
• Instead consider the relaxed condition
Martingale condition
E[V̇ (x)] ≤ c
![Page 15: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/15.jpg)
• Instead consider the relaxed condition
• theorems giving finite-time guarantees (Kushner 1965)
Martingale condition
E[V̇ (x)] ≤ c
![Page 16: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/16.jpg)
• Instead consider the relaxed condition
• theorems giving finite-time guarantees (Kushner 1965)
• todo: choose and optimize over a family of functions V
Martingale condition
E[V̇ (x)] ≤ c
![Page 17: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/17.jpg)
Choosing V
![Page 18: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/18.jpg)
Choosing V
• V(x) = exp(xTJx)
![Page 19: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/19.jpg)
Choosing V
• V(x) = exp(xTJx)
• NB: using xTJx instead has poor results (no bound at all after a few seconds)
![Page 20: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/20.jpg)
Choosing V
• V(x) = exp(xTJx)
• NB: using xTJx instead has poor results (no bound at all after a few seconds)
• E[dV/dt] = p(x)exp(xTJx)
![Page 21: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/21.jpg)
Verifying the Martingale Condition
![Page 22: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/22.jpg)
• from previous slide:
Verifying the Martingale Condition
E[V̇ (x)] = p(x)exT Jx
![Page 23: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/23.jpg)
• from previous slide:
Verifying the Martingale Condition
E[V̇ (x)] = p(x)exT Jx
p(x)exT Jx ≤ c
⇐⇒ p(x) ≤ ce−xT Jx
⇐= p(x) ≤ c(1− xTJx)
![Page 24: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/24.jpg)
• from previous slide:
• polynomial condition: optimize with SOS programming
Verifying the Martingale Condition
E[V̇ (x)] = p(x)exT Jx
p(x)exT Jx ≤ c
⇐⇒ p(x) ≤ ce−xT Jx
⇐= p(x) ≤ c(1− xTJx)
![Page 25: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/25.jpg)
Results: Quadrotor
![Page 26: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/26.jpg)
Results: UAV
![Page 27: Jacob Steinhardt and Russ Tedrake Massachusetts Institute](https://reader034.vdocument.in/reader034/viewer/2022042221/625a5f48efb7836a9d707f6f/html5/thumbnails/27.jpg)
Further reading
• http://groups.csail.mit.edu/locomotion/
• LQR-trees tutorial on Friday in the “Integrated Planning and Control” workshop