optimal control: variational approach to optimal control...

Optimal Control: Variational Approach Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL

28.05.2013

Optimal Control: Indirect Solution Methods and Research Outlook

Timm Faulwasser <[email protected]>

Laboratoire d’Automatique

Overview

Indirect Solution Methods for OCPs

Research Outlook – Model Predictive Control

– Real Time Optimization


28.05.2013

Overview of Numerical Methods for Optimal Control

Continuous Time Optimal Control

Hamilton-Jacobi-Bellman-Equation

• tabulation in state space • closed-loop opt. control

Indirect Methods

• Pontryagin’s Maximum Principle (NCO of OCP)

Direct Solution Methods

• convert OCP into NLP

Direct Single Shooting

• only discretized controls in NLP • sequential approach

Direct Collocation

• discretized states and controls in NLP (collocation) • simultaneous approach

Direct Multiple Shooting

• discretized controls and node start values in NLP • simultaneous approach

First optimize then discretize!

Indirect and Direct Solution Methods

Indirect Methods

• based on Pontryagin’s Maximum Principle or variational principles (NCO of OCP) • no discretization of control profile needed • guess for optimal solution structure needed


• convert OCP into NLP • no knowledge about solution structure required • approximate solutions (control parametrization)

First discretize then optimize!

Lecture II on Optimal Control this lecture


28.05.2013

Optimal Control Problem with Terminal Constraints

Necessary Conditions of Optimality

Indirect Solution Approaches

General Idea

• solve NCO directly

• split NCO into two parts

– NCOs enforced at each iteration

– NCOs which are modified at each iteration

Basic Indirect Shooting

1. Guess

2. Integrate from

3. Compute defect of transversality and terminal conditions

4. Update to enforce that


28.05.2013

Indirect Solution Approaches

Basic Algorithm for Indirect Shooting

1. Guess , choice tolerance , set

2. Integrate from

3. Compute defect of transversality conditions

If STOP

4. Compute gradient

and solve

5. Update

6. Set . Goto 1.

Example: Pontryagin‘s Maximum Principle

• Formulate the NCO. • Formulate the defect .


28.05.2013

Indirect Gradient Projection Method

Input constraints? e.g. via gradient projection methods

Necessary Conditions of Optimality

Indirect Gradient Projection Method

0. Guess input trajectory , choose tolerance , set

1. Integrate forward

2. Integrate backward

3. Compute search direction

4. Compute step size (line search)

5. Compute control trajectory via projection onto input constraints

6. Integrate forward

7. IF STOP

ELSE set GOTO 2.

Dunn, J. (1996). On l2 conditions and the gradient projection method for optimal control problems, SIAM Journal on Control and Optimization 34(4): 1270–1290. K. Graichen and B. Käpernick, A real-time gradient method for nonlinear model predictive control, in Frontiers of Model Predictive Control, T. Zheng, Ed. Rijeka, Croatia: InTech, Feb. 2012, pp. 9–28.


28.05.2013

First optimize then discretize!

Numerical Methods for Optimal Control

Indirect Methods

• based on Pontryagin’s Maximum Principle or variational principles (NCO of OCP) • no discretization of control profile needed • guess for optimal solution structure needed


• convert OCP into NLP • no knowledge about solution structure required • approximate solutions (control vector parametrization)

First discretize then optimize!

Which method should be used? • no conclusive answer depends on considered OCP, preferences, … • accurate solution to OCP first approximate solution by direct method

refine solution with indirect approach • fast online solutions (e.g. NMPC):

• without state constraints direct and indirect methods suitable • with state constraints direct methods + safety margin on constraints

Overview

Indirect Solution Methods for OCPs

Research Outlook – Model Predictive Control

– Real Time Optimization


28.05.2013

Model Predictive Control: Basic Principle

1. Obtain state

2. Predict system and optimize input

3. Apply “optimal” input signal

model predictive control = control based on repeated prediction/optimization

Model Predictive Control

1. State measurement at

2. Solve

3. Apply for

Main Idea of NMPC

Stability, robustness, achievable sampling rates, …?


28.05.2013

Detour: Does the application of optimal controls lead to stability?

• optimal control often computed over finite horizon (e.g. MPC)

• stability deals with long term behavior of trajectories

• even for optimal feedbacks stability conditions need to be checked

• typical example – infinite horizon LQR control

optimal feedback for infinite horizon via (static) Riccati equation

sufficient stability condition: observable and controllable

Kalman, R. Contributions to the theory of optimal control. Bol. Soc. Mat. Mexicana, 1960, 5, 102-119

Optimality does not imply stability!

Stability of MPC?


1. State measurement at

2. Solve

3. Apply for

• stability via design of and/or via controllability assumptions • additional restrictions on stage cost • guaranteed satisfaction of constraints stability can be rigorously shown (details beyond the scope of this course) • MPC can be extended to various problems beyond stabilization: tracking, path following, …


28.05.2013

Output Path-following Problem

Control Task

• convergence to path

• convergence on path

• satisfaction of constraints

Questions

• problem structure suitable formulation? • constraints controller design?

• dynamic system

• output path regular, 1d curve

Analysis of Path-following Problems

state space output space

• = zero-path-error manifold

Path Followability Given a system and a path , under which conditions is it possible to follow exactly? I.e., does an input exist, such that while ?

Augmented System Description

Augmented system:


28.05.2013

Principle of Predictive Path Following

Idea • prediction based on augmented dynamics • costs penalize path-following error

Optimal Control Problem

Path convergence, stability? Faulwasser, T. Optimization-based solutions to constrained trajectory-tracking and path-following problems. Shaker, Aachen, Germany, 2013

Outlook on Applications – Implementation on 2-DoF Robot

KUKA LWR Robot

Unscented Kalman Filter

Predictive Path-following Controller

• two actuated axes • continuous-time nonlinear model • input constraints

Performance?


28.05.2013

Outlook on Applications – Implementation on 2-DoF Robot

Circular Path States and Inputs Cartesian Workspace

• Coriolis and friction terms neglected • prediction horizon 200ms • sampling time 5ms • constraint satisfaction • disturbance attenuation

Predictive Path-following: Implementation on 2-DoF Robot

Circular Path Path Deviation Cartesian Workspace

Benefits?

• Coriolis and friction terms neglected • prediction horizon 200ms • sampling time 5ms • constraint satisfaction • disturbance attenuation


28.05.2013

Example: 2-DoF Robot

Circular Path

Trajectory Tracking (KUKA controller) Path Following

improved performance

J. Matschek. Echtzeitfähige Implementierung einer optimierungsbasierten prädiktiven Pfadverfolgung. Master thesis, OvG University Magdeburg, Germany, 2013.


Remarks:

• achievable sampling rate depends on problem size, problem formulation and computational platform

• special case linear time-discrete systems convex quadratic programs

• down to submillisecond sampling rates for small or linear systems

• nonlinear systems (< 10 states) down to millisecond range

• large progress in theory and computation (Moore‘s Law & improved algorithms) over last two decades

• highly active research field & many open issues

• extensions to trajectory tracking, set-point tracking, path-following, economic MPC, …

Further reading: • Mayne, D.; Rawlings, J.; Rao, C. & Scokaert, P. Constrained model predictive control: Stability and optimality.

Automatica, 2000.

• Rawlings, J. & Mayne, D. Model Predictive Control: Theory & Design, Nob Hill Publishing, Madison, WI, 2009

• Grüne, L. & Pannek, J. Nonlinear Model Predictive Control: Theory and Algorithms, Springer Verlag, 2011

model predictive control = optimization-based control

Feedback-based optimization?


28.05.2013

Static Real-time Optimization

subject to subject to

Consider: • repeated static optimization of real plant • uncertain plant data /plant-model mismatch

Inputs which are optimal for the model are usually not be optimal for the plant!

Plant RTO

Static Real-time Optimization

Remedies • quantify uncertainty and compute min-max solution conservative • update plant model during run time parameter estimation • update optimization problem based on measurements, e.g. modifier adaptation

Inputs which are optimal for the model might not be optimal for the plant!

RTO Plant

subject to

Main idea

Modifier Adaptation

• use linear update terms in optimization problem • needs information about plant gradients estimation problem

-


28.05.2013

Modifier Adaptation with Noise Filtering

RTO Plant

Filter

subject to

Main idea

Filter update

Further reading on modifier adaptation: A. Marchetti, B. Chachaut and D. Bonvin. Modifier-Adaptation Methodology for Real-Time Optimization, in Industrial & Engineering

Chemistry Research, vol. 48, num. 13, p. 6022-6033, 2009.

Upon convergence a KKT point of the plant is obtained!

Modifier Adaptation - Example

QP with uncertain parameters

Example taken from: A. G. Marchetti. Modifier-adaptation methodology for real-time optimization. Thèse EPFL, n° 4449

Filter

Parameters:


28.05.2013

Real-time Optimization

Main idea Use feedback to achieve optimal process operation in the presence of uncertainties. Remarks • static and dynamic optimization problem can be considered • different methods available: G. François and D. Bonvin. Measurement-based Real-Time Optimization of Chemical Processes, in Advances in Chemical Engineering, p. 1-50, Identification, Control and Optimisation of Process Systems 43, 2013

• NCO tracking Bonvin, D. & Srinivasan, B. On the role of the necessary conditions of optimality in structuring dynamic real-time optimization schemes. Computers & Chemical Engineering, Elsevier, 2012

• self-optimizing control S. Skogestad, ``Near-optimal operation by self-optimizing control: From process control to marathon running and business systems'', Computers and Chemical Engineering, 29 (1), 127-137 (2004).

• static RTO, e.g. modifier adaptation A. Marchetti, B. Chachaut and D. Bonvin. Modifier-Adaptation Methodology for Real-Time Optimization, in Industrial & Engineering Chemistry Research, vol. 48, num. 13, p. 6022-6033, 2009.

How to Earn Credits for this Course?

• project-based evaluation (max 2 persons on 1 project)

– apply optimal control/calculus of variations to one of your research problems

– simplify models if necessary

– numerical implementation and theoretical discussion are both possible

– in both cases you should be able to explain what you have been doing ;-)

• written report due on June 28th (hard deadline)

– approximately 10 pages

– earlier submission of reports is welcome

• presentation to teachers and class on July 4th

– 15 minutes plus 10 minutes of discussion

– room ME C2 405

optimal control: variational approach to optimal control...

Documents