ucla chemical engineering

29
Process & Control Systems Engineering Laboratory UCLA Chemical Engineering

Upload: others

Post on 10-Jan-2022

1 views

Category:

Documents


0 download

TRANSCRIPT

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Constrained In�nite�Time Nonlinear Quadratic

Optimal Control

V� Manousiouthakis D� Chmielewski

Chemical Engineering Department

UCLA

���� AIChE Annual Meeting

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Outline

Unconstrained In�nite�Time Nonlinear Quadratic�Optimal Control

� Hamilton�Jacobi�Bellman Equation

� State Dependent Riccati Equation and the Curl Condition

� Closed�Loop Stability

� Example

Constrained In�nite�Time Nonlinear Quadratic�Optimal Control

� Family of Finite Time Problems

� Equivalence to the In�nite Time Problem

� Example

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

In�nite�time Nonlinear Quadratic Optimal Control

Problem

minu

Z�

�xTQ�xx uTR�xu

�dt

s�t� �x � f�x B�xu

The solution can be found by solving the HJB equation

�J�x

Tf�x xTQ�xx�

�J�x

TB�xR���xBT �x�J

�x� �

The optimal policy is given by u��x � ���R���xBT �x�

�J�x

�T

The determination of J�x is nearly impossible in all but the simplest cases�

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

In�nite�time Nonlinear Quadratic Optimal Control

J�x can be written as� J�x � xTP �xx� where P �x is symmetric�

In this case�

�J�x

� ��

B�P �x �

�xT�P

�x�

CAx

De�ne

��x ��P �x �

�xT�P

�x

Then the HJB becomes�

xT�

��xA�x AT �x��x Q�x� ��xB�xR���xBT �x��x�

x � �

where A�x s�t� f�x � A�xx�

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Assumptions

�H� R�x � �� � x � �n� and Q�x � �� � x � D � �n

�H� A�x and B�x are analytic matrix valued functions � x � D

�H� The pair �A�x� B�x is controllable �in the linear system sense � x � D

�H The gradients ��A�x�x�

�x

and �bi�x�

�x

exist and are continuous and bounded

� x � D �where bi�x is the ith column of B�x

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

State Dependent Riccati Equation

If ��x satis�es the SDRE�

��xA�x AT��x� ��xB�xR���xBT �x��x Q�x � �

then the HJB will appear to be satis�ed� In this case� the optimal policy is

u��x � �R���xBT �x��xx

and the optimal value function isJ�x � xTP �xx

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Gradient of a Scalar Function

For the HJB to actually be satis�ed

�xT��x must equal�J

�x

The conditions for a vector function� v�x� to be a gradient are �

�vi�xj

��vj

�xi� � i� j

Assumption� �H� A�x is s�t� the solution to the SDRE� ��x� satis�es�

curl�xT��x � � � � x � D

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Existence of a ��x�

The following guarantees the existence of a SDRE such that the curl

condition is satis�ed �Lu and Huang� ���

If there exists a solution to the HJB equation�

J�x�� then there exists A�x� s�t� the solution

to the SDRE equation� ��x�� satis�es

�J�x

� xT��x��

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Stability and Optimality

Theorem��Manousiouthakis and Chmielewski� ����

Let �H���H� hold� Then the scalar function

J�x � xTZ �

� ����xd�

x

satis�es the necessary conditions for optimality �HJB for all x � D�

Furthermore� the feedback policy

u�x � �R���xBT �x��xx

is asymptotically stabilizing�

Corollary� Let �H���H� hold� and assume D � �n�

Then the feedback policy is globally asymptotically stabilizing�

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

An Inverse Method

Consider ��x and A�x to be design functions�

First we must guarantee

��x � � � curl�xT��x � � � A�xx � f�x

Then simply calculate Q�x from the SDRE�

Q�x � ��xB�xR���xBT �x��x� AT �x��x� ��xA�x

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

An Inverse Method �continued�

Let Z� be de�ned as �a positively invariant set

Z� ��

x � �njJ�x � � � J�x � xT�Z �

� ����xd�

x�

De�ne �� s�t�

xTQ�xx uTR�xu � xT�

Q�x ��xB�xR���xBT �x��x�

x

� � � x � Z��

Then a su�cient condition closed�loop stability is that x�� � Z���

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Example

Consider a CSTR Reactor with reaction �Aka

��kb

B�

���������x�

�x��

������� ��

���������kax�

� ��

kaCAss FV

�x� �kbx�

kax�

� �kaCAssx� ��

FV

kb�

x�

��������

��������F

V�

�������� u

x� � CA � CAss � x� � CB � CBss � u � CAin � CAinss

CBss �

ka

FV

kbC�Ass �

��

�CAinss � CAss

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Example �continued�

The following SS design parameters are used �all in SI units

ka � ��� kb � ��� FV

� ���

CAinss � �� CAss � ��� CBss � ��

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Example �continued�

Let ��x � Po where Po satis�es

ATo Po PoAT

o Qo � PoBR��BTPo � �

where R � �VF ��

Ao ��

��� ��

kaCAss FV

�kb

�kaCAss ��

FV

kb�

���� and Qo �

���� qo

�� �

� qo���

���

Clearly this choice of ��x satis�es the �Curl� condition�

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Example �continued�

Q�x is calculated� via the SDRE� to be�

Q�x ��

�������qo�� �kax���p�� � p�� kax��p�� � �p��

kax��p�� � �p�� qo��

��������

Then Q�x PoBR��BTPo is positive de�nite i�

qo�� �kax���p�� � p�� p��� � �

�qo�� �kax���p�� � p�� p���

�qo�� � �kax��p�� � �p��p��p��� � �

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Example �continued�

For qo�� � �� and qo�� � ��

x� � � ��� � ��� � Q�x PoBR��BTPo � �

The largest positively invariant set

Z� ��

� � ��j�TPo� � ��

such that Q�x PoBR��BTPo � � � x � Z�� corresponds to

� � ����

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

10

5

0

-5

Conce

ntr

atio

n o

f B

1050-5-10

Concentration of A

Q(x

) +

PoB

R-1

BTP

o >

0

Q(x) + PoB R-1BTP

o >0

Example 1: Closed-Loop Stability Region

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

5

4

3

2

1

0

Conce

ntr

atio

n

250200150100500

Time

CA

CBCB

Nominal Inputs Optimal Control

Example 1: Closed-Loop Simulations

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Constrained ITNQOC Problem

��� � infu

�Z�

�xTQx �T �uR��u

�dt

s�t� �x � A�xx B�x��u� x�� � �

where ��� � �M U � is de�ned as

��u � arg min��Ujju� jj

Further Assumptions�

�H� U is convex and contains the origin in its interior

�H� � � Xo �� f� � �nj u s�t� ��� ��g

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Constrained FTNQOC Problem

Consider the following family of �nite�time optimal control problems�

�T �� � infu

�Z T�

�xTQx �T �uR��u

�dt J�x�T

s�t� �x � A�xx B�x��u� x�� � �

where

J�x � xT�Z �

� ����xd�

x

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Equivalence of Constrained and Unconstrained

Problems

De�ne

�X � f� � �nj K��� � Ug

O� ���

� � �nj x�t � �X � t � ��

where K�x is the unconstrained optimal feedback gain�

Lemma� Let K�x be bounded for all x in a neighborhood of the origin� Then

� � intfUg �� � � intfO�g

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Finite�Time Solution to In�nite�Time Problem

Lemma� Let �H��H� hold and assume the solution to �T is unique� Then

�i �� �� limT���T exists for all � � Xo

�ii �T � �� � �T � �T�� � � � � � u�T � u�T�� � � � � � x�T �T � O�

�iii If T � � such that x�T �T � O� then �T � �

Theorem� Let �H��H� hold and assume the solution to �T is unique� Then

� � � Xo T s�t� x�T �T � O�

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Equivalence of Initial Condition Sets

De�ne the set of constrained stabilizable initial conditions as

Xmax ���

�� � �nj u s�t� limt��

jjx�tjj � ����

Theorem� Let �H��H� hold and assume the solution to �T is unique� Then

Xmax � Xo

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Example �

Consider the CSTR of Example � with kb � �

�x� � ��kax�

� ��

B� kaCAss F

V�

CAx� F

Vu

with objective function �r � �FV �

Z�

�x��t� ru�t��

dt

and constraints

u�t CAinss � ��� ��� � t � � Xmax � �������

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Example � �continued�

The resulting SDRE is�

� ���kax� � �kaCAss ��x� �� ���x� � �

Which has solution

��x� � �

kax� kaCAss r

�kax� kaCAss� �

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Example � �continued�

The Unconstrained Optimal Control Policy is

u�x � �V

F

kax� kaCAss r

�kax� kaCAss� �

x�

and

O� � ���� ��

Finally� the Closed�Loop System is

�x � �x�r

�kax� kaCAss� �

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

1.0

0.8

0.6

0.4

0.2

0.0

Conce

ntr

atio

n o

f A

2520151050

Time

No Constraints Constrained Input Nominal Input

Example 2: Optimal State Trajectories

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

60

40

20

0

CA

in

2520151050

Time

No Constraints Constrained Input

Example 2: Optimal Input Policies

Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering

Acknowledgments

The Following Financial Support is Gratefully Acknowledged�

� NSF GER���� ���

� NSF CTS����� ��

� DOEd P���A �������