distributed coordination in multi-agent control systems ...jwinter/phd_proposal_slides.pdf ·...

Distributed Coordination In Multi-Agent Control Systems

Through Model Predictive Control

Distributed Coordination In Multi-Agent Control Systems

Through Model Predictive Control

Dong JiaAdvisor: Bruce H. Krogh

Department of Electrical and Computer Engineering

Carnegie Mellon University

December 17, 2001

Ph.D. Thesis Proposal

�

Motivating ApplicationsMotivating Applications

Subsystem1

Subsystem2

Subsystem3

Interaction

Inte

ract

ion Interaction

/DUJH�6FDOH'\QDPLF6\VWHP

Electric PowerSystems

ManufacturingPlants

TrafficSystems

Multi-RobotSystems

Decentralized Control

�

Automatic Generation Control(ABC Power System)

Automatic Generation Control(ABC Power System)

• To optimize the generation allocation

• To maintain the balance between the power generation and the power consumption

• To maintain the power flow between areas at the scheduled value

4

3

1

2

5

6

B

A

C

AB

SwingZeq9-10

TCSC

SVC

Zeq5-6

L6

L4

L3

L9L10

#8

#7

#6 #9

#5

#4

#3

#2

#1

#13 #14

#10

#11

#12

Agent B

Agent A

Agent C

�

Plant-Wide Process Control(Tennessee Eastman Chemical Plant)

Plant-Wide Process Control(Tennessee Eastman Chemical Plant)

• To maintain the production rate and composition at the set-points

• To keep other variables within specified limits

CWR Stripper

Product

CWR

Compressor

Condenser

CWS

CWS

Stm

Cond

PI

TI

TIFI

TI

FI

Purge

FI

LI

FI

TI

PI

JI

PI

TI

FI

FI

LI

LI

SC

Reactor

ANALYZER

ANALYZER

ANALYZER

C

E

A

D

FI

FI

FI

XA

XB

XC

XD

XE

XF

XA

XB

XC

XD

XE

XF

XG

XH

XD

XE

XF

XG

XH

FI

1

2

3

4

5

6

7

8 9

10

11

12

Vap/LiqSeparator

SeparatorAgent

ReactorAgent

StripperAgent

CompressorAgent

�

Motivating ConsiderationsMotivating Considerations

✦ Decentralized control actions without coordination are typically

conservative;

✦ For existing decentralized control systems, introducing coordination

among decentralized controllers is cheaper than rebuilding a

centralized control system;

✦ Systems controlled by decentralized controllers are more survivable

than those controlled by centralized controllers;

✦ Hierarchical control requires models of the global systems, which is

hard to develop, or even unavailable.

Dynamic Information Sharing Among Decentralized Controllers

�

Structure of Multi-Agent Control SystemStructure of Multi-Agent Control System

$JHQW�1HWZRUN

Communication

Comm

unicatio

n Comm

unication

Agent1

Agent3

Agent2

Control Feedback

Control

Feedback

Control

Feedback

Subsystem1

Subsystem2

Subsystem3

Interaction

Inte

ract

ion Interaction


�

Research ObjectiveResearch Objective

7R� GHYHORS� D� IXOO\� GLVWULEXWHG� FRRUGLQDWLRQ�

PHFKDQLVP�IRU�GHFHQWUDOL]HG�FRQWURO

á No agent manipulates the whole system;

á Each agent has a model of only its local subsystem;

á Online information exchange helps to achieve coordination.

�

Problems Considered in this ResearchProblems Considered in this Research

✟ Framework

✟ Communication Scheme for Coordination

✟ Method to Handle Interaction

✟ Formulation of MPC Optimization to Achieve Coordination

✟ Approaches

✟ Method for Reducing the Conservativeness in Control Actions

✟ Feasibility of the Control Decision

✟ Stability of the System

✟ Set Approximation Method

✟ Computation of Control Invariant Set

✟ Uncertainties in State Estimation

✟ Information Compensation in Asynchronous Control

✟ Reachable Set Computation in Hybrid Systems

✟ Demonstrations

�

OutlineOutline

• Problem Description

• Previous Work

• Approaches

• An Example

• Future Work

��

OutlineOutline

• Problem Description

– Iteration Process

– Minimax Parameterized Optimization

• Previous Work

• Approaches

• An Example

• Future Work

��

One Step-Delayed Communication

Iteration ProcessIteration Process

Exchange information with other agentsExchange information with other agents

Measure local variables and Update state estimateMeasure local variables and Update state estimate

Solve the local model predictive control problemSolve the local model predictive control problem

Apply the control signal for the current instantApply the control signal for the current instant

��

min max J(Θ,V)

whereΘ = {θ (0), θ (1), …, θ (N-1)}V = {v(0), v(1), …, v(N-1)}s.t.x(j+1) = f(x(j),u(j),v(j)), j = 0, 1, …, N-1u(j) = h(x(j),θ (j)), j = 0, 1, …, N-1Θ ∈ Ξv(j) ∈ 9 (j), j = 0, 1, …, N-1x(0) = x0

for all v(j) ∈ 9 (j), j = 0,1,…,N-1x(j+1)=f(x(j),h(x(j),θ (j)),v(j)) ∈ ; (j)g(x(j),h(x(j),h(x(j),θ (j)),v(j)),h(x(j),θ (j)),v(j)) ≤ 0

Min-Max Parameterized Optimization with Constraint Sets C (M2P(C))

Min-Max Parameterized Optimization with Constraint Sets C (M2P(C))

Θ V

The agent solves the following min-max parameterized optimization problem with constraint sets C = { ; (1), …, ; (N-1), 9 (0), …, 9 (N-1) } :

u(j) = h(x(j),θ (j)), j = 0, 1, …, N-1 Parameterized Control Policy

~ ~~

~

~

~~

~ ~

~ ~Quantified Constraints

for all v(j) ∈ 9 (j), j = 0,1,…,N-1x(j+1)=f(x(j),h(x(j),θ (j)),v(j)) ∈ ; (j)g(x(j),h(x(j),h(x(j),θ (j)),v(j)),h(x(j),θ (j)),v(j)) ≤ 0~ ~~

~

~

~~

~ ~

~ ~

prediction horizon

k

optimal control sequence

predicted system trajectory

k+N

implement this part

current state

time

��

Model RequirementModel Requirement

The model in the agent MUST be an abstraction of the real dynamics.

x=*(z)

z

=

;*(=)

*-1(x0)

x0

z0

;(1)

=(1)

*(=(1))

State Space of The Real Dynamics

State Space of The Agent ModelModel in Agent

x(k+1)=f(x(k),u(k),v(k))

g(x(k),x(k+1),u(k),v(k))≤0

Real Dynamics

z(k+1)=F(z(k),u(k),w(k))

G(z(k),z(k+1),u(k),w(k))≤0

��

OutlineOutline

á Problem Description

• Previous Work

• Approaches

• An Example

• Future Work

��

Previous WorksPrevious Works

• Model Predictive Control (MPC)

– Min-Max MPC

[Genceli and Nikolaou, 1993, Zhang and Morari,1993]

– Feedback MPC

[Bemporad, 1993, Scokaert and Mayne, 1998]

• Set Approximation

– Polyhedral Method

[Barmish, Lin and Sankaran, 1978, Farina and Benvenuti,1997, Chutinan and Krogh, 1999]

– Ellipsoidal Method

[Kurzhanski and Valyi, 1991, 1992, 1996]

• Control of Hybrid Systems

– MPC for Hybrid Systems

[Bemporad and Morari, 1999, Stursberg and Engell, 2001]

��

OutlineOutline


á Previous Work

• Approaches

– Methods for Feasibility

– Set Approximation

– Current Results

• An Example

• Conclusion

• Future Work

��

Information Flow in Multi-Agent Control SystemInformation Flow in Multi-Agent Control System

$JHQW�1HWZRUN

Communication

Comm

unicatio

n Comm

unication

Agent1

Agent3

Agent2

Control Feedback

Control

Feedback

Control

Feedback

Subsystem1

Subsystem2

Subsystem3

Interaction

Inte

ract

ion Interaction


• Measurements about the local subsystem• Reachable set of other subsystems

Future reachable set of state variable

��

Feasibility ProblemFeasibility Problem

At stage k-1, the quantified constraints are satisfied with 9 k-1(j):

for all v(j) ∈ 9 k-1(j), j = 0,1,…,N-1

x(j+1)=f(x(j),h(x(j),θ k-1*(j)),v(j)) ∈ ; (j)

g(x(j),h(x(j),h(x(j),θ k-1*(j)),v(j)),h(x(j),θ k-1*(j)),v(j)) ≤ 0~ ~

~

~

~

~~

~ ~

~ ~

At stage k, we check these conditions with 9 k(j):

for all v(j) ∈ 9 k(j), j = 0,1,…,N-1

x(j+1)=f(x(j),h(x(j),θ k-1*(j)),v(j)) ∈ ; (j)

g(x(j),h(x(j),h(x(j),θ k-1*(j)),v(j)),h(x(j),θ k-1*(j)),v(j)) ≤ 0~ ~

~

~

~

~~

~ ~

~ ~

��

Requirements On Updating Uncertainty BoundsRequirements On Updating Uncertainty Bounds

Updating Uncertainty Bounds:

9k(j) ⊆ 9k-1(j+1), j = 0,1,…,N-2

9k(N-1) ⊆ 9

Initial Setting of Uncertainty Bounds:

90(j) = 9, j = 0,1,…,N-1

9k-1(j+1)

9k(j)

Updating Uncertainty Bounds:

9ik(j) = ∏;l (k+j |k-1), j = 0,1,…,N-1

Initial Setting :

9i0(j) = ∏;l, j = 0,1,…, N-1

^M

l=1l≠i

M

l=1l≠i

Full-StateCommunication

��

Requirements On Predicting Reachable SetsRequirements On Predicting Reachable Sets

Updating State Constraints:

;(k+j|k) ⊆ ; (k+j |k-1), j = 1, 2, …, N-1

;(k+N|k) ⊆ ;

Predicting Future Reachable Sets:

; (k+j |k) = {x | x = f (x’,h(x’,θk*(j)),v),x’∈; (k+j -1|k),v∈9k(j-1)}, j = 1,2,…,N

; (k|k) = {x(k)}

^

^ ^

^

^

^

Updating State Constraints:

;k(j) = ; (k+j |k-1), j = 1, 2, …, N-1

;k(N) = ;

Initial Setting:

;0(j) = ;, j = 1, 2, …, N

^

��

θ -Control Invariant Setsθ -Control Invariant Sets

θ-Control Invariant Set:

x ∈ 7x ⇒ f (x,h(x,θ7x),v) ∈ 7x for all v ∈ 9

End State Constraints:

; (N) = 7x

7x

The initial solution can be constructed from the previous solution.

θ k(j) = θ k-1*(j+1), j = 0,1,…,N-2 and θ k(N-1) = θ k-1*(N-1)

��

Min-Max Feedback DMPC (M2F-DMPC)Min-Max Feedback DMPC (M2F-DMPC)

min { max Ji (ΘI , Vi )}

whereΘi = {θi (0),θi (1),…,θi (N-1)}Vi = {vi (0),vi (1),…,vi (N-1)}vi (j) = [x1

T(j)…xi-1T(j) xi+1

T(j)…xMT (j)]T

s.t.xi (j+1) = fi (xi (j),ui (j),vi (j)), j = 0,1,…,N-1ui (j) = hi (xi (j),θi (j)), j = 0,1,…,N-1Θi ∈ Ξi

vi (j) ∈ 9i (j), j = 0,1,…,N-1

9i (j) = ∏ ;l (k+j |k-1)

xi (0) = xi0

For all vi (j) ∈ 9i (j), j = 0,1,…,N-2

xi (j+1) = fi (xi (j),hi (xi (j),θi (j)),vi (j)) ∈ ;i (k+j|k-1)

xi (N) = fi (xi (N-1),hi (xi (N-1),θi (N-1)),vi (N-1)) ∈ 7ix

gi (xi (j),hi (xi (j),θi (j)),vi (j)) ≤ 0

Θi Vi

~

^

~

^ ^ ^ ^

l =1l ≠ i

M

~

~

~

~

~

~

~ ~

~ ~

^

��

OutlineOutline


á Previous Work

• Approaches



– Current Results

• An Example

• Conclusion

• Future Work

��

Ellipsoidal ApproximationEllipsoidal Approximation

Ellipsoidal method is used to approximate

reachable sets:

; (k+j |k) = ε (q (k+j |k),Q (k+j |k))

q : the center of the ellipsoid;

Q : the configuration matrix of the ellipsoid.

~̂

Optimize the sum of the square of semi-axes

Use a single ellipsoid

~̂;

;

��

; (k+j |k-1)~̂

; (k+j |k-1)^

Set ApproximationSet Approximation

Since the set inclusion is not invariant under

set approximation, the following steps are

used to maintain set inclusion relationship.

Step 1: Let j = 0 and calculate the set

approximation ; (k|k) of the set ; (k|k) such

that ; (k|k) ⊆ ; (k|k-1);

Step 2: Let j = j+1 and calculate the set

approximation ; (k+j |k) of the set ; (k+j |k)

Step 3: If ; (k+j |k) ⊆ ; (k+j |k-1), let ; (k+j |k) =

; (k+j |k-1) or ; (k+N |k-1) = 7 ;

Step 4: If j = N, stop; otherwise, go to Step 2.

^^

^ ^

^

^

^

^

^ ^

^

~

~ ~

~

~~

~

~

~

; (k+j |k)^

; (k+j |k)~̂

; (k+j |k)~̂

��

OutlineOutline


á Previous Work

• Approaches



– Current Results

• An Example

• Conclusion

• Future Work

��

Current Results – Bounded StabilityCurrent Results – Bounded Stability

M2P(C)

If the M2P(C) optimization is feasible at the first step k = 0, then it

is feasible at all control steps k ≥ 0 with updated state constraints

and uncertainty bounds. And furthermore, x(k) ∈ 7, i.e. z(k) ∈*-1(7x), k = N, N+1, …

M2F-DMPC with Set Approximation

With the set approximation method, if M2F-DMPC is feasible at

the first step k = 0 in all agents, it is feasible at all control steps k

≥ 0. Furthermore, the state of the whole system z goes into

∩*i-1(7i

x).

��

Current Results – Asymptotical StabilityCurrent Results – Asymptotical Stability

In LTI systems, we use the following method to update the end

constraint

αs7i0, k = sN, s ≥ 0 is an integer;

7ik =

7ik-1, otherwise. 7i

07iN7i2N

M2F-DMPC with Updated End Constraints

If M2F-DMPC in LTI is feasible at the first step k = 0 in all agents,

it is feasible at all control steps k ≥ 0 with the updated end constraints. Furthermore, the system goes to ∩*i

-1(0) asymptotically.

��

OutlineOutline


á Previous Work

á Approaches

• An Example

• Future Work

��

Control Object

• To keep the solenoids at the equilibrium points

State Variables

• xi1: the position of the i th solenoid;

• xi2: the velocity of the i th solenoid.

Control Variables

• ui: the magnetic force applied to the i th solenoid

An ExampleAn Example

y1 y2 y3

k

m m m

k k k

��

Study CasesStudy Cases

• Parameterized Open-Loop Control Policy in Optimization:

u(j) = u0(j)

• Parameterized Feedback Control Policy in Optimization:

u( j ) = K( j )x( j )+u0( j )

��

Simulation Results(Parameterized Open-Loop Policy)

Simulation Results(Parameterized Open-Loop Policy)

-1 -0.8-0.6-0.4-0.2 0 0.2 0.4 0.6 0.8 1-4

-3

-2

-1

0

1

2

3

4Predicted Reachable Set χ1(1|0)

x1,1

x 1,2

-1 -0.8-0.6-0.4-0.2 0 0.2 0.4 0.6 0.8 1-4

-3

-2

-1

0

1

2

3


x3,1

x 3,2

-1 -0.8-0.6-0.4-0.2 0 0.2 0.4 0.6 0.8 1-4

-3

-2

-1

0

1

2

3


x2,1x 2

,2

^ ^ ^

-1 -0.8-0.6-0.4-0.2 0 0.2 0.4 0.6 0.8 1-4

-3

-2

-1

0

1

2

3


x1,1

x 1,2

-1 -0.8-0.6-0.4-0.2 0 0.2 0.4 0.6 0.8 1-4

-3

-2

-1

0

1

2

3


x3,1

x 3,2

-1 -0.8-0.6-0.4-0.2 0 0.2 0.4 0.6 0.8 1-4

-3

-2

-1

0

1

2

3


x2,1

x 2,2

Predicted Reachable Set State Constraint

^^^

��

Simulation Results(Parameterized Feedback Policy)

Simulation Results(Parameterized Feedback Policy)

-0.5-0.4-0.3-0.2-0.1 0 0.1 0.2 0.3 0.4 0.5-4

-3

-2

-1

0

1

2

3

4Predicted Reachable Set By Controller 1

x11

x 21

χ1(k+2|k)χ1(k+1|k)

-0.5-0.4-0.3-0.2-0.1 0 0.1 0.2 0.3 0.4 0.5-4

-3

-2

-1

0

1

2

3


x12

x 22

χ2(k+2|k)χ2(k+1|k)

-0.5-0.4-0.3-0.2-0.1 0 0.1 0.2 0.3 0.4 0.5-4

-3

-2

-1

0

1

2

3


x13

x 23

χ3(k+2|k)χ3(k+1|k)

0 2 4 6 8 10 12 14 16 18 20-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3Response Of The Sub-System 1

Time Step

x11(k)

x21(k)

0 2 4 6 8 10 12 14 16 18 20-0.4

-0.3

-0.2

-0.1

0

0.1

0.2


Time Step

x13(k)

x23(k)

0 2 4 6 8 10 12 14 16 18 20-0.4

-0.3

-0.2

-0.1

0

0.1

0.2


Time Step

x12(k)

x22(k)

state trajectories of three sub-system

state predictions of three sub-system

^^

^^

^^

��

OutlineOutline


á Previous Work

á Approaches

á An Example

á Conclusion

• Future Work

��

Contributions to DateContributions to Date

• Framework

• Communication Scheme for Coordination

• Method to Handle Interaction

• Formulation of MPC Optimization to Achieve Coordination

• Approaches

• Method for Reducing the Conservativeness in Control Actions

• Feasibility of the Control Decision

• Stability of the System

• Set Approximation Method

✟ Computation of Control Invariant Set

✟ Uncertainties in State Estimation

✟ Information Compensation in Asynchronous Control

✟ Reachable Set Computation in Hybrid Systems

• Demonstrations

��

Problems Considered In ResearchProblems Considered In Research

• Computational Issues

– θ-Control Invariant Set

– Set-Membership Estimation

• Asynchronous Agents

– Information Compensation

• Hybrid systems

– Reachable Set Prediction

• Applications

– Automatic Generation Control (ABC Power System)

– Plant-Wide Process Control (The Tennessee Eastman Plant)

��

Time ScheduleTime Schedule

DefenseWriting Thesis

AnalyzingScheme Performance

DesigningSimulation Code

Defining Concrete Example

CombingMembership Estimation

ComputingInvariant Set

PredictingReachable Set for Hybrid System

Dec. 2002Dec. 2001

ComputationalIssue

HybridSystems

Application

Thesis

Time Horizon

CompensatingAsynchronous Information

Apr. 2001 Aug. 2002

AsynchronousAgents

distributed coordination in multi-agent control systems ...jwinter/phd_proposal_slides.pdf ·...

Documents