optimal control of linear systems with limited control...

12
IEEE TRANSACTIONS ON CONTROL OF NETWORK SYSTEMS, VOL. 5, NO. 3, SEPTEMBER 2018 1275 Optimal Control of Linear Systems With Limited Control Actions: Threshold-Based Event-Triggered Control Burak Demirel , Euhanna Ghadimi, Daniel E. Quevedo , and Mikael Johansson Abstract—We consider a finite-horizon linear-quadratic optimal control problem where only a limited number of control messages are allowed for sending from the controller to the actuator. To restrict the number of control actions computed and transmitted by the controller, we employ a threshold-based event-triggering mechanism that decides whether or not a control message needs to be calculated and delivered. Due to the nature of threshold-based event-triggering algorithms, finding the optimal control sequence requires minimizing a quadratic cost function over a nonconvex domain. In this paper, we first provide an exact solution to this nonconvex problem by solving an exponential number of quadratic programs. To reduce computational complexity, we then propose two efficient heuristic algorithms based on greedy search and the alternating direction method of multipliers technique. Later, we consider a receding horizon control strategy for linear systems controlled by event-triggered controllers, and we further provide a complete stability analysis of receding horizon control that uses finite-horizon optimization in the proposed class. Numerical exam- ples testify to the viability of the presented design technique. Index Terms—Event-triggered control, linear systems, optimal control, receding horizon control. I. INTRODUCTION T HE PROBLEM of determining optimal control policies for discrete-time linear systems has been extensively investi- gated in the literature; see, for example, [1]–[3]. The standard optimal control problem assumes that an unlimited number of control actions is available at the actuator. Although this as- sumption is valid for many applications, it does not hold for some specific scenarios where communication and computation resources become scarce; for example: 1) control systems with constrained actuation resources [4]–[6]; 2) control systems with shared processor resources [7]; or 3) information exchange over a shared communication channel [8]. Manuscript received October 10, 2016; revised January 17, 2017 and March 21, 2017; accepted April 17, 2017. Date of publication May 4, 2017; date of current version September 17, 2018. Recommended by Associate Editor P. Cheng. (Corresponding author: Burak Demirel.) B. Demirel and D. E. Quevedo are with the Chair of Automatic Control (EIM- E), Paderborn University, Paderborn 33098, Germany (e-mail:,burak.demirel@ protonmail.com; [email protected]). E. Ghadimi is with Huawei Technologies Sweden AB, Kista SE-164 94, Sweden (e-mail:, [email protected]). M. Johansson is with , ACCESS Linnaeus CenterSchool of Electrical Engi- neering, , KTH Royal Institute of Technology, Stockholm, SE 10044, Sweden (e-mail:, [email protected]). Digital Object Identifier 10.1109/TCNS.2017.2701003 To reduce the communication burden between the controller and the actuator, Imer and Bas ¸ar [9] introduced a constraint on the number of control actions while designing optimal con- trol policies for linear scalar systems. Later, Bommannavar and Bas ¸ar [10] and Shi et al. [11] extended the work of [9] to a class of higher-order systems. The problem, proposed in [9], can be also formulated as a cardinality-constrained linear- quadratic control problem. The introduction of cardinality con- straint changes the standard linear-quadratic control problem from a quadratic programming (QP) to a mixed-integer QP prob- lem. As has been proved in [12], it is an NP-complete problem. Therefore, Gao and Lie [13] provided an efficient branch-and- bound algorithm to compute the optimal control sequence. Al- ternatively, a convex relaxation of the nonconvex cardinality constraint based on a 1 -regularized 2 optimization was con- sidered in the literature; see, for example, [14]–[17]. Although the 1 -norm regularization of the cost function is an effective way of promoting sparse solutions, no performance guarantees can be provided in terms of the original cost function due to the transformed cost function. Event- and self-triggered control systems have been broadly used in the literature to reduce the amount of communication between the controller and the actuator while guaranteeing at- tainable control performance; see, for example, [18]–[22]. As distinct from sparse control techniques as mentioned earlier, the event- and self-triggered control require the design of both a feedback controller that computes control actions and a trig- gering mechanism that determines when the control input has to be updated. A vast majority of the works in the literature first designed a controller without considering sparsity constraint, and then in the subsequent design phase, they developed the triggering mechanism for a fixed controller. In contrast, another line of research concentrates on the design of feedback control law while respecting a predefined triggering condition. The design of event- and self-triggered algorithms can be extremely beneficial in the context of model predictive control (MPC) strategies; see, for example, [23]–[31]. MPC is a con- trol scheme that solves a finite-horizon optimal control prob- lem at each sampling instant and only applies the first element of the resulting optimal control input trajectories. The use of event-triggered algorithms, therefore, reduces the frequency of solving optimization problems and transmitting control actions from the controller to the actuator, and, consequently, saves com- putational and communication resources. Lehmann et al. [26] 2325-5870 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

Upload: others

Post on 05-Jun-2020

9 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Optimal Control of Linear Systems With Limited Control ...controlsystems.upb.de/fileadmin/Files_Gruppe/pdfs/artikel/demgha18a.pdfOptimal Control of Linear Systems With Limited Control

IEEE TRANSACTIONS ON CONTROL OF NETWORK SYSTEMS, VOL. 5, NO. 3, SEPTEMBER 2018 1275

Optimal Control of Linear Systems WithLimited Control Actions: Threshold-Based

Event-Triggered ControlBurak Demirel , Euhanna Ghadimi, Daniel E. Quevedo , and Mikael Johansson

Abstract—We consider a finite-horizon linear-quadratic optimalcontrol problem where only a limited number of control messagesare allowed for sending from the controller to the actuator. Torestrict the number of control actions computed and transmittedby the controller, we employ a threshold-based event-triggeringmechanism that decides whether or not a control message needs tobe calculated and delivered. Due to the nature of threshold-basedevent-triggering algorithms, finding the optimal control sequencerequires minimizing a quadratic cost function over a nonconvexdomain. In this paper, we first provide an exact solution to thisnonconvex problem by solving an exponential number of quadraticprograms. To reduce computational complexity, we then proposetwo efficient heuristic algorithms based on greedy search andthe alternating direction method of multipliers technique. Later,we consider a receding horizon control strategy for linear systemscontrolled by event-triggered controllers, and we further providea complete stability analysis of receding horizon control that usesfinite-horizon optimization in the proposed class. Numerical exam-ples testify to the viability of the presented design technique.

Index Terms—Event-triggered control, linear systems, optimalcontrol, receding horizon control.

I. INTRODUCTION

THE PROBLEM of determining optimal control policies fordiscrete-time linear systems has been extensively investi-

gated in the literature; see, for example, [1]–[3]. The standardoptimal control problem assumes that an unlimited number ofcontrol actions is available at the actuator. Although this as-sumption is valid for many applications, it does not hold forsome specific scenarios where communication and computationresources become scarce; for example: 1) control systems withconstrained actuation resources [4]–[6]; 2) control systems withshared processor resources [7]; or 3) information exchange overa shared communication channel [8].

Manuscript received October 10, 2016; revised January 17, 2017 and March21, 2017; accepted April 17, 2017. Date of publication May 4, 2017; date ofcurrent version September 17, 2018. Recommended by Associate Editor P.Cheng. (Corresponding author: Burak Demirel.)

B. Demirel and D. E. Quevedo are with the Chair of Automatic Control (EIM-E), Paderborn University, Paderborn 33098, Germany (e-mail:,[email protected]; [email protected]).

E. Ghadimi is with Huawei Technologies Sweden AB, Kista SE-164 94,Sweden (e-mail:,[email protected]).

M. Johansson is with , ACCESS Linnaeus CenterSchool of Electrical Engi-neering, , KTH Royal Institute of Technology, Stockholm, SE 10044, Sweden(e-mail:,[email protected]).

Digital Object Identifier 10.1109/TCNS.2017.2701003

To reduce the communication burden between the controllerand the actuator, Imer and Basar [9] introduced a constraint onthe number of control actions while designing optimal con-trol policies for linear scalar systems. Later, Bommannavarand Basar [10] and Shi et al. [11] extended the work of [9]to a class of higher-order systems. The problem, proposedin [9], can be also formulated as a cardinality-constrained linear-quadratic control problem. The introduction of cardinality con-straint changes the standard linear-quadratic control problemfrom a quadratic programming (QP) to a mixed-integer QP prob-lem. As has been proved in [12], it is an NP-complete problem.Therefore, Gao and Lie [13] provided an efficient branch-and-bound algorithm to compute the optimal control sequence. Al-ternatively, a convex relaxation of the nonconvex cardinalityconstraint based on a �1-regularized �2 optimization was con-sidered in the literature; see, for example, [14]–[17]. Althoughthe �1-norm regularization of the cost function is an effectiveway of promoting sparse solutions, no performance guaranteescan be provided in terms of the original cost function due to thetransformed cost function.

Event- and self-triggered control systems have been broadlyused in the literature to reduce the amount of communicationbetween the controller and the actuator while guaranteeing at-tainable control performance; see, for example, [18]–[22]. Asdistinct from sparse control techniques as mentioned earlier,the event- and self-triggered control require the design of botha feedback controller that computes control actions and a trig-gering mechanism that determines when the control input hasto be updated. A vast majority of the works in the literature firstdesigned a controller without considering sparsity constraint,and then in the subsequent design phase, they developed thetriggering mechanism for a fixed controller. In contrast, anotherline of research concentrates on the design of feedback controllaw while respecting a predefined triggering condition.

The design of event- and self-triggered algorithms can beextremely beneficial in the context of model predictive control(MPC) strategies; see, for example, [23]–[31]. MPC is a con-trol scheme that solves a finite-horizon optimal control prob-lem at each sampling instant and only applies the first elementof the resulting optimal control input trajectories. The use ofevent-triggered algorithms, therefore, reduces the frequency ofsolving optimization problems and transmitting control actionsfrom the controller to the actuator, and, consequently, saves com-putational and communication resources. Lehmann et al. [26]

2325-5870 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

Page 2: Optimal Control of Linear Systems With Limited Control ...controlsystems.upb.de/fileadmin/Files_Gruppe/pdfs/artikel/demgha18a.pdfOptimal Control of Linear Systems With Limited Control

1276 IEEE TRANSACTIONS ON CONTROL OF NETWORK SYSTEMS, VOL. 5, NO. 3, SEPTEMBER 2018

proposed an event-based strategy for discrete-time systems un-der additive, bounded disturbances. The controller only com-putes a new control command whenever the difference be-tween actual state and predictive state exceeds a threshold.Sijs et al. [23] combined state estimation with MPC in orderto design an event-based estimation framework. The authorsof [24] and [25] combined MPC with event-triggered samplingstrategies based on the input-to-state concept. In contrast tothe aforecited works, the authors of [19]–[21] studied infinitehorizon quadratic cost. All works discussed before focus ondiscrete-time linear/nonlinear systems; however, there are alsoa substantial number of works, which consider continuous-timesystems. For instance, the authors of [30] and [32] investigatedthe stability of event-based MPC algorithms for continuous-timenonlinear systems yet they did not consider disturbance. Later,Li and Shi [31] studied the MPC problem for continuous-timenonlinear systems subject to bounded disturbances.

A. Contributions

In this paper, we formulate a finite-horizon optimal controlproblem where a threshold-based event-triggering algorithmdictates the communication between the controller and the actu-ator. Then, we propose various algorithms to compute a controlaction sequence that provides optimal and suboptimal solutionsfor this problem. The control problem, in general, is hard tosolve due to two main reasons: 1) it has the combinatorial na-ture since the decisions are binary variables (i.e., transmit ornot transmit), and 2) introduction of a threshold-based trigger-ing condition leads to optimizing a convex cost function overa nonconvex domain. The main contributions of this paper arethreefold as follows.

1) We show that the optimal solution of the control problem,mentioned above, can be determined via solving a set ofQP problems.

2) We provide an efficient heuristic algorithm based on alter-nating direction method of multiplier (ADMM) to designa control input sequence that provides a suboptimal solu-tion for the finite-horizon optimal event-triggered controlproblem.

3) We describe the receding horizon implementation of theevent-triggered control algorithm, including a proof ofpractical stability.

B. Outline

The remainder of this paper is organized as follows.Section II formulates the event-triggered finite-horizon LQ con-trol problem and introduces assumptions. Section III providesa simple procedure for designing optimal control laws to mini-mize the linear-quadratic cost function while Section IV presentstwo heuristic methods that usually achieve tolerable suboptimalperformance while significantly reducing computational com-plexity. In Section V, a receding horizon control scheme withan event-triggered algorithm is presented. Section VI demon-strates the effectiveness and advantages of the presented ap-proach while Section VII concludes this paper.

Fig. 1. Event-triggered control system with the process G, the actuator A, thesensor S, and the controller K.

C. Notation

We write N for the positive integers, N0 for N ∪ {0}, andR for the real numbers. Let Rn

�0 denote the set of non-negativereal vectors of dimension n, and Rn be the set of real vectors ofdimension n. Vectors are written in bold lowercase letters (e.g.,u and v) and matrices in capital letters (e.g., A and B). If uand v are two vectors in Rn , the notation u ≤ v correspondsto component-wise inequality. The set of all real symmetricpositive semidefinite matrices of dimension n is denoted bySn�0 . We let 0n be the n–dimensional column vectors of all

zeros, 1n be the vectors of all ones. The Kronecker product oftwo matrices (e.g., A and B) is denoted by A⊗B. For any givenx ∈ Rn , the �∞–norm is defined by ‖ x ‖∞= max1≤i≤n |xi |.For a square matrix A, λmax(A) denotes its largest eigenvaluein magnitude. The notation{xk}k∈K stands for {x(k) : k ∈ K},where K ⊆ N0 . The power set of any set N , written as P(N ),is the set of all subsets of N , including the empty set and Nitself. The cardinality of a set is denoted by |N |.

II. FINITE-HORIZON OPTIMAL EVENT-TRIGGERED CONTROL

We consider the feedback control loop, depicted in Fig. 1.The dynamics of the physical plant G can be described by thediscrete-time linear time-invariant system

G : x(t + 1) = Ax(t) + Bu(t) , x(0) = x0 (1)

where x(t) ∈ Rn is the state variable at time instant t, u(t) ∈Rm is the control input at time instant t, A ∈ Rn×n andB ∈ Rn×m are system matrices of appropriate dimensions, andx0 ∈ Rn is the given initial condition. The pair (A,B) is as-sumed to be stabilizable, and A is not necessarily Schur stable.

In this paper, we assume that the sensor S takes periodic noise-free samples of the plant state x(t) and transmits these samplesto the controller node. The controller K is event-triggered, i.e.,it computes new control commands and transmits them to theactuator A only at times when x(t) ∈ S � Rn \C0 with

C0 � {x ∈ Rn :‖ x ‖∞< ε} (2)

for a given threshold ε > 0. In case x(t) ∈ C0 , the controllerdoes not compute any new control actions, and the actuatorinput to the plant is set to zero

u(t) = 0m ∀t ∈ T (3)

Page 3: Optimal Control of Linear Systems With Limited Control ...controlsystems.upb.de/fileadmin/Files_Gruppe/pdfs/artikel/demgha18a.pdfOptimal Control of Linear Systems With Limited Control

DEMIREL et al.: OPTIMAL CONTROL OF LINEAR SYSTEMS WITH LIMITED CONTROL ACTIONS 1277

where

T � {t ∈ N0 ; t < N : x(t) ∈ C0} . (4)

It is worth noting that T denotes a set of time instants at which nocontrol computations are needed and, in turn, the plant runs openloop. Notice that T is not a given set of variables; in contrast, it isgenerated by the initial state x0 and the tentative control actionsu(t) for all t ∈ {0, · · · ,T− 1}. Hence, to determine the set T ,it is necessary to compute the tentative control actions u(t). Inother words, the set of time instants T is strongly coupled to thetentative control inputs u(t).

Throughout this paper, we aim at designing an admissibleoptimal control sequence π = {u(0),u(1), · · · ,u(N − 1)} tominimize the quadratic cost function

J(x(0),π) = x�(N)Px(N)

+N−1∑

t=0

(x�(t)Qx(t) + u�(t)Ru(t)

)(5)

where the matrices Q and P are symmetric and positive semidef-inite while R is symmetric and positive definite. Then, we con-sider the constrained finite-time optimal control problem

J�(x(0)) : minimizeπ

J(x(0),π)

subject to x(t + 1) = Ax(t) + Bu(t)

∀t ∈ {0, · · · , N − 1}x(0) = x0

u(t) = 0m ∀t ∈ T . (6)

Recall that the set of time instants, at which no computations(and also transmissions) are needed, i.e., T , is defined in (4).

Remark 1: Note that the event-triggered control problem,described in Section II, is a particular class of hybrid systems.One may, therefore, convert it into an equivalent mixed logicaldynamical (MLD) system that represents the system by using ablend of linear and binary constraints on the original variablesand some auxiliary variables; see, for example, [33], [34]. Inthis paper, instead of using the MLD system formulation, wewill exploit geometric properties of the problem at hand, whichallows us to propose an optimization problem for optimal finite-horizon control and establish stability result for its recedinghorizon implementation.

Discussion: Although the problem (6) resembles thecardinality-constrained optimal control problem (proposedin [9]–[17]), there is a fundamental difference between them,which stems from the question whether scheduling is an ex-ogenous input or autonomously generated? While solving thecardinality-constrained control problem, one needs to optimizeboth control action and scheduling sequence, which are stronglycoupled to each other in the optimization process. On the otherhand, the event-triggered control systems are switched systemswith internally forced switchings. In these problems, schedulingsequences are generated implicitly based on the evolution of thestate x(t) and the control signal u(t). The major difficulty ofthis problem is that scheduling depends on the particular initial

Fig. 2. Decision tree diagram. Denote T � {t ∈ N0 ; t < N : x(t) ∈ C0}.This means that T represents a set of time instances at which a new controlsignal does not need to be computed and transmitted from the controller to theactuator. In the figure, dark colored circles represent the case x(t) ∈ C0 forany t ∈ T whereas light colored circles represent the case x(t) ∈ Rn \C0 forany t /∈ T . For instance, when N = 4, the set T gets values in P({1, 2, 3}) ={∅, {1}, {2}, {3}, {1, 2}, {1, 3}, {2, 3}, {1, 2, 3}

}.

condition x0 and the tentative control input u(t), and cannot beexplicitly determined unless a specific control signal was given;see the survey paper [35] and references therein.

III. COMPUTATION OF OPTIMAL CONTROL ACTIONS

In this section, we concentrate on finding the control inputsequence π that solves the finite-horizon optimal control prob-lem, proposed in (6). Therefore, we present a framework whichis based on dividing the nonconvex domain into convex sub-domains. This is a simple yet effective procedure to follow;however, the number of convex optimization problems, whichneeds to be solved, grows exponentially in the length of thecontrol horizon.

The optimal control problem (6) is hard to solve due to therestriction on the control space. Nevertheless, without loss ofgenerality, it is possible to convert this problem to a set ofoptimization problems of the form

J�T (x(0)) : minimize

πJ(x(0),π)

subject to x(t + 1) = Ax(t) + Bu(t)

∀t ∈ {0, 1, · · · , N − 1}x(t) ∈ C0 ∀t ∈ Tx(t) ∈ Rn \C0 ∀t /∈ Tu(t) = 0m ∀t ∈ Tx(0) = x0 (7)

where T ∈ P(N ) with N = {1, 2, · · · , N − 1}. To obtain theoptimal solution of the problem (6), it is necessary to solve theproblem (7) for all subsets of the power set of N , i.e., P(N ),and select the one providing the lowest cost; see Fig. 2. Moreprecisely, this can be written as

J�(x(0)) : minimizeT ∈P(N )

J�T (x(0)) . (8)

Page 4: Optimal Control of Linear Systems With Limited Control ...controlsystems.upb.de/fileadmin/Files_Gruppe/pdfs/artikel/demgha18a.pdfOptimal Control of Linear Systems With Limited Control

1278 IEEE TRANSACTIONS ON CONTROL OF NETWORK SYSTEMS, VOL. 5, NO. 3, SEPTEMBER 2018

The number of subproblems, which are needed to be solved,grows exponentially with the control horizon N (i.e., |P(N )| =2N−1). Bear in mind that since these subproblems are indepen-dent of each other, they can be also solved in parallel. In additionto its combinatorial nature, the problem (7) requires the opti-mization of a convex function over a nonconvex domain. In theliterature, there exist some available tools to solve the convexobjective nonconvex optimization; see, for example, [36]–[38].

As the feasible region S can be expressed as a finite union ofpolyhedra, the disjunctive formulation can be applied. In par-ticular, assuming n-dimensional problem instance (6), its feasi-bility region can be divided into 2n + 1 disjunctive polyhedra.1

Following the construction above, we write

S �2n⋃

p=1

Cp (9)

where n is the state dimension, and Cp ⊂ Rn are full-dimensional polyhedra, i.e.,

Cp �{x ∈ Rn : Tpx ≤ d

}

for some Tp ∈ R(2n−1)×n and d ∈ R2n−1 .We define a piece-wise constant function of time σ(t), which

takes on values in {0, 1, · · · , 2n} and whose value p determines,at each time t ∈ {0, · · · ,T− 1}, the state variable x(t) tobelong to the interior of the polyhedron Cp . We divide the non-convex set S into convex subsets Cp with p ∈ {0, 1, . . . , 2n}and, at each time step t ∈ {0, . . . ,T− 1}, we choose only oneactive set. The switching signal

σ(t) =

{0 if t ∈ Tp otherwise

(10)

gives a sequence Σ = {σ(0), . . . , σ(N − 1)}. Then, for givenT and Σ, we rewrite the optimization problem (7) as

J�Σ(x(0)) : minimize

πJ(x(0),π)

subject to x(t + 1) = Ax(t) + Bu(t)

∀t ∈ {0, 1, . . . , N − 1}x(t) ∈ Cσ (t) ∀σ(t) ∈ Σ

u(t) = 0m ∀t ∈ Tx(0) = x0 . (11)

It is necessary to solve all possible combination of prob-lem (11) to determine the global optimal solution. Particularly,the problem (6) can be solved as a sequence of (2n + 1)N−1 QPproblems. Hence, we have the following series of optimizationproblems:

J�(x(0)) : minimizeΣ∈{0,··· ,2n}N −1

J�Σ(x(0)) . (12)

It is worth noting that even though the number of QP sub-problems grows exponentially with N and n, all these problems

1See Fig. 3 for an example of two dimensional problem where each disjunctiveset is shown in different colors.

Fig. 3. Nonconvex set S � R2 \C0 can be divided into four convex regions,which are illustrated by different colors. For a given scheduling sequence,i.e., T = {6, 7}, the optimal control inputs can be computed by solving theoptimization problem (11) for all possible combinations of five convex sets. ForN = 7, finding optimal control actions requires solving 15 625 problems, andselecting the minimum control cost among 2650 feasible solutions. In Fig. 3,the blue curve represents the state trajectory for Σ1 = {4, 4, 4, 1, 1, 0, 0}, thered curve represents the state trajectory for Σ2 = {4, 1, 1, 2, 2, 0, 0}, and themagenta curve denotes the state trajectory for Σ3 = {4, 1, 2, 2, 3, 0, 0}.

are independent of each other; therefore, one can parallelize theproblems and reduce the computation time.

Corollary 1: The global optimal solution of the control prob-lem (6) can be computed by solving a set of convex QP problemsof the form (11), in parallel, and selecting the solution that pro-vides the lowest control cost among all feasible solutions.

The following examples should give the reader a better un-derstanding of the problem.

Example 1: Let n = 2 and S = R2 \C0 with C0 = {(x1 , x2)∈ R2 : −ε < x1 < ε,−ε < x2 < ε}. The nonconvex set S canbe divided into the following four convex subsets:

C1 = {(x1 , x2) ∈ R2 : x1 ≥ x2 , x1 ≥ −x2 , x1 ≥ ε}

C2 = {(x1 , x2) ∈ R2 : x1 ≤ x2 , x1 ≥ −x2 , x2 ≥ ε}

C3 = {(x1 , x2) ∈ R2 : x1 ≤ x2 , x1 ≤ −x2 , x1 ≤ −ε}

C4 = {(x1 , x2) ∈ R2 : x1 ≥ x2 , x1 ≤ −x2 , x2 ≤ −ε}

which are illustrated by different colors in Fig. 3 . The optimalcontrol inputs can be computed by solving quadratic programsfor different combinations of convex sets. In order to computeoptimal control inputs, it is necessary to solve 5N−1 convex QPproblems.

Example 2: Let us consider the second-order system

x(t + 1) =

[0.9 0.20.8 1.5

]x(t) +

[0.60.8

]u(t) (13)

with the initial condition x0 = [ 0 −1 ]�. The event-triggeredcontroller transmits control messages if the �∞-norm of the state

Page 5: Optimal Control of Linear Systems With Limited Control ...controlsystems.upb.de/fileadmin/Files_Gruppe/pdfs/artikel/demgha18a.pdfOptimal Control of Linear Systems With Limited Control

DEMIREL et al.: OPTIMAL CONTROL OF LINEAR SYSTEMS WITH LIMITED CONTROL ACTIONS 1279

variable x(t) is greater than ε = 0.25. The performance indicesare given by Q = diag{2, 2} and R = 5. The control horizonis chosen as N = 7. The state trajectories, depicted in Fig. 3,can be obtained via solving the optimization problem (11) forgiven sets Σ1 = {4, 4, 4, 1, 1, 0, 0}, Σ2 = {4, 1, 1, 2, 2, 0, 0},and Σ3 = {4, 1, 2, 2, 2, 0, 0}. Fig. 3 shows the state trajecto-ries for two different sets. The set Σ1 leads to the global optimalsolution.

Example 1 demonstrates that obtaining the optimal solutionto the finite-horizon optimal control problem, mentioned earlier,requires solving an exponential number of QPs. Therefore, inthe next section, we propose heuristic algorithms that signifi-cantly reduce computational complexity, by compromising theoptimality guarantees slightly.

IV. COMPUTATION OF SUBOPTIMAL CONTROL ACTIONS

In this section, we propose two heuristic algorithms that usu-ally achieve a low control loss while significantly decreasingthe computation time. We, first, develop a simple algorithm thattakes the initial state x0 as input and finds a suitable sequenceΣ in a greedy manner by increasing the horizon of the problemfrom 1 to N . Second, we apply an algorithm based on ADMMto compute the heuristic scheduling sequence Σ. This sequenceis then used to solve the disjunctive problem (11) and obtain agood approximate control sequence π. Both techniques are ofpolynomial time complexity, hence, reducing the computationalefforts significantly. Numerical investigations (see Section VI)indicate that these heuristic algorithms usually provide close tooptimal solutions.

A. Greedy Heuristic Method

We construct an intuitive greedy algorithm to solve the opti-mal control problem, proposed in (6), efficiently. As discussedearlier, given a feasible switching sequence Σ, a correspondingoptimal control problem can be formulated as a QP problem ofthe form (11). One, then, solves the combinatorial problem (12)to find the optimal switch sequence Σ� , thereby solving theoptimal control problem (6).

A good way to deal with the combinatorial nature of thistype of algorithms and reduce the computational complexity isto employ a greedy algorithm, which solves a global problemby making a series of locally optimal decisions. Greedy algo-rithms do not necessarily provide optimal solutions; however,they might find local optimal solutions, which can be used toestimate a globally optimal solution, in a reasonable time.

Our proposed greedy algorithm works as follows. Since x0 isknown, we can compute corresponding switching signal σ(0).Then, we set Σ = {σ(0), p} for all p ∈ {0, 1, . . . , 2n} and solve2n + 1 number of QPs of the form (11) to obtain the opti-mal switch signal σ(1). The same procedure then repeats fork = 2, . . . , N − 1, where at each step k, we solve 2n + 1 timesthe QP problem (11) to find the best σ(k) keeping the pre-viously found scheduling sequence Σ = {σ(0), . . . , σ(k − 1)}unchanged.

Algorithm 1 provides the detailed steps of discussed greedyheuristic method. Algorithm 1 executes (2n + 1)(N − 1) num-

ber of QPs compared to (2n + 1)N−1 as in optimal controlprocedure. As a result, it runs in polynomial time.

B. Heuristic Method Based on ADMM

In this section, we develop a computationally efficient methodfor solving (6). Our approach is based on, ADMM, a well-developed technique to solve large-scale disciplined prob-lems [39]. The idea of utilizing ADMM as a heuristic to solvenonconvex problems has been recently considered in litera-ture [38]–[41]. In particular, [38] and [40] employ ADMM toapproximately solve problems with convex costs and nonconvexconstraints. The approximate ADMM solutions are improved bymultiplicity of random initial starts and a number of local searchmethods applied to ADMM solutions.

In this paper, using some of the techniques introduced above,we first apply ADMM to original nonconvex problem to achievean intermediate solution. This algorithm, for a general noncon-vex problem, does not necessarily converge; however, whenit does, the corresponding solution gives a good initial guessfor finding the optimal solution to this problem. In the secondphase of our heuristic method, we perform a polishing techniquewith the aim of achieving a feasible and hopefully closer to theoptimal solution (see [38] for an overview of polishing tech-niques). Our polishing idea is based on disjunctive programmingproblem (11) with the fixed event index T and trajectory se-quence Σ.

We, first cast (6) to the ADMM form. Essentially, we have

minimizez

12z�Fz + I(w)

Gz = h

z = y (14)

where z collects the state and control components; i.e.,z � [x(0)�, . . . ,x(N)�,u(0)�, . . . ,u(N − 1)�]�, F repre-sents the positive definite Hessian of the quadratic cost. Thatis F � blkdiag(Qs,Rs) with Qs � IN +1 ⊗Q and Rs � IN ⊗R. Moreover, I(y) is the indicator function enforcing the

Page 6: Optimal Control of Linear Systems With Limited Control ...controlsystems.upb.de/fileadmin/Files_Gruppe/pdfs/artikel/demgha18a.pdfOptimal Control of Linear Systems With Limited Control

1280 IEEE TRANSACTIONS ON CONTROL OF NETWORK SYSTEMS, VOL. 5, NO. 3, SEPTEMBER 2018

event-triggering control law; that is

I(y) =

{∞ if (16) holds

0 otherwise(15)

{∃t ∈ {0, . . . , N − 1}|y(t + 1) ∈ C0and y(t + N + 2) �= 0m} (16)

where (16) detects the constraint violation in (6) in terms of hav-ing x(t) ∈ C0 while corresponding u(t) �= 0m . The constraintsin (14) are twofold. The first constraint in (14) describes the LTIsystem dynamics (1). Here G ∈ RnN×(n(N +1)+mN ) have thefollowing i = {1, . . . , N}, j = {1, . . . 2N + 1}- blocks

Gij =

⎧⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎩

Ai−1 for i = 1 . . . N, j = 1−In j = i and j > 1

Ai+N−jB if j ≥ N + 2 and i + N ≥ j

0 otherwise.

Furthermore, h ∈ Rn(N +1)+mN � [x(0)�,0�n , . . . ,0�m ]�. Thelast constraint in (14) is of consensus-type to ensure that non-convex constraint in (6) is asymptotically satisfied. After formu-lating the problem, each iteration of ADMM algorithm consistsof (see [39] for a complete treatment of the subject):

zk+1/2 = argminz

⎝12z�Fz

2

∥∥∥∥∥

[G

I

]z −

[0I

]zk −

[h

0

]+ uk

∥∥∥∥∥

2⎞

zk+1 = Π(zk+1/2 +[0 I

]uk )

uk+1 = uk +

[G

I

]zk+1/2 −

[0I

]zk −

[h

0

]. (17)

Here, k denotes the iteration counter, ρ > 0 is the step-size(penalty) parameter, and Π denotes the projection onto noncon-vex constraint and is given by

Πi(x) =

{0 if i ≥ N + 2 and x(i−N − 1) ∈ C0x(i) otherwise.

(18)When the cost function in hand is convex and the constraintsset is closed and convex then the ADMM algorithm convergesto the optimal solution of the problem. In our case, since I(y)is nonconvex the ADMM algorithm (17) may not converge tooptimal point. Moreover, examples can be found in which (17)does not even converge into a feasible point. To further improvethe ADMM solution, we utilize an additional convex problemto polish the intermediate results. That is, we restrict the searchspace to a convex set that includes the ADMM solution pat-tern. Let z � [x(0)�, . . . , x(N)�, u(0)�, . . . , u(N − 1)�]�

be the output of the ADMM algorithm and the sets Σ andT � {t ∈ N0 ; t < N : x(t) ∈ C0} denote the trajectory se-quence associated with [x�1 , . . . , x�N ]� and the restriction indexof x intoC0 , respectively. Then our heuristic polishing procedure

formulates (11) with T = T and Σ = Σ. Algorithm 2 providesa summary of our heuristic method. The input variables includethe error-tolerance threshold εtol , randomized initial state z0 ,zbest, and fbest to store the final solution.

A few comments related to Algorithm 2 are in order. First,in contrast to the ADMM algorithms for convex problems thatconverge to the optimum for all positive range of the step-sizeparameter ρ, the stability of the ADMM algorithm for noncon-vex problems is sensitive to the choice of ρ. Moreover, simi-lar to the convex case, the convergence speed is also affectedby the choice of ρ. A variety of techniques including properstep-size selection, over relaxation, constraint matrix precondi-tioning and caching can be employed to accelerate the ADMMprocedure (17) (see [42], [43] for a reference on the topic).

Second, one can utilize an adaptive iteration count procedureto improve the probability of finding feasible solution to (6). Theprocedure works as the following. Run the ADMM algorithm fora fixed number of iterations and then check if the disjunctive QPproblem has feasible solution. If not, then increase the number ofiterations (e.g., double it) and repeat the procedure (run ADMMand disjunctive QP) until finding a feasible solution or reachingto a point where the current ADMM solution does not improvethe previous one in terms of constraint violations and control lossvalue. Third, the variable z0 is initialized randomly using a nor-mal distribution. In general, repeating the ADMM algorithm fora multiple of initial points may improve the approximate solu-tion at the cost of extra computational complexity (as suggestedin, for example, [40]). However, in our application, we did notexperience significant performance improvement by repeatingthe ADMM algorithm (steps 1–6 in Algorithm 2) with differ-ent initializations. One reason could be that in Algorithm 2, weemploy the solution trajectories Σ and T based on the ADMMsolution z and not z itself in the disjunctive problem (11).

Fourth, using caching and LDL� matrix factorizationtechniques in the first-step of (17) [39], each z-update inAlgorithm 2 costs on the order of O((n(N + 1) + mN)2) fordense P matrices. Furthermore, if we assume N ≥ max{n,m}then each ADMM iteration costs on the order of O(N 2). Thismeans that overall cost of the ADMM algorithm is of O(KN 2)where K is the number of ADMM iterations. In our applica-tion, with proper algorithm parameter selection, the ADMM

Page 7: Optimal Control of Linear Systems With Limited Control ...controlsystems.upb.de/fileadmin/Files_Gruppe/pdfs/artikel/demgha18a.pdfOptimal Control of Linear Systems With Limited Control

DEMIREL et al.: OPTIMAL CONTROL OF LINEAR SYSTEMS WITH LIMITED CONTROL ACTIONS 1281

algorithm converges almost always within a few hundreds ofiterations (see the results presented in Section VI).

Finally, the computational complexity of the disjunctivestep in our heuristic method falls into the generic complex-ity of convex QPs on the order of O((|T |2n + (N − |T |)(2n− 1))2) [44].

V. RECEDING HORIZON CONTROL

Except a few special cases, finding a solution to an infinitehorizon optimal control problem is not possible. Receding hori-zon control is an alternative scheme to infinite horizon problemthat repeatedly requires solving a constrained optimization prob-lem. At each time instant t ≥ 0, starting at the current state x(t)(assuming that a full measurement of the state x(t) is availableat the current time t ∈ N0), the following cost function:

J(x(t),π�N−1t ) = x�(t + N | t))Px(t + N | t))︸ ︷︷ ︸

Vf (x(t+N |t))

+t+N−1∑

k=t

(x�(k | t)Qx(k | t) + u�(k | t)Ru(k | t)

)︸ ︷︷ ︸

�(x(k |t),u(k |t))

(19)

subject to system dynamics and constraints involving states andcontrols is minimized over a finite horizon N . Here, the function� defines the stage cost, and Vf defines the terminal cost. Denotethe minimizing control sequence, which is a function of thecurrent state x(t), by

π�N−1t =

[u�(t | t)�, · · · ,u�(t + N − 1 | t)�

]�

then the control applied to the plant at time t ≥ 0 is the firstelement of this sequence, that is

u(t) =[I 0 · · · 0

]π�N−1

t . (20)

Time is then stepped forward one instant, and the procedure de-scribed above is repeated for another N -step ahead optimizationhorizon.

It is worth noting that, due to the fundamental limitation of theoptimization problem (6), the state x(t) never converges to theorigin if the system is unstable. It can only converge into a setincluding the origin. The following result shows the existenceof this set.

Theorem 1: Suppose that Dμ � {x ∈ Rn : ‖ x ‖∞≤ μ} isa neighborhood of the origin, where

μ �√

κλmax(A�PA + Q)nε2

λ2min(Q)

(21)

for some κ > 0. Then, the system (1) with event-triggering con-straints (3) and (4) is uniformly practically asymptotically sta-ble, i.e., limt→∞ ‖ x(t) ‖∞≤ μ.

Theorem 1 establishes practical stability of the system (1)with event-triggering constraints (3) and (4). It shows that ifprovided conditions are met, then the plant state will be ulti-mately bounded in an �∞-norm ball of radius μ. It is worthnoting that, as the other stability results, which use Lyapunovtechniques, this bound will not be tight. Next, we would like to

Fig. 4. Evolution of the state trajectories of the system (13) under the recedinghorizon strategy (20) and constraints (2)–(4) for a set of initial conditions x0 .

exemplify the convergence of the state x(t) to the set Dμ froma set of initial conditions x0 .

Example 3: Consider the second-order system, givenby (13), with initial conditions x0 = 1.2[ sin(πk

6 ) cos(πk6 ) ]�

for all 0 ≤ k ≤ 12. The controller transmits a control messagewhenever the event-triggering condition, i.e., ‖ x(t) ‖∞> 0.25,holds. The performance indices are chosen as Q = diag{2, 2}and R = 5 while the prediction horizon is set to N = 6. Fig. 4shows the state trajectories of the receding horizon implementa-tion of the event-triggered control system, described by (1)–(4),for different initial conditions x0 . As can be seen in the samefigure, the number of transmitted control commands depends onthe initial condition.

VI. NUMERICAL EXAMPLES

A. Performance of Suboptimal Event-Triggered ControlActions

To illustrate performance benefits of the proposed heuristicalgorithms, we consider a third-order plant with the followingstate-space representation:

x(t + 1) =

⎢⎣0.53 −2.17 0.62

0.22 −0.06 0.51

−0.92 −1.01 1.69

⎥⎦x(t) +

⎢⎣0.4

0.7

0.9

⎥⎦u(t)

+ Bww(t) . (22)

In the first experiment, we assume that there is no disturbanceacting on the plant, i.e., Bw = 0. Our aim is to minimize (5)with Q = P = diag{2, 2, 2} and R = 5, and the horizon lengthN = 8. The initial condition is chosen as

x0 =[sin θ cos φ sin θ sin φ cos θ

]�

where 0 ≤ θ ≤ π and 0 ≤ φ ≤ π. To perform simulations, weselected 577 different data points, which are equidistant overa half spherical surface. We evaluated the performance of thegreedy and the ADMM-based heuristic algorithms for all these

Page 8: Optimal Control of Linear Systems With Limited Control ...controlsystems.upb.de/fileadmin/Files_Gruppe/pdfs/artikel/demgha18a.pdfOptimal Control of Linear Systems With Limited Control

1282 IEEE TRANSACTIONS ON CONTROL OF NETWORK SYSTEMS, VOL. 5, NO. 3, SEPTEMBER 2018

Fig. 5. Relative error (Jadm m − J� )/J� and control-action cardinality dif-ference of the (suboptimal) ADMM-based heuristic solution from the optimalcost for 577 problem instances of third-order plant model (22).

Fig. 6. Relative error (Jgreedy − J� )/J� and control-action cardinality dif-ference of the greedy solution from the optimal cost for 577 problem instancesof third-order plant model (22).

initial conditions. For each data point on the half sphere, we run823,543 number of QPs to obtain the global optimal solution.In total, we run more than 475 million number of QPs in ourevaluation. We used a linux machine with 32 cores in Intel Corei7 Extreme Edition 980X to solve these QPs in parallel threads.Within this computing resources, it takes approximately 6 daysto perform simulations for one problem setup.

To compare the (true) optimal control performance withour heuristic method, we performed Algorithm 2 on the sameproblem set. In particular, we evaluated three sets of prob-lem with ε = 0.2, ε = 0.4, and ε = 0.6. It is generally ex-pected that the control problem becomes more challenging withincreasing the event-threshold ε. We run the ADMM-basedheuristic algorithm with adaptive iteration count proceduredescribed in Section IV-B. Moreover, we picked ρ = 9.8,ρ = 5.8, and ρ = 6.9 respectively, for the aforementioned prob-lem scenarios. For cases ε = 0.2 and ε = 0.4, the ADMMmethod produced always feasible solutions for maximum300 number of ADMM-iterations. However, when ε = 0.6and ρ = 6.9, the method created infeasible solutions for twoinitial conditions: x0 = [0.3036, 0.2330, 0.9239]� and x0 =[0.6830, 0.1830, 0.7071]�. To compute feasible solutions, weneeded to pick a different ρ for these initial conditions.

Fig. 5 shows the outcome of the comparison. As it shows,for ε = 0.2, the ADMM-based heuristic algorithm is able to

find near optimal solutions (with relative error less than 5%)in almost all cases, and for ε = 0.4 and ε = 0.6, the algo-rithm finds solutions with relative error less than 5% in morethan 80% of problem instances. The average optimality gap(Jadmm − J�)/J� for the three cases were 0.0035, 0.0237, and0.0716, respectively. Finally, by comparing the cardinality ofcontrol event index T for the optimal solutions and our ADMMmethod, one concludes that the heuristic method in more than50% of cases finds similar-sparse solution as the optimum ones,and in the rest of cases it tends to find more sparse controlactions.

We also compared the true optimal control performance witha greedy heuristic method, shown in Algorithm 1, for threesets of problems, mentioned above, with ε = 0.2, ε = 0.4, andε = 0.6. Fig. 6 demonstrates the outcome of the comparison.The greedy algorithm detected 22, 90, and 135 optimal solu-tions of the aforementioned 577 problems, respectively. Also,it computed the optimal solution with relative error less than5% in more than 65% of problems at hand for all three problemsetups. It is important to note that the greedy heuristic methodalways provided feasible solutions for all problem cases. The av-erage optimality gaps (Jgreedy − J�)/J� in all three cases were0.0514, 0.0780, and 0.0826, respectively. As compared to theADMM-based heuristic method, the optimality gaps becamelarger.

B. Receding Horizon Control Implementation

We consider the open-loop unstable discrete-time system rep-resented by (22) with Bw = 0. The prediction horizon is chosento be N = 6 with performance indices Q = P = diag{2, 2, 2}and R = 5. The event-threshold ε for the event-triggered reced-ing horizon implementation is set to 0.4. Fig. 7 demonstratesthe time responses of the event-triggered receding horizon con-trol system for the initial condition x0 = [0,

√2/2, −

√2/2]�.

Here, two different approaches for computing control input tra-jectories are considered: the exhaustive search algorithm andthe heuristic ADMM algorithm with ρ = 4.8. As can be seenin Fig. 7, while the state trajectories associated to two controlapproaches deviate from one another (except for the few firstsampling instants that they match together), the correspond-ing control inputs follow a rather similar sparsity pattern. Theevent-triggered receding horizon control via exhaustive searchalgorithm uses the communication channel 14 times over the50 sampling instants, which results in a reduction of 72% incommunication and computational burden, whereas the event-triggered receding horizon control via ADMM algorithm usesthe channel 16 times over the 50 sampling instants, whichleads to a reduction of 68% in communication and computa-tional burden. For the aforementioned initial value, the exhaus-tive search achieves a better control cost: 65.42 compared to77.72 as of the control cost of the ADMM-based heuristic. How-ever, this conclusion does not hold in general. In particular, weexperienced cases where starting from a different initial value,the receding horizon controller derived by exhaustive searchperforms worse than the one based on ADMM-based heuristic.

Page 9: Optimal Control of Linear Systems With Limited Control ...controlsystems.upb.de/fileadmin/Files_Gruppe/pdfs/artikel/demgha18a.pdfOptimal Control of Linear Systems With Limited Control

DEMIREL et al.: OPTIMAL CONTROL OF LINEAR SYSTEMS WITH LIMITED CONTROL ACTIONS 1283

Fig. 7. States and input versus time for the event-triggered model predictivecontroller presented in Section VI-B. In this figure, x1 (t), x2 (t) and x3 (t)represent state variables whereas u(t) and uadm m (t) denote the optimal andthe heuristic control inputs computed based on the exhaustive search and theADMM algorithm, respectively.

C. Tradeoff Between Communication and Control Costs

We consider the case where a random disturbance w(t) ∈ Rn ,which is modeled by a discrete-time zero-mean Gaussian whiteprocess with unity covariance, is acting on the plant charac-terized by the injection matrix Bw = diag{1, 1, 1}. The initialcondition is chosen as

x0 =[r sin θ cos φ r sin θ sinφ r cos θ

]�

where θ ∼ Uni(0, π), φ ∼ Uni(0, 2π), and r ∼ Uni(0, 5). Theprocess noise w(t) is independent of the initial condition x0 .The control performance is measured by a quadratic function

J∞ =1

500

499∑

t=0

(x�(t)Qx(t) + u�(t)Ru(t)

)(23)

while the communication rate is measured by

π∞ =1

500

499∑

t=0

11{‖x(t)‖∞≥0} . (24)

Fig. 8. Comparison of the control performance and the communication fre-quency for different event-threshold values ε > 0 (shown in gray scale). Themarked curve with is obtained by averaging 2500 Monte Carlo simulations forthe horizon length 500 samples with the process noise {w(t)}t∈N0

and theinitial condition x0 generated randomly.

The visualization of the tradeoff between the control loss andthe communication rate is demonstrated in Fig. 8. Different datapoints on the curve correspond to different event-threshold val-ues ranging from 0.5 to 4.0, and the control loss (23) and thecommunication rate (24) are evaluated via Monte Carlo simula-tions. Note that the communication rate decreases dramaticallywith an increased control loss as the threshold ε varies between0 and 2.25 (dark colors). For ε > 2.25 (lighter color), both quan-tities become less sensitive to changes in the threshold value.We can also identify 0 ≤ ε ≤ 1.5 as a particularly attractiveregion where a large decrease in the communication rate can beobtained for a small loss in control performance.

VII. CONCLUSION

In this paper, we considered a feedback control system wherean event-triggering rule dictates the communication between thecontroller and the actuator. We investigated the optimal controlproblem with an additional constraint on the event-triggeredcommunication. While this optimization problem, in general, isnonconvex, we used a disjunctive programming formulation toobtain the optimal solution via solving an exponential number ofquadratic programs. Then, we proposed a heuristic algorithms toreduce the computational efforts while achieving an acceptableperformance. Later, we provided a complete stability analysis ofreceding horizon control that uses a finite-horizon optimizationin the proposed class. Our numerical study confirmed the theory.

Further work is necessary to investigate the effects of theprocess noise on the optimality of the proposed method.

APPENDIX

We begin with the definition of the practical stability of con-trol systems.

Definition 1: (practical stability [45]). The system (1) is saidto be uniformly practically asymptotically stable in A ⊆ Rn ifA is a positively invariant set for (1) and if there exist a KL-function β, and a nonnegative constant δ ≥ 0 such that

‖ x(t) ‖≤ β(‖ x0 ‖, t) + δ .

Page 10: Optimal Control of Linear Systems With Limited Control ...controlsystems.upb.de/fileadmin/Files_Gruppe/pdfs/artikel/demgha18a.pdfOptimal Control of Linear Systems With Limited Control

1284 IEEE TRANSACTIONS ON CONTROL OF NETWORK SYSTEMS, VOL. 5, NO. 3, SEPTEMBER 2018

Definition 2: (practical-Lyapunov function [45]). A functionV : Rn ← R≥0 is said to be a practical-Lyapunov function inA for the system (1) ifA is a positively invariant set and if thereexist a compact set, Ω ⊆ A, neighborhood of the origin, someK∞-functions α1 , α2 and α3 , and some constants d1 , d2 ≥ 0,such that

V (x) ≥ α1(‖ x ‖) ∀x ∈ A (25)

V (x) ≤ α2(‖ x ‖) + d1 ∀x ∈ Ω (26)

V (x+)− V (x) ≤ −α3(‖ x ‖) + d2 ∀x ∈ A (27)

where x and x+ denoting consecutive state variables in (1).Proof of Theorem 1: To prove the practical stability of the

system (1) with a set of control actions π� = {u�(t | t), . . . ,u�(t + N − 1 | t)}, which fulfills constraints (3) and (4), weanalyze the value function

V �N (x(t)) � min

πJ(x(t),π)

where J(x(t),π) is as seen in (19). We borrow the shiftedsequence approach, described in [46], and we use a feasiblecontrol sequence, i.e., π = {u�(t + 1 | t), · · · ,u�(t + N − 1 |t), u} and its associated value function VN (x(t + 1), π). Bythe optimality property, we obtain the following bound (withx(t | t) = x(t) and u�(t | t) = u(t)):

V �N (x(t + 1))− V �

N (x(t)) ≤ VN (x(t + 1), π)− V �N (x(t))

= −�(x(t),u(t)) + Vf

(Ax(t + N | t) + Bu

)

− Vf (x(t + N | t)) + �(x(t + N | t), u) (28)

for all x(t) ∈ Rn . We now investigate the quantity of

ΔVf (t) + �(x(t + N | t), u)

where ΔVf (t) � Vf

(Ax(t + N | t) + Bu

)− Vf (x(t + N |

t)). Based on x(t + N | t), there are following two possibil-ities.

1) Suppose that ‖ x(t + N | t) ‖∞> ε, then for any feasi-ble control law, i.e., u = Kx(t + N | t), the followinginequality holds:

ΔVf (t) + �(x(t + N | t), u) =

x(t + N | t)�(A�K PAK − P + Q∗

)x(t + N | t) < 0

(29)

where AK = A + BK (i.e., Schur) and Q∗ = Q +K�RK. From (28) and (29), it follows that:

V �N (x(t + 1))− V �

N (x(t)) < 0 .

2) Suppose that ‖ x(t + N | t) ‖∞≤ ε, then u = 0. Hence

ΔVf (t) + �(x(t + N | t), u)

= x(t + N | t)�(A�PA− P + Q

)x(t + N | t)

≤ x(t + N | t)�(A�PA + Q

)x(t + N | t)

≤ λmax(A�PA + Q

)‖ x(t + N | t) ‖22

≤ λmax(A�PA + Q

)n ‖ x(t + N | t) ‖2∞

≤ λmax(A�PA + Q

)nε2

where in third inequality, we used the relation ‖ v ‖2≤√n ‖ v ‖∞ for any v ∈ Rn ; see [47].

As a result, we conclude that, for any x(t + N | t) ∈ Rn , thefollowing inequality holds:

ΔVf (t) + �(x(t + N | t), u) ≤ η (30)

where η � λmax(A�PA + Q

)nε2 . Inserting (30) and a1 ‖

x(t) ‖22≤ �(x(t),u(t)) where a1 � λmin(Q) into (28) yields

V �N (x(t + 1))− V �

N (x(t)) ≤ −λmin(Q) ‖ x(t) ‖22 +η. (31)

Now, let the binary variable δ(t) ∈ {0, 1}, for each time instantt, represent the transmission of the control action as follows:

δ(t) =

{0 if ‖ x(t) ‖∞≤ ε

1 otherwise.(32)

For any feasible control sequence, i.e., π′ = {Kx(t), . . . ,Kx(t + N − 1 | t)}, there exists an associated scheduling se-quence, denoted by Δ(t) = {δ(t), . . . , δ(t + N − 1)}. Apply-ing these feasible control inputs, the upper bound of V �

N (x)becomes

V �N (x(t)) ≤ x�(t)SΔ (t)x(t) ≤ λmax(SΔ (t)) ‖ x(t) ‖22

≤ a3 ‖ x(t) ‖22 (33)

where a3 � maxΔ∈S λmax(SΔ ) = κ > 0 and

SΔ � Qδ(0) +N−1∑

i=1

i−1∏

j=0

A�δ(j )Qδ(j )Aδ(j ) +N−1∏

k=0

A�δ(k)QAδ(k)

with Qδ(i) � δ(i)Q∗ + (1− δ(i))Q and Aδ(i) � δ(i)AK +(1− δ(i))A for all i ∈ {0, · · · , N − 1}. Here,S denotes a set ofall possible feasible schedules generated by a stabilizing controlsequence π′.

To obtain the lower bound, we have x�(t)Qx(t) ≤ V �N (x(t))

and hence a2 ‖ x(t) ‖22≤ V �N (x(t)) where a2 � λmin(Q).

From (31) and (33) we establish the following relation:

V �N (x(t + 1)) ≤

(1− a2

a3

)V �

N (x(t)) + η (34)

which from (33) and a2 ‖ x(t) ‖22≤ V �N (x(t)) it yields

‖ x(t + 1) ‖22≤ γa3

a2‖ x(t) ‖22 +

η

a2(35)

with γ � 1− a2/a3. Note that since 0 < a2 ≤ a3 , it follows

that 0 ≤ γ < 1. Therefore, by iterating (34), it is possible to

Page 11: Optimal Control of Linear Systems With Limited Control ...controlsystems.upb.de/fileadmin/Files_Gruppe/pdfs/artikel/demgha18a.pdfOptimal Control of Linear Systems With Limited Control

DEMIREL et al.: OPTIMAL CONTROL OF LINEAR SYSTEMS WITH LIMITED CONTROL ACTIONS 1285

exponentially bound the state evolution via

‖ x(t) ‖22≤ γt a3

a2‖ x(0) ‖22 +

1− γt

1− γ

η

a2.

Using the norm inequality ‖ v ‖∞≤‖ v ‖2≤√

n ‖ v ‖∞ for anyv ∈ Rn (see [47]), we get

‖ x(t) ‖2∞≤ γtna3

a2‖ x(0) ‖2∞ +

1− γt

1− γ

η

a2.

Thus, limt→∞‖ x(t) ‖∞ ≤ μ with μ given in (21). The proof is

complete. �

REFERENCES

[1] B. D. O. Anderson and J. B. Moore, Optimal Control: Linear QuadraticMethods. Englewood Cliffs, NJ, USA: Prentice-Hall, 1990.

[2] D. P. Bertsekas, Dynamic Programming and Optimal Control. Belmont,MA, USA: Athena Scientific, 2000.

[3] K. J. Astrom, Introduction to Stochastic Control Theory. New York, NY,USA: Dover, 2006.

[4] M. Gallieri and J. M. Maciejowski, “�asso MPC: Smart regulation ofoveractuated systems,” in Proc. Amer. Control Conf., 2012, pp. 1217–1222.

[5] E. N. Hartley, M. Gallieri, and J. M. Maciejowski, “Terminal spacecraftrendezvous and capture with LASSO model predictive control,” Int. J.Control, vol. 86, no. 11, pp. 2104–2113, Aug. 2013.

[6] M. Chyba, S. Grammatico, V. T. Huynh, J. Marriott, B. Piccoli, and R. N.Smith, “Reducing actuator switchings for motion control for autonomousunderwater vehicle,” in Proc. Amer. Control Conf., 2013, pp. 1406–1411.

[7] V. Gupta, “On a control algorithm for time-varying processor availability,”in Proc. Hybrid Syst. Control Comput. Conf., 2010, pp. 81–90.

[8] B. Demirel, V. Gupta, D. E. Quevedo, and M. Johansson, “Thresholdoptimization of event-triggered multi-loop control systems,” in Proc. 13thInt. Workshop Discrete Event Syst., 2016, pp. 203–210.

[9] O. C. Imer and T. Basar, “Optimal control with limited controls,” in Proc.Amer. Control Conf., 2006, pp. 298–303.

[10] P. Bommannavar and T. Basar, “Optimal control with limited controlactions and lossy transmissions,” in Proc. 47th IEEE Conf. Decis. Control,2008, pp. 2032–2037.

[11] L. Shi, Y. Yuan, and J. Chen, “Finite horizon LQR control with lim-ited controller-system communication,” IEEE Trans. Autom. Control,vol. 58, no. 7, pp. 1815–1841, Jul. 2013.

[12] J. Gao, “Cardinality constrained discrete-time linear-quadratic control,”Master philosophy thesis, Chin. Univ. Hong Kong, Hong Kong, Jul. 2005.

[13] J. Gao and D. Li, “Cardinality constrained linear-quadratic optimal con-trol,” IEEE Trans. Autom. Control, vol. 56, no. 8, pp. 1936–1941, Aug.2011.

[14] M. Nagahara and D. E. Quevedo, “Sparse representations for packe-tized predictive networked control,” in Proc. IFAC World Congr., 2011,pp. 84–89.

[15] M. Gallieri and J. M. Maciejowski, “Stabilising terminal cost and terminalcontroller for �asso -MPC: Enhanced optimality and region of attraction,”in Proc. Eur. Control Conf., 2013, pp. 524–529.

[16] M. Nagahara, D. E. Quevedo, and J. Østergaard, “Sparse packetized pre-dictive control networked control over erasure channels,” IEEE Trans.Autom. Control, vol. 59, no. 7, pp. 1899–1905, Jul. 2014.

[17] R. P. Aguilera, R. Delgado, D. Dolz, and J. C. Aguero, “QuadraticMPC with �0 -input constraint,” in Proc. 19th IFAC World Congr., 2014,pp. 10888–10893.

[18] K. J. Astrom and B. M. Bernhardsson, “Comparison of riemann andlebesgue sampling for first order stochastic systems,” in Proc. 41st IEEEConf. Decis. Control, 2002, pp. 2011–2016.

[19] D. Antunes and W. P. M. H. Heemels, “Rollout event-triggered control:Beyond periodic control performance,” IEEE Trans. Autom. Control, vol.59, no. 12, pp. 3296–3311, Dec. 2014.

[20] T. M. P. Gommans, D. Antunes, T. Donkers, P. Tabuada, and W. P. M. H.Heemels, “Self-triggered linear quadratic control,” Automatica, vol. 50,pp. 1279–1287, 2014.

[21] T. M. P. Gommans and W. P. M. H. Heemels, “Resource-aware MPC forconstrained nonlinear systems: A self-triggered control approach,” Syst.Control Lett., vol. 79, pp. 59–67, May 2015.

[22] B. Demirel, V. Gupta, D. E. Quevedo, and M. Johansson, “On the trade-off between communication and control cost in event-triggered dead-beat

control,” IEEE Trans. Autom. Control, vol. 62, no. 6, pp. 2973–2980, Jun.2017.

[23] J. Sijs, M. Lazar, and W. P. M. H. Heemels, “On integration of event-basedestimation and robust MPC in a feedback loop,” in Proc. 13th ACM Int.Conf. Hybrid Syst.: Comput. Control, 2010, pp. 31–40.

[24] A. Eqtami, D. V. Dimarogonas, and K. J. Kyriakopoulos, “Event-triggeredcontrol for discrete-time systems,” in Proc. Amer. Control Conf., 2010,pp. 4719–4724.

[25] A. Eqtami, D. V. Dimarogonas, and K. J. Kyriakopoulos, “Event-triggeredstrategies for decentralized model predictive controllers,” in Proc. 18thIFAC World Congr., 2011, pp. 10068–10073.

[26] D. Lehmann, E. Henriksson, and K. H. Johansson, “Event-triggered modelpredictive control of discrete-time linear systems subject to disturbances,”in Proc. Eur. Control Conf., 2013, pp. 1156–1161.

[27] E. Henriksson, D. E. Quevedo, E. G. W. Peters, H. Sandberg, andK. H. Johansson, “Multiple-loop self-triggered model predictive controlfor network scheduling and control,” IEEE Trans. Control Syst. Technol.,vol. 23, no. 6, pp. 2167–2181, Nov. 2015.

[28] N. He and D. Shi, “Event-based robust sampled-data model predic-tive control: A non-monotonic Lyapunov function approach,” IEEETrans. Circuits Syst. I, Reg. Papers, vol. 62, no. 10, pp. 2555–2564,Oct. 2015.

[29] F. D. Brunner, W. P. M. H. Heemels, and F. Allgower, “Robust self-triggered MPC for constrained linear systems: A tube-based approach,”Automatica, vol. 72, pp. 73–83, Oct. 2016.

[30] P. Varutti, B. Kern, T. Faulwasser, and R. Findeisen, “Event-based modelpredictive control for networked control systems,” in Proc. 48th IEEEConf. Decis. Control 28th China Control Conf., 2009, pp. 567–572.

[31] H. Li and Y. Shi, “Event-triggered robust model predictive control ofcontinuous-time nonlinear systems,” Automatica, vol. 50, no. 5, pp. 1507–1513, May 2014.

[32] P. Varutti and R. Findeisen, “Event-based NMPC for networked controlsystems over UDP-like communication channels,” in Proc. Amer. ControlConf., 2011, pp. 3166–3171.

[33] A. Bemporad, G. Ferrari-Trecate, and M. Morari, “Observability and con-trollability of piecewise affine and hybrid systems,” IEEE Trans. Autom.Control, vol. 45, no. 10, pp. 1864–1876, Oct. 2000.

[34] W. P. M. H. Heemels, B. D. Schutter, and A. Bemporad, “Equivalenceof hybrid dynamical models,” Automatica, vol. 37, no. 7, pp. 1085–1091,Jul. 2001.

[35] F. Zhu and P. J. Antsaklis, “Optimal control of hybrid switched systems:A brief survey,” Discrete Event Dyn. Syst., vol. 25, no. 3, pp. 345–364,May 2014.

[36] A. Michalka, “Cutting planes for convex objective nonconvex optimiza-tion,” Ph.D. dissertation, , Columbia Univ., New York, NY, USA, 2013.

[37] D. Bienstock and A. Michalka, “Cutting-planes for optimization of convexfunctions over nonconvex sets,” SIAM J. Optimiz., vol. 24, no. 2, pp. 643–677, 2014.

[38] S. Diamond, R. Takapoui, and S. P. Boyd, “A general system for heuristicminimization of convex functions over non-convex sets,” Optimiz. Meth.Softw., pp. 1–29, 2017, doi: 10.1080/10556788.2017.1304548.

[39] S. P. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, “Distributedoptimization and statistical learning via the alternating direction methodof multipliers,” Found. Trends Mach. Learn., vol. 3, no. 1, pp. 1–122,2011.

[40] R. Takapoui, N. Moehle, S. P. Boyd, and A. Bemporad, “A simple effectiveheuristic for embedded mixed-integer quadratic programming,” in Proc.Amer. Control Conf., 2016, pp. 5619–5625.

[41] N. Derbinsky, J. Bento, V. Elser, and J. S. Yedidia, “An improved three-weight message-passing algorithm,” 2013, arXiv:1305.1961.

[42] E. Ghadimi, A. Teixeira, I. Shames, and M. Johansson, “Optimal parame-ter selection for the alternating direction method of multipliers (ADMM):Quadratic problems,” IEEE Trans. Autom. Control, vol. 60, no. 3,pp. 644–658, Mar. 2015.

[43] P. Giselsson and S. P. Boyd, “Linear convergence and metric selection inDouglas-Rachford splitting and ADMM,” IEEE Trans. Autom. Control,vol. 62, no. 2, pp. 532–544, Feb. 2017.

[44] S. P. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.:Cambridge Univ. Press, 2004.

[45] Z.-P. Jiang and Y. Wang, “A converse Lyapunov theorem for discrete-timesystems with disturbances,” Syst. Control Lett., vol. 45, pp. 49–58, Aug.2002.

[46] J. B. Rawlings and D. Q. Mayne, Model Predictive Control: Theory andDesign. Nob Hill Publishing, Madison, WI, USA, 2009.

[47] D. S. Bernstein, Matrix Mathematics. Princeton, NJ, USA: Princeton Univ.Press, 2009.

Page 12: Optimal Control of Linear Systems With Limited Control ...controlsystems.upb.de/fileadmin/Files_Gruppe/pdfs/artikel/demgha18a.pdfOptimal Control of Linear Systems With Limited Control

1286 IEEE TRANSACTIONS ON CONTROL OF NETWORK SYSTEMS, VOL. 5, NO. 3, SEPTEMBER 2018

Burak Demirel received the B.Sc. (Hons.) degreein mechanical engineering in 2007 and in controlengineering (double major) in 2009 (Hons.), andthe M.Sc. degree in mechatronics engineering fromIstanbul Technical University, Istanbul, Turkey, in2007 and 2009, and the Ph.D. degree in automaticcontrol from the School of Electrical Engineering,KTH Royal Institute of Technology, Stockholm, Swe-den, in 2015.

He is currently a Postdoctoral Researcher in theDepartment of Automatic Control, Paderborn Univer-

sity, Paderborn, Germany. His current research interests include cyber-physicalsystems, distributed algorithms, and mechatronic systems.

Dr. Demirel has served as a Guest Editor of IET Control Theory and Appli-cations: Special Issue on Resource-efficient Control in Cyber-Physical Systems.

Euhanna Ghadimi received the M.Sc. degree incomputer engineering from the University of Tehran,Tehran, Iran, in 2008, and the Ph.D. degree intelecommunications from KTH Royal Institute ofTechnology, Stockholm, Sweden, in 2015.

He was a Visiting Researcher at Delft Universityof Technology, Delft, the Netherlands, in 2012 andthe University of California, Berkeley, CA, USA, in2011. He is a Senior Researcher at Huawei Tech-nologies Sweden AB, Sweden. His research interestsinclude optimization, machine learning, and wirelessnetworks.

Dr. Ghadimi was a finalist for NecSys 2012 best paper award.

Daniel E. Quevedo received the Ingeniero Civil andM.Sc. degree in electronics engineering from the Uni-versidad Tecnica Federico Santa Marıa, ValparaısoChile, in 2000. He received the Ph.D. degree in elec-trical engineering from the University of Newcastle,Newcastle, Australia, NSW, Australia, in 2005.

He is the Chair of Automatic Control (Regelungs-und Automatisierungstechnik) at Paderborn Uni-versity, Paderborn, Germany. His research interestsare in control of networked systems and powerconverters.

Dr. Quevedo is an Associate Editor of the IEEE CONTROL SYSTEMS MAG-AZINE, Editor of the International Journal of Robust and Nonlinear Control,Steering Committee Member of the IEEE Internet of Things Initiative, andserves as Chair of the IEEE Control Systems Society Technical Committee onNetworks & Communication Systems. He was supported by a full scholarshipfrom the alumni association during his time at the Universidad Tecnica Fed-erico Santa Marıa and received several university-wide prizes upon graduating.He received the IEEE Conference on Decision and Control Best Student PaperAward in 2003 and was also a finalist in 2002. In 2009, he received a five-yearResearch Fellowship from the Australian Research Council.

Mikael Johansson received the M.Sc. and Ph.D. de-grees in electrical engineering from the University ofLund, Lund, Sweden, in 1994 and 1999, respectively.

He held postdoctoral positions at Stanford Uni-versity, Stanford, CA, USA, and the University ofCalifornia, Berkeley, CA, USA, Berkeley before join-ing KTH Royal Institute of Technology, Stockholm,Sweden, in 2002, where he now serves as a Full Pro-fessor. His research interests include distributed op-timization, wireless networking, and control. He haspublished one book and more than 100 papers, sev-

eral of which are ISI-highly cited.Dr. Johansson has served on the editorial board of Automatica and on the

program committee for conferences, such as IEEE CDC, IEEE Infocom, ACMSenSys, and played a leading role in several national and international researchprojects in control and communications.