ieee transactions on control systems ...manuscript received june 5, 2015; revised march 22, 2016;...

IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 25, NO. 3, MAY 2017 979

Control Engineering Methods for the Designof Robust Behavioral Treatments

Korkut Bekiroglu, Constantino Lagoa, Suzan A. Murphy, and Stephanie T. Lanza

Abstract— In this paper, a robust control approach is used toaddress the problem of adaptive behavioral treatment design.Human behavior (e.g., smoking and exercise) and reactionsto treatment are complex and depend on many unmeasurableexternal stimuli, some of which are unknown. Thus, it iscrucial to model human behavior over many subject responses.We propose a simple (low order) uncertain affine model subject touncertainties whose response covers the most probable behavioralresponses. The proposed model contains two different types ofuncertainties: uncertainty of the dynamics and external pertur-bations that patients face in their daily life. Once the uncertainmodel is defined, we demonstrate how least absolute shrinkageand selection operator (lasso) can be used as an identificationtool. The lasso algorithm provides a way to directly estimatea model subject to sparse perturbations. With this estimatedmodel, a robust control algorithm is developed, where one relieson the special structure of the uncertainty to develop efficientoptimization algorithms. This paper concludes by using theproposed algorithm in a numerical experiment that simulatestreatment for the urge to smoke.

Index Terms— Adaptive treatment design, adaptive-robustintervention, behavioral treatment design, min–max structuredrobust optimization, receding horizon control.

NOMENCLATURE

N Natural numbers.A Matrix in R

m×n .A ≥ 0 Positive semidefinite matrix.AT Transpose of the matrix A.A:,i i th column of matrix A.I Identity matrix in R

m×m .w Signal in L∞.

Manuscript received June 5, 2015; revised March 22, 2016; acceptedMay 28, 2016. Date of publication June 28, 2016; date of current versionApril 11, 2017. Manuscript received in final form June 7, 2016. This workwas supported in part by the National Institute on Drug Abuse underGrant P50 DA039838 and Grant P50 DA100075, in part by the NationalInstitutes of Health Clinical Center under Grant R01 AA023187 andGrant U54EB020404, and in part by the Center for Hierarchical Manufac-turing through the National Science Foundation under Grant CNS-1329422and Grant ECCS-1201973. Recommended by Associate Editor N. H. El-Farra.

K. Bekiroglu and C. Lagoa is with the Department of Electrical Engineering,The Methodology Center, The Pennsylvania State University, University Park,PA 16802 USA (e-mail: [email protected]; [email protected]).

S. A. Murphy is with the Quantitative Methodology Program, Institutefor Social Research, 2068, University of Michigan, Ann Arbor,MI 48106-1248 USA (e-mail: [email protected]).

S. T. Lanza is with the Department of Biobehavioural Health, TheMethodology Center, The Pennsylvania State University, University Park,PA 16802 USA (e-mail: [email protected]).

Color versions of one or more of the figures in this paper are availableonline at http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TCST.2016.2580661

wk kth element of signal w, wk ∈ R,and k = [1, 2, . . . ,∞].

w(k:k+K−1) Vector w(k:k+K−1) ∈ RK that is

segment of signal w: w(k:k+K−1) =[wk wk+1 . . . wk+K−1].

dim(w(k:k+K−1)) Dimension of vector w(k:k+K−1).diag(w(k:k+K−1)) Elements of vector w(k:k+K−1) ∈ R

K

to the diagonal of a RK×K matrix.

�y� Largest integer less than or equal to y.N (μ, σ 2) Standard Gaussian distribution with

mean μ and variance σ 2.‖w(k:k+K−1)‖p p-norm of vector w(k:k+K−1) that is:

‖w(k:k+K−1)‖p.= p(

∑k+K−1i=k wp

i )1/2.‖w(k:k+K−1)‖0 Number of the nonzero element of

vector w(k:k+K−1) that is‖w(k:k+K−1)‖0

.= #{i : wi �= 0}.

I. INTRODUCTION

PROLIFERATION of portable devices that collect patientinformation very frequently (i.e., intensive longitudinal

data) and enable timely treatment has opened the possibilityof developing effective personalized interventions [1]. Thesetypes of interventions can be behavioral or pharmacologicalor a combination according to the structure of the specificbehavioral problem [2]. This is currently being investigatedin smoking addiction, alcohol addiction, exercise behavior,and so on. Dynamical modeling of behavior is critical indeveloping control algorithms [3]–[6]. However, very littleattention has been paid to the fact that human behavior hasa significant amount of uncertainty and that this uncertaintyshould be addressed systematically. This paper provides anapproach for modeling patient behavior and designing a robustadaptive treatment that considers both the present state of thepatient and the probable perturbations to expected behavior.

Previous researchers have applied control concepts in behav-ioral research [3]–[5], [7]–[9], but the use of feedback is still anovel approach. In addition to feedback, additional constraintsfor patients’ limitations and measured disturbances can beeasily addressed in controller-design-based interventions [10].In this paper, we discuss a possible control engineering-based approach and explain how robust control tools canbe used to design robust adaptive treatments. To design arobust algorithm, the first step is to define the uncertaintiesobserved in the treatment responses. Therefore, we proposea class of uncertain affine models suitable for this task and

1063-6536 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

980 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 25, NO. 3, MAY 2017

discuss a way to identify such models using available tools,such as lasso [11]. The second step is to show that a robustcontrol algorithm can be developed that relies on the specialstructure of the uncertainty in the model to develop efficientoptimization algorithms.

Only simple affine models will be considered here, becausevery little is known about the structure of human behavior.Nevertheless, this approach can be extended to more complexmodels, such as nonlinear and time-varying models if one hasbetter knowledge about the dynamics of the human behavior.

Besides the problem of choosing the right model structure,behavioral data have other remarkable challenges. The col-lection of behavioral data is often done over long periods oftime and one cannot usually perform repeated experimentsto improve the quality of the data. As a result, unlike mostphysical systems, behavioral data are generally more noisy,incomplete, and inconsistent. Hence, designers of treatmentstypically work with an incomplete, noisy data set from multi-ple patients [12]–[15]. Multiple participants are used to attemptto capture a more complete data set such that the gaps in oneparticipant’s data are filled by another participant. Therefore,we also use data from several participants to determine a modelthat both is meaningful and can be used to design treatmentfor a specific group of individuals.

In this paper, we discuss a specific structure for modeluncertainty that can be used for modeling human behavior.When selecting a type of uncertainty, the objective is toaddress: 1) differences in the behavior of different participantsmodeled as white Gaussian noise and 2) sparse externalperturbations that can be used to model life events that affectthe response to treatment. With the model structure definedearlier, lasso [11] is used for parameter identification anduncertainty quantification because it provides a way to directlyestimate a model subject to sparse perturbations. Given thatthe models extracted from behavioral data usually have alarge amount of uncertainty, there is a need for feedback and,especially, robustness. Hence, in the second part of this paper,an algorithm is proposed for a controller design that is robustwith respect to the specific type of uncertainties considered.

The specific control design technique used in this paperis based on a robust model predictive controller (MPC)approach [16]. A computationally simple cost function isused, namely, a quadratic one, and the corresponding robustoptimization problem is formulated. In formulating the controlproblem, one considers all different combinations of treat-ments available. The resulting robust optimization problemis then solved by relying on results from robust optimiza-tion [17]–[19] and by exploiting the specific structure of theuncertainty.

Throughout this paper, the problem of designing a treatmentfor smoking urge is used as our example. Although the treat-ment is assumed as ON/OFF (apply treatment or no treatment)in this specific example, this is not always the case. Thereare situations where one not only has to decide when/whichtreatment to apply but also the dosage (see [20] for a moredetailed explanation of characteristics of behavioral problems).The presented method can be easily modified to accommodatedifferent types (behavioral, pharmacological, or combined) and

levels (dosage) of treatments. Note that in this example, theterm treatment is used to refer to an intervention where textmessages are sent to the subject.

A. Previous Work

Adaptive interventions are sequences of treatments thatare adapted and readapted to individual circumstances andbehaviors in order to achieve and maintain health behaviorchange [12], [13]. These interventions may be provided manytimes (i.e., tens, hundreds, or even thousands of treatmentoccasions during the entire treatment period). The interven-tions are often delivered via portable devices, as intensiveinterventions. Since these interventions adapt to individualprogress, it is effective to prevent insufficient response andreact immediately against the unexpected shock on behavior.Therefore, scientists in the social and behavioral sciences havebeen working on finding systematic ways to use portabledevices such as smart phones to change health behavior ormaintain healthy behavior in real time. Consequently, adaptiveinterventions are being developed by social and behavioralscientists for different areas, such as hypertension [14], sub-stance abuse [15], criminal justice [21], mental health [22],and Alzheimer’s disease [23].

Engineering concepts, such as dynamical modeling, havebeen applied to modeling and controlling behavior [6].References [3]–[5] present special issues that arise in applyingsuch approaches to behavioral research. These studies employboth time-invariant and time-varying models, depending on thespecific problem. Some extend studies use controller designmethods to design adaptive interventions for special problemsin behavioral science [7]–[9]. This paper defines uncertaintiesin behavior more explicitly than previous studies and discussestheir effects on the robustness of the algorithm. In general,according to the intervention design problem, the MPC designmethod is preferred. Preliminary results of this paper werepresented in [24]. More detailed explanations of modelingbehavior and identification are presented in this paper. Thecomplete proofs are also provided, along with simulations.

This paper is organized as follows. In Section II, weintroduce the model that will be used to approximate patientbehavior and justify its use. In Section III, we discuss a wayto identify the model’s parameters from patient study data.In Section IV, an MPC formulation of the robust controlproblem is introduced. In Section V, a simulation of theapplication of the proposed approach is presented with asimulated smoking cessation treatment from [25]. In thissection, some hypothetical individual results are also given toshow how a control algorithm works. Finally, in Section VI,we present some concluding remarks.

II. MODEL

As mentioned in Section I, data on human behavior usuallyinvolve several subjects with many gaps in information overtime. Moreover, behavior might change with time and varyfrom subject to subject. Therefore, it is very unlikely that onecan accurately specify a complex model of behavior. One canuse a simple model to approximate patient behavior or the

BEKIROGLU et al.: CONTROL ENGINEERING METHODS FOR THE DESIGN OF ROBUST BEHAVIORAL TREATMENTS 981

main dynamics of the subject and, at the same time, highlightthe fact that there is a substantial amount of uncertainty in it.Hence, a possible model for identification is a set of affinedifference equations. The difference model used in this paperis of the form

yk+1 = a f +n−1∑

i=0

[Aiyk−i + Bi Tk−i ] + Dεk + Ewk (1)

where yk ∈ Rm is the measured output vector at time k,

Tk ∈ R, and Tk ∈ T is the control input (i.e., treatment),where T is a set of all available treatments. In many cases,this set has a finite number of elements. Unmeasurable sparseexogenous perturbations are denoted by wk ∈ R and εk ∈ R

m ,which represents uncertainty in the model. The affine termvector is a f ∈ R

m , and Ai ∈ Rm×m , Bi ∈ R

m , D ∈ Rm×m ,

and E ∈ Rm are coefficients matrices.

The dynamics of behavior are very complex and hard tocontrol because of the uncertainties on behaviors. Therefore,the uncertain difference equation in (1) is used to model alarge set of responses to behavioral treatment. In addition tothe dynamics of behavior, which are contaminated by Gaussiannoise, an unmeasured exogenous input is added to the modelto represent unexpected influences.

We use as a motivating example a hypothetical study ofsmoking cessation. In this example, we address the prob-lem of treatment design after quit day (i.e., day on whichan individual quits smoking). Patients usually have differentdynamics before and after quit day and different treatmentsshould be designed for each of these cases. Since the objectiveof this paper is to develop a general design procedure, we onlyconsider the after quit day case as an example of treatmentdesign. The procedure developed can also be used for beforequit day case.

In the smoking cessation model, there are three differentvariables that one can measure: smoking urge (denoted by su);negative affect, (na), which is a single scale indicating anadverse mood state; and self-efficacy, (se), which representsan individual’s belief in their ability to abstain from smoking.With these variables, the following model is proposed:

⎡

⎣suk+1nak+1sek+1

⎤

⎦ =⎡

⎣a f

d f

g f

⎤

⎦ +n−1∑

i=0

⎧⎨

⎩

⎡

⎣ai bi ci

di ei fi

gi hi ri

⎤

⎦

⎡

⎣suk−i

nak−i

sek−i

⎤

⎦

⎫⎬

⎭

+n−1∑

i=0

⎧⎨

⎩

⎡

⎣0qi

si

⎤

⎦ Tk−i

⎫⎬

⎭+

⎡

⎣100

⎤

⎦ wk

+⎡

⎣1 0 00 1 00 0 1

⎤

⎦

⎡

⎢⎢⎣

ε(su)k

ε(na)k

ε(se)k

⎤

⎥⎥⎦ . (2)

Remark 1: The signs of several of these coefficients areassumed to be known in advance for this example. For exam-ple, negative affect (na) increases smoking urge (su), whereasself-efficacy (se) decreases it. More information about thedynamics of the smoking urge can be found in [26] and [27].This information can be used in two different ways: 1) it can be

given to the identification algorithm as additional constraintsor 2) it can be utilized to validate the model.

This model also shows that smoking urge cannot be directlycontrolled, but if the negative affect can be decreased and self-efficacy can be increased by applying treatment T, the desiredsmoking urge level can be achieved under the uncertainties.

One should first note that the model above is an uncertainaffine model. One of the reasons why this structure was chosenwas the fact that a model should not be obtained just from data,and it should also leverage characteristics determined by priorbehavioral research. For instance, patients smoke cigarettesto regulate the smoking urge set point (see [3], [28] formore detailed explanation of smoking dynamics). Therefore,an affine term (constant term in the model) is added to themodel to represent this set point.

The model in (1) is called a structural equationmodel (SEM) in social science [29]. Similar to SEM models,an affine model for smoking cessation is preferred, becausethe equilibrium point of the uncontrolled system is often notthe origin. The model is obtained by using data from severalpatients and differences in patients are typically modeledas Gaussian random variables ε. The external perturbationw ∈ L∞ is motivated by a different kind of uncertainty inhuman behavior. The way the subjects behave is influenced byexternal sparse events that temporarily affect his/her responseto treatment. Hence, one needs an uncertainty that is sparse,bounded in magnitude, and with limited cumulative effect.As a result, it is assumed that the segment of perturbationsignal w ∈ L∞ is bounded in the �1-norm, �∞-norm, and�0-norm. For this situation, a possible signal set W , which isused in this paper, is the following. Given an MPC controlwindow of length K and k, the set W is

W = {w ∈ L∞ : for all k, ‖w(k:k+K−1)‖∞ ≤ α,

‖w(k:k+K−1)‖1 ≤ γ, ‖w(k:k+K−1)‖0 = K/ς}. (3)

α ∈ N is bound on the magnitude of the perturbation,γ ∈ N is bound on cumulative effect, and ς ∈ N enforcesthe sparsity constraint on vector w(k:k+K−1).

III. IDENTIFICATION OF THE MODEL

AND PERTURBATIONS

To estimate the coefficients from study data, we start bynoting that the model above can be taken to represent therelations among the variables as

Y = Hβ + ε. (4)

In this model, the unknown vector β contains the parame-ters of the model and vector w, vector Y is a function ofthe measurements, and matrix H is a function of the mea-surements and inputs (treatment). For the smoking cessationexample, the structure and the dimension of the measurementvector Y , unknown vector β, and matrix H in (4) are givenin Appendix A.

There are several ways to estimate vector β. In the examplesprovided in this paper, lasso [11] is used as an identificationtool. Assuming P is the number of the patients, N is the lengthof the data from each patient, and n is the order of the system,


Algorithm 1 Lasso Iterative Identification Algorithm

this algorithm provides a way to determine an estimate of theparameters that balances the size of the noise ε ∈ N (0, σ 2)and the sparsity of the exogenous perturbation w in the model.Note that there is also a balance between the order of thesystem and the size of the perturbation (magnitude, sparsity,and so on). Sparsity of signal w is crucial in this context, asit represents perturbations that a patient faces infrequently

{β, σ } = arg minβ,σ,w

‖Y − Hβ‖22

2nσ+ σ

2

s.t. ‖w‖1 ≤ μ. (5)

The optimization problem in (5) is a convex minimizationof a penalized joint loss function with a regularization para-meter μ given for the regression coefficient, noise level, andvector w, which is a finite dimensional signal.

The design matrix H and the response vector Y areconstructed from measurements from P different patients(see Appendix A). Since the problem is jointly convex withrespect to β and σ , if the initial estimate of β is knownor given, then σ , which minimizes the objective function,can be calculated by differentiating the objective function (5)with constant β vector (step 3 in Algorithm 1). Then, theoptimization problem in step 4 in Algorithm 1 is solved toestimate vector β.

The following lasso algorithm shows the identification pro-cedure in [11], where j shows the iteration number.

The terms in the objective function (5) aim at finding themaximum likelihood estimate of the parameters of the modeland the variance of the noise ε while the �1 constraint searchesfor a sparse exogenous input. Here, the �1-norm is used as aconvex approximation of the �0-norm [30]. Thus, given thedata and the order of the model n, at the end of the identifica-tion algorithm, the parameters of the model in (2), the noiselevels of ε ∈ N (0, σ 2), and unmeasurable exogenous inputw ∈ R

P(N−n) are estimated. Then, the statistic of identifiedvector w ∈ R

P(N−n) is used to define signal set W in (3).In the smoking cessation example, since an uncertain model

is searched for all P individuals (patients), all observationsof sui

k, naik and sei

k (i = 1, 2, . . . ,P) are used to constructthe regression model (4). Again the mathematical detailsof model (1) for smoking cessation example are definedin Appendix A.

Fig. 1. Adaptive intervention algorithm.

Finally, given a time interval of length K , output vector yis a function of uncertainties w ∈ L∞, ε ∈ N (0, σ 2), andthe state of the system. As mentioned earlier, w is a boundedsparse uncertainty and ε is Gaussian noise that models dif-ferences between different subjects. Now, to be able to designrobust controllers, we need to have a bounded support set for ε.

Remark 2: Since Gaussian distributions have unboundedsupport, we choose a set of high probability. More precisely,given a window of size K , the density of εk:k+K−1 hashyperspherical contours and we take a set of high probabilityof the form

‖εk:k+K−1‖2 ≤ ρ (6)

where ρ is chosen based on the identified standard deviationof the noise. This has the added advantage of leading to aformulation of the robust treatment design that is computation-ally tractable. We refer the reader to [31] for a more detaileddiscussion of the advantages of using this approach to describethis type of noise.

IV. ROBUST MODEL PREDICTIVE CONTROLLER

The method for controller design used in this paper isbased on MPC [16]. Here, the usual approach in robustMPC is taken: one estimates the present value of thestate variable and determines the value of the controlvariables over the horizon that minimizes a given objectivefunction. The first of these control signals is applied, andthe process is then repeated. The control algorithm issummarized in Fig. 1. In step 1, initial conditions or theinformation yk−1 . . . yk−n that the control algorithm will use iscollected. In the smoking cessation study, this information issuk−1 . . . suk−n, nak−1 . . . nak−n, sek−1 . . . sek−n. In addition,after consultation with a practitioner in the field, the totaltreatment to be given to the patient is limited to an allowableset [32]–[34]. Such limitations might require the knowledgeof how much treatment was given in the recent past. As aresult, recent control input information Tk−1 . . . Tk−l (l ≥ n)is used in order to enforce the constraints on the totaltreatment provided.

Consistent with the usual MPC approach, in step 2, oneminimizes a given cost function subject to constraints on thetotal applied control in a certain range. Then, in step 3, theresult of the control algorithm is applied to the patient. Thus, inthis section, a decision rule is developed that dictates whethertreatment is applied to each individual at each time point underthe uncertainties.

The difference equation (1) is used to determine aclosed-form objective function for MPC formulation with


a given receding horizon K . In the objective function,we represent Yk+1 ∈ R

K m as

Yk+1 = A f + AY0 + BTk + Dεk + Ewk (7)

where Y0 ∈ Rnm is the vector containing the state of the

model. Tk ∈ RK+n−1 is the treatment. wk ∈ R

K is sparsedisturbance and εk ∈ R

K is noise. A f , A, B, D, and Eare matrices, which are calculated recursively from differenceequation (1)

Yk+1 =

⎡

⎢⎢⎢⎣

yk

yk+1...

yk+K−1

⎤

⎥⎥⎥⎦

, Y0 =

⎡

⎢⎢⎢⎣

yk−n

yk+1−n...

yk−1

⎤

⎥⎥⎥⎦

, Tk =

⎡

⎢⎢⎢⎣

Tk−n

Tk+1−n...

Tk+K−2

⎤

⎥⎥⎥⎦

wk =

⎡

⎢⎢⎢⎣

wk−1wk...

wk+K−2

⎤

⎥⎥⎥⎦

, εk =

⎡

⎢⎢⎢⎣

εk−1εk...

εk+K−2

⎤

⎥⎥⎥⎦

.

Then, in this paper, the objective function of the followingform is considered:

(Yk+1 − θ)X (Yk+1 − θ)T

although more complex convex functions can be addressedby the framework presented here. For the smoking cessationexample, since one minimizes smoking urge, matrix X isdefined to choose smoking urge measure in vector Yk+1 as

X ={

1, if xi,i = k, k + m, k + 2m, . . . , k + (K − 1)m

0, otherwise.

Given the tuning parameter θ ∈ R, approximation of desiredaverage level of smoking urge, we aim to solve the followingrobust optimization problem at each step of the algorithm:

minT∈T

max‖εk‖2≤ρwk∈Wk

(Yk+1 − θ)X (Yk+1 − θ)T (8)

which is subject to the system dynamics described in (7).We also define Wk from the set W as

Wk = {vector wk ∈ RK satisfies : ‖wk‖∞ ≤ α,

‖wk‖1 ≤ γ, ‖wk‖0 = K/ς}. (9)

The intersection of the norms in the set Wk forms convexpolytopes. It should be noted that set Wk is a union of theseconvex polytopes; this fact is explored later when solvingthe robust optimization problem resulting from the MPCformulation of the control problem.

Smoking Cessation Example: We now discuss the set T ofallowable treatments used for the smoking cessation example.In this case

T =⎧⎨

⎩T : Tη ∈ {0, 1} for η = k, . . . , k + K − 1 and

k+K−1∑

η=k−l

Tη ≤ Ttotal

⎫⎬

⎭(10)

represent the fact that one can either apply or not applytreatment at each time. In addition, the number of treatmentsapplied in the last K + l instances is constrained to be lessthan or equal to Ttotal. This constraint in the number oftreatments is aimed at addressing the problem of treatmentburden [32]–[34], which is decreasing effectiveness as thenumber of applied treatments increases. In this specificexample, treatment T is binary; but in reality, it can have adifferent level and type of treatment.

One should note that the optimization problem describedearlier is complex. Therefore, a problem equivalent to theoriginal problem (8) is now presented that is suitable forimplementation in Theorem 1. Using the results in [17]–[19](provided in Appendix B for completeness), one can prove thefollowing result.

Theorem 1: Consider the following semidefinite problem:min

τ,Tk∈T ,λτ

subject to⎡

⎣τ − λρ2 0 ∗

0 λI (DX)T

(A f − θ + AY0 + BTk + Ewk)X DX I

⎤

⎦ ≥ 0

(11)

for all wk ∈ Wext, where Wext values are the extremes of thepolytopes whose union is Wk .

The optimum of the problem above is an upper bound onthe optimum of the original optimization problem (8) and isequal if only one of the elements in the set Wext leads to anactive constraint at the optimum.

In the LMI in (11), ∗ is ((A f −θ + AY0 + BTk + Ewk)X)T .Proof: See Appendix C. �

We now characterize the set of extremes, Wext.Theorem 2: Consider set Wk defined in (9), where α, γ ,

and ς define the bounds on the �∞-norm, �1-norm, andsparsity, respectively. Let nnz = dim(wk)/ς .

1) If α ≥ γ , then Wext is the set of all signed permutationsof

[γ 0 · · · 0︸︷︷︸K−1

].

2) If nnzα ≤ γ , then Wext is the set of all signedpermutations of

[ α · · · α︸︷︷︸nnz

0 · · · 0︸︷︷︸K−nnz

].

3) If γ < nnzα < nnzγ , then Wext is the set of all signedpermutations of

[ α · · · α︸︷︷︸n+

h 0 · · · 0︸︷︷︸K−n+−1

]

where

n+ =⌊γ

α

⌋and h = γ − n+α.

Proof: See Appendix D. �After finding the extreme points of the set Wk , a semidefi-

nite programming or mixed integer semidefinite programmingsolver could be used to solve problem (11); but for a large


receding horizon K , the number of extremes and, thus, thenumber of LMI constraints might be fairly large.

Section V presents a simulated application of the algorithmusing a model that mimics data that might have been collectedin smoking cessation studies, such as [25].

V. SIMULATION RESULTS

A. Implementation

As an implementation of the adaptive treatment algorithmthat is proposed for an intervention using portable devices,we consider individuals who wish to quit smoking, and wepresent a hypothetical algorithm for treatment via a smartphone. In addition, the quit day (i.e., day on which anindividual quits smoking) and the time immediately followingthe quit day are extremely important on smoking cessationinterventions and in the prevention of relapse. The simulationsin this example address smoking dynamics after the quit day.

In the simulation, the participant provides information onhis/her level of smoking urge, negative affect, and self-efficacyon the morning of day 1; the smart phone then providesthe behavioral treatment. Robust adaptive treatment begins atmidday after the participant again provides his/her level ofsmoking urge, negative affect, and self-efficacy. Right now,sui

0, nai0, sei

0, Ti0 and sui

1, nai1, sei

1 are known, where Ti1

is treatment provided in the morning. The proposed adaptivetreatment algorithm is run and decides Ti optimal and set Ti

2 =Ti optimal. That is, if Ti optimal = 1, treatment is provided atmidday; otherwise, it is not. Afterward, new sui

3, nai3, sei

3 arecollected; then by using these new values and sui

2, nai2, sei

2,

the process Ti2 is repeated to obtain Ti

3 = Ti optimal. Thisprocess is repeated throughout the entire course of treatment.

B. Identification Results

In order to test the performance of the approach presented inthis paper, we emulate a real application. We develop a second-order (n = 2) model for smoking urge that approximatesthe behavior observed in empirical studies, such as [25].Note that, in this paper, subjects smoke heavily. Therefore,the intervention is not expected to drive cravings to zero inthis short-term program. Moreover, during the initial phase oftreatment, the proposed algorithm aims to decrease smokingurge of patients as much as possible, while simultaneouslyreducing the amount of treatment. We aim to increase initialimpact of treatment.

Here is the true model of smoking urge⎡

⎢⎢⎢⎢⎢⎢⎢⎢⎣

a f

a0a1b0b1c0c1

⎤

⎥⎥⎥⎥⎥⎥⎥⎥⎦

=

⎡

⎢⎢⎢⎢⎢⎢⎢⎢⎣

1.06.50.35.010

−.12−.09

⎤

⎥⎥⎥⎥⎥⎥⎥⎥⎦

,

⎡

⎢⎢⎢⎢⎢⎢⎢⎢⎣

d f

d0d1e0e1f0f1

⎤

⎥⎥⎥⎥⎥⎥⎥⎥⎦

=

⎡

⎢⎢⎢⎢⎢⎢⎢⎢⎣

.15

.010

.49

.25−.05

0

⎤

⎥⎥⎥⎥⎥⎥⎥⎥⎦

,

⎡

⎢⎢⎢⎢⎢⎢⎢⎢⎣

g f

g0g1h0h1r0r1

⎤

⎥⎥⎥⎥⎥⎥⎥⎥⎦

=

⎡

⎢⎢⎢⎢⎢⎢⎢⎢⎣

.52−.01

0−.02

0.49.36

⎤

⎥⎥⎥⎥⎥⎥⎥⎥⎦

.

The studies mentioned earlier do not contain treatment.Hence, we augmented the model with a nonlinear effectof treatment in order to represent treatment burden. In theliterature, the treatment burden is defined as increasing the

level of treatment to the extent that it may cause suboptimaladherence and even negative outcomes. More informationabout treatment burden in social and medical sciences can befound in [32]–[34]. More precisely, the following terms wereintroduced:

q0Tk−1 + q1sig

[u=k−2∑

u=k−44

Tu

]

Tk−2 (12)

s0Tk−1 + s1sig

[u=k−2∑

u=k−44

Tu

]

Tk−2

where

sig(x) = 1

1 + e−(x−17).

sig(x) represents the sigmoid function or special case oflogistic function. Then, the following parameter values areintroduced:

[q0 q1 s0 s1] = [−.2 .2 .2 − .2].This model for the effect of treatment is designed to emulatethe case where the treatment has a positive impact, but itseffectiveness decreases when the treatment is applied toofrequently.

For this highly uncertain model, 500 different trajectories ofthe system are generated, with random initial conditions andrandom realizations of uncertainty. This is done to simulatethe behavior of 500 subjects in a study where the treatmentwas provided at random times. Note that this is a reason-able surrogate for a study, since the model is designed toapproximate real data, except for the influence of treatment.The model identification algorithm described in Section IIIis then applied, and a model to be used by the robust MPCalgorithm was obtained. To validate the model, we can use theinformation in [26] and [27], showing that negative affect (na)increases smoking urge (se), where the sign of the coefficientof negative affect (na) is positive; self-efficacy (se) decreasessmoking urge (su), where the sign of the coefficient of self-efficacy (se) is negative. These effects also can be seen inFig. 3 and Table II. Moreover, the dynamical equations ofnegative affect and self-efficacy indicate that the effect oftreatment is very small. This results in small improvement insmoking urge. However, this improvement is very importantfor heavy smokers at the beginning of the treatment process.

To do this, a model of the form (2) is used with order n = 2.The model coefficients obtained for a randomly chosen virtualpatient are

⎡

⎢⎢⎢⎢⎢⎢⎢⎢⎣

a f

a0a1b0b1c0c1

⎤

⎥⎥⎥⎥⎥⎥⎥⎥⎦

=

⎡

⎢⎢⎢⎢⎢⎢⎢⎢⎣

1.11.47.34.010

−.10−.08

⎤

⎥⎥⎥⎥⎥⎥⎥⎥⎦

,

⎡

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

d f

d0d1e0e1f0f1q0q1

⎤

⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

=

⎡

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

.13.005

0.5.24

−.050

−0.2070.15

⎤

⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

,

⎡

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

g f

g0g1h0h1r0r1s0s1

⎤

⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

=

⎡

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

.52−.011

0−.02

0.49.35

0.202−0.142

⎤

⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦


TABLE I

INITIAL CONDITION OF STATES AND INPUT

TABLE II

INTERVENTION PERFORMANCE OF SDP

and the description of the uncertainty is, for robust MPCwindow of K = 8, the following: wk has 25% nonzero termswith ‖wk‖1 ≤ 5 and ‖wk‖∞ ≤ 2.5. In order to obtain theseparameters, the statistic of the identified w in Section IIIis used. As for Gaussian uncertainty, εsu ∼ N (0, 1.3),εna ∼ N (0, 0.78), and εse ∼ N (0, 0.55) are identified. Thesampling period here is 8 h and, hence, data are collected threetimes per day. The simulation is run for 50 days, and 150 datapoints are collected.

C. Experiments

We then applied the proposed controller in this paper toa nonlinear model mentioned earlier, again with randomlygenerated uncertainty/noise and the initial conditions given atTable I. In other words, we simulate the application of ourrobust MPC algorithm to a patient. The parameters used arespecified as follows.

1) MPC window size: K = 8.2) Constraints in control: Tk ∈ {0, 1} and

Ttotal =k+K−1∑

η=k−3

Tη ≤ Ttotal = 10/3.

In other words, one either applies or does not applytreatment at each time point, and there is a constraintof a maximum of approximately one treatment per threesample times. Full treatment is defined as Tk = 1 forall values of k.

3) Objective function to be minimized

k+K∑

η=k+1

(su(η) − 2.2)2

where the target value 2.2 was chosen, so that one hasa significant decrease in the mean smoking value.

4) Bound on the two-norm of the Gaussian noise: ρ = 30.5) Initial conditions of the states and input in Table I.

D. Simulations Results for a Patient

The optimization problem to be solved here is a mixed-integer, semidefinite convex problem. There are many ways tosolve it, but in the simulations performed here, the requirementT = Tk ∈ {0, 1} is relaxed to T = Tk ∈ [0, 1] to

Fig. 2. Performance of adaptive intensive intervention.

use a general, semidefinite programming solver. The controlapplied is

Tappliedk = round(Tk).

The simulation was run for 150 time instances corre-sponding to a real time interval of 50 days. CVX is usedwith SeDuMi as the semidefinite programming solver toproblem (11) [35]. The developed control law systematicallyimproved smoking urge, since the treatment is only appliedwhen needed. At the same time, our control law increasedself-efficacy while reducing negative affect in order to reducesmoking urge. Table II represents a typical example of theresults obtained.

Table II shows the performance of the adaptive treatmentdesign in terms of objective function, average values ofsmoking urge, self-efficacy, negative affect, and total amountof treatment that the adaptive intervention applied. Moreover,to show the benefit of our robust adaptive intervention, thesimulation is also done where controller design did notconsider uncertainty (no robustness in Table II). Resultsindicated that the robust adaptive intervention applies theoptimum amount of treatment to improve smoking urge,self-efficacy, and negative affect. The improvement insmoking urge is small, likely because this is a population ofheavy smokers [25]. However, this improvement is extremelyimportant at the beginning of the treatment process and leadsto significant benefit in long term period. Note that thisimprovement was obtained with a minimized treatment effort.Therefore, one can see that there is a significant improvementin smoking urge not only in terms of the average but also inthe fact that a consistent decrease in urge is obtained. If thetreatment was more diverse than a binary signal [see thetreatment set T in (10)] (e.g., varying dosages), the controllerbenefits from adaptation would be greater.

Fig. 2 shows decrease in smoking urge. In Fig. 2,suFull Treatment − suAdaptive Treatment is depicted, wheresuAdaptive Treatment is smoking urge measured under theadaptive treatment and suFull Treatment is smoking urge


Fig. 3. Smoking urge, negative affect, and self-efficacy under adaptiveintervention.

Fig. 4. Sparse disturbance.

measured under the full treatment. Although the fulltreatment performs better than the adaptive treatment in thebeginning of the treatment process, in the long term, theadaptive treatment works better, because the algorithm appliesthe treatment only when it is really needed, thus reducingtreatment burden.

As expected, the algorithm carefully chooses when to applytreatment. It is mainly applied when external perturbationslead to a significant increasing trend in smoking urge. Finally,Fig. 3 shows the smoking urge, self-efficacy, and negative-affect measure of a virtual patient under both full treatmentand adaptive intervention. Note that the responses shownin Fig. 3 are reflective of long-time, post-quit behavior. Whileimportant, this only represents a subset of the behavior stagesseen in smoking cessation. Asterisks in Fig. 3 indicate whentreatment is provided by the adaptive treatment algorithm.Fig. 4 shows the exogenous sparse disturbance that is appliedto this particular patient for this simulation.

Remark 3: To be able to formulate the treatment designproblem (11) as a convex optimization problem, in the

Fig. 5. Adaptive and full treatment intervention results.

simulations above, we replaced the constraint Tk ∈ {0, 1} bythe relaxation Tk ∈ [0, 1]. Then, the treatment applied wasround (Tk). To test the effectiveness of such relaxation, branchand bound techniques were used to solve the optimizationproblem under the true constraint Tk ∈ {0, 1}. The resultsobtained were similar and, hence, in this example, there isno advantage in solving the more complex mixed integeroptimization problem.

E. SDP Results of a Population

Since some of the parameters are as random in the simu-lation, the algorithm is run 400 times to estimate the averageimprovement in smoking urge, self-efficacy, and negativeaffect. The same parameters in Section V-C except that l = 6in (10) are used to run the algorithm. Moreover, for the randomparameters, the disturbances wk are generated uniformly, suchthat wk ∈ Wk . The Gaussian disturbances are generatedunder a normal distribution, such that εsu ∼ N (0, 1.3),εna ∼ N (0, 0.78), and εse ∼ N (0, 0.55). For the initialconditions, su0 ∈ U(3.6, 0), se0 ∈ U(4, 0), and na0 ∈ U(2, 0)are generated. Fig. 5 shows average improvement in smokingurge (suNo Treatment − suAdaptive or Full Treatment), self-efficacy(seAdaptive or Full Treatment − seNo Treatment), and negative affect(naNo Treatment − naAdaptive or Full Treatment). It is shown that theadaptive treatment increases self-efficacy while it decreasessmoking urge and negative affect, even with less treatment.The average number of treatments provided is 54.6.

1) suno, seno, nano: Smoking urge, self-efficiency, andnegative affect without treatment.

2) sufull, sefull, nafull: Smoking urge, self-efficiency, andnegative affect under full treatment.

3) suadaptive, seadaptive, naadaptive: Smoking urge, self-efficiency, and negative affect with adaptive treatment.

VI. CONCLUSION

Behavioral and social scientists have been considering theadvantages of adaptive treatments. Research has indicated thatadaptive treatment strategies might give better results than


fixed treatment (i.e., all patients get the same type and levelof treatment) [12]. Since adaptive treatments usually providebetter results than the usual methods, in this paper, we arguethat control engineering methods, such as feedback or adapta-tion and robust optimization, can provide a systematic way todesign a personalized treatment algorithm. Control engineer-ing methods can be used to design personalized behavioralinterventions while reducing treatment burden. Existing theoryfor adaptive treatment design is primarily qualitative and thusdoes not provide precise guidance regarding how much ofwhich treatment treatment to provide to which individualsat what times. With this adaptive intervention design algo-rithm, treatments can be adapted and readapted in responseto individuals’ progress over a long period of time. Thesemethods hold promise for maintaining desired behavior insituations where controlling behavior is challenging due tocomplex dynamics. In terms of future work, effort is beingput into developing more efficient numerical algorithms forsolving the optimization problem for MPC. This will allowfor the consideration of a larger receding horizon and, hence,better performance.

APPENDIX AFORMULATION OF THE IDENTIFICATION PROBLEM

In this section, we show how general equation (4) can bereorganized for the lasso algorithm. Start with the data set(su, na, se) of P patients. Assume that for each i th patient,we have collected data set sui

k, naik, sei

k for k = 1, 2, . . . , Nand i = 1, 2, . . . ,P . Also assume that the order of themodel n is given. Then, for each patient i , build the matricesin (13) and (14), shown at the bottom of this page.

Also, define

sui = [sui

n+1 suin+2 · · · sui

N

]T where sui ∈ RN−n

nai = [nai

n+1 nain+2 · · · nai

N

]T where nai ∈ RN−n

sei = [sei

n+1 sein+2 · · · sei

N

]T where sei ∈ RN−n

and

SU =

⎡

⎢⎢⎢⎣

su1

su2

...

suP

⎤

⎥⎥⎥⎦

NA =

⎡

⎢⎢⎢⎣

na1

na2

...

naP

⎤

⎥⎥⎥⎦

SE =

⎡

⎢⎢⎢⎣

se1

se2

...

seP

⎤

⎥⎥⎥⎦

where SU ∈ RP(N−n), NA ∈ R

P(N−n), and SE ∈ RP(N−n)

X1 =

⎡

⎢⎢⎢⎣

X1,1X1,2

...X1,P

⎤

⎥⎥⎥⎦

X2 =

⎡

⎢⎢⎢⎣

X2,1X2,2

...X2,P

⎤

⎥⎥⎥⎦

.

Finally, let

θsu = [ a f a0 a1 · · · an b0 · · · bn c0 c1 · · · cn ]θna = [ d f d0 d1 · · · an e0 e1 · · · bn f0 f1 · · · fn

q0 q1 · · · qn ]θse = [ g f g0 g1 · · · an h0 h1 · · · bn r0 r1 · · · rn

s0 s1 · · · sn ]β = [

θsu θna θse w]T

where w is a w ∈ RP(N−n) dimensional vector. Then, if all

the patients satisfy the model provided in (2), we have

Y = Hβ (15)

where Y ∈ R3P(N−n), H ∈ R

3P(N−n)×3(3n+1)+N , and coeffi-cient vector β ∈ R

(11n+3+P(N−n))

Y =⎡

⎣SUN ASE

⎤

⎦ H =⎡

⎣X1

00

0X2

0

00

X2

I00

⎤

⎦

and ε is a vector containing all noise.

APPENDIX BTHEOREM IN [17]–[19]

Theorem 3 [17]–[19]: Assume that the following problemis defined:

φ(A, b, x) � minx

max‖ε‖≤ρ‖A(ε)x − b(ε)‖2

2. (16)

Given A0, . . . , A p ∈ Rn×m , b0, . . . , bp ∈ Rn , and ε ∈ Rp ,the following uncertainty structure can be defined:

A(ε) � A0 +p∑

i=1

εi Ai , b(ε) � b0 +p∑

i=1

εi bi . (17)

Then, for ρ ≥ 0, define

F(x) � [A1x − b1 . . . A px − bp]. (18)

X1,i =

⎡

⎢⎢⎢⎢⎢⎣

1 suin .. sui

1 nain .. nai

1 sein .. sei

1

1 suin+1 .. sui

2 nain+1 .. nai

2 sein+1 .. sei

2...

......

......

......

......

1 suiN−1 .. sui

N−n naiN−1 .. nai

N−n seiN−1 .. sei

N−n

⎤

⎥⎥⎥⎥⎥⎦

(13)

X2,i =

⎡

⎢⎢⎢⎢⎢⎣

1 suin .. sui

1 nain .. nai

1 sein .. sei

1 T in .. T i

1

1 suin+1 .. sui

2 nain+1 .. nai

2 sein+1 .. sei

2 T in+1 .. T i

2...

......

......

......

......

......

...

1 suiN−1 .. sui

N−n naiN−1 .. nai

N−n seiN−1 .. sei

N−n T iN−1 .. T i

N−n

⎤

⎥⎥⎥⎥⎥⎦

(14)


The solution of the structured robust least square problem (16)can be calculated by solving the following semidefiniteproblem:

minτ,x,λ

τ

s.t.

⎡

⎣τ − λρ2 0 (A0x − b0)

T

0 λI F(x)T

(A0x − b0) F(x) I

⎤

⎦ ≥ 0. (19)

APPENDIX CPROOF OF THEOREM 1

Proof: We use the structure of Yk+1 in (7) to analyzethe robustness of the optimization problem. Thus, Yk+1 in theobjective function in (8) is replaced by Yk+1 in (7) as

minT∈T

max‖εk‖2≤ρwk∈Wk

‖(A f − θ + AY0 + BTk + Dεk + Ewk)X‖22.

(20)

Given objective function in (20), we can define vectors andmatrix in (17) as

A0 = B, Ai = 0, b0(wk) = A f − θ + AY0 + Ewk

and bi = D:,i .Then, for fixed Tk and wk , worst case residual is defined

by using the methods in [18, Sec. 4] as

rs(A, b, Tk, wk)2 = max

‖εk‖2≤ρ‖BTk − b(ε, wk)‖2

2

and from (18), define

M.= FT F, g(Tk, wk)

.= FT (BTk − b0)

h(Tk , wk).= ‖ BTk − b0‖2

2.

Then, without loss of generality, assuming ρ = 1 yields

rs(A, b, Tk, wk)2 = max

‖εk‖2≤1

[1ε

]T

×[

h(Tk , wk) g(Tk, wk)T

g(Tk, wk) M

] [1ε

]

. (21)

Given τ ≥ 0, using the S-procedure [18, Lemma 2.1],(21) can be converted as

[1ε

]T [h(Tk , wk) g(Tk, wk)

T

g(Tk, wk) M

] [1ε

]

≤ τ (22)

for every ε, εT ε ≤ 1 if and only if there exists a scalar λ ≥ 0,such that

[1ε

]T [τ − λ − h(Tk , wk) −g(Tk, wk)

T

−g(Tk, wk) λI − M

] [1ε

]

≥ 0 (23)

for every ε ∈ Rp and fixed Tk and wk .

Using the fact that λ ≥ 0 is implied by λI ≥ M , then theabove condition can be rewritten as

F(τ, λ, Tk , wk).=

[τ − λ − h(Tk , wk) −g(Tk, wk)

T


]

≥ 0.

Then, the upper bound on the residual suchthat rs(A, b, Tk, wk)

2 ≤ τ can be obtained by solving the

following optimization problem [18, Th. 4.1]:τ ∗ = min

τ,λτ

s.t.


T


]

≥ 0 (24)

for all wk ∈ Wk .In addition, instead of checking all wk ∈ Wk in LMI

constraint (24), one can use the result in [36], which statesthat all the members of a polytope of matrices are positivesemidefinite if and only if all extremes of the polytope arepositive semidefinite. Furthermore, the set Wk is a unionof sets bounded by polytopes. Using the same reasoning asin [36], it suffices for the LMI to be satisfied at the extremesof each of the polytopes. Therefore, we need to check allextremes of these polytopes. The LMI in (24) is satisfied inthe convex hull of these extremes. It is just a consequence ofthe fact that an LMI is satisfied in a set of points; it is alsosatisfied in the convex hull of these points. Then, we have thefollowing result:

τ ∗ = minτ,λ

τ

s.t.


T


]

≥ 0 (25)

for all wk ∈ Wext and forfixed Tk , where Wext values are theextremes of the polytopes whose union is Wk . Hence, τ ∗ isan upper bound on the robust performance for fixed Tk .

For the optimality, assume that there is only one extremew∗ ∈ Wext, such that one of the eigenvalues of LMI in (25) isequal to zero for some τ ∗ and λ∗, and LMI is positive definitefor the rest of the extremes. Then, if the τ ∗ is not optimum,we can perturb τ ∗ or decrease it for this specific w∗ ∈ Wextwhile keeping the LMI still positive definite for the rest ofthe extreme points. This contradicts the assumption that τ ∗is optimum. As a result, this optimization problem gives theoptimum τ ∗ if only one of the extremes is an active constraint.

Thus, for every fixed Tk and for all wk ∈ Wext, the resultsin [18, Th. 4.2] or Theorem 3 can be used to convert problem(25) to (26) for ellipsoid uncertainty ‖εk‖2 ≤ ρ and wk ∈ Wk .Finally, problem in (25) is modified as follows:

minτ,Tk∈T ,λ

τ

s.t.⎡

⎣τ − λρ2 0 ∗

0 λI (DX)T

(A f − θ + AY0 + BTk + Dεk +wk)X DX I

⎤

⎦≥0

(26)

for all wk ∈ Wext where again Wext values are the extremesof the polytopes, whose union is Wk .

In this LMI, ∗ is ((A f − θ + AY0 + BTk + Ewk)X)T . �

APPENDIX DPROOF OF THEOREM 2

The first two cases (α ≥ γ and nnzα ≤ γ ) are immediate,since they correspond to the cases where either �1 or �∞ is the


only binding constraints. Hence, in this proof, we concentrateon the third case (γ < nnzα < nnzγ ).

Note that, given the symmetry of the problem, we canconcentrate on the subset of the elements of Wk that arepositive and satisfy

wi+1 ≤ wi for all i = k : k + K − 1, and wi ∈ wk .

All other extremes will be obtained from this by permutationsand sign changes of the entries. Recall that, in this case,we are considering

γ < nnzα < nnzγ

and the elements of vector wk are in a nonincreasing order.Therefore, the extreme of the set is given by pushing as manyof the first few elements as possible to their maximum value.Given that ‖wk‖∞ ≤ α and ‖wk‖1 ≤ γ , one can only havethe first

n+ =⌊γ

α

⌋

elements of vector wk equal to α. To reach the extreme andrecalling that the entries of the vector w are in a nonincreasingorder, the (n+ + 1)th element of the vector must be at itsmaximum value. Given that ‖wk‖1 ≤ γ and the first N+ termsare equal to α

wn++1 ≤ γ − n+α.

Hence, for vectors with entries in a decreasing order, theextreme of the set is attained by a vector of the form

[ α · · · α︸︷︷︸n+

h 0 · · · 0︸︷︷︸K−n+−1

].

ACKNOWLEDGMENT

The content is solely the responsibility of the authors anddoes not necessarily represent the official views of the NationalInstitute on Drug Abuse or the National Institutes of Health.

REFERENCES

[1] K. L. Bierman, R. L. Nix, J. J. Maples, and S. A. Murphy, “Examiningclinical judgment in an adaptive intervention design: The fast trackprogram,” J. Consulting Clin. Psychol., vol. 74, no. 3, pp. 468–481,2006.

[2] D. E. Rivera, “Optimized behavioral interventions: What does systemidentification and control engineering have to offer?” in Proc. 16th IFACSymp. Syst. Identificat., 2012, pp. 882–893.

[3] K. P. Timms, D. E. Rivera, L. M. Collins, and M. E. Piper, “Continuous-time system identification of a smoking cessation intervention,” Int.J. Control, vol. 87, no. 7, pp. 1423–1437, 2014.

[4] J.-E. Navarro-Barrientos, D. E. Rivera, and L. M. Collins, “A dynamicalmodel for describing behavioural interventions for weight loss andbody composition change,” Math. Comput. Model. Dyn. Syst., vol. 17,pp. 183–203, Jan. 2011.

[5] Y. Dong et al., “A dynamical systems model for improving gesta-tional weight gain behavioral interventions,” in Proc. Amer. ControlConf. (ACC), 2012, pp. 4059–4064.

[6] D. E. Rivera, M. D. Pew, and L. M. Collins, “Using engineering controlprinciples to inform the design of adaptive interventions: A conceptualintroduction,” Drug Alcohol Dependence, vol. 88, no. 2, pp. S31–S40,May 2007.

[7] N. N. Nandola and D. E. Rivera, “Model-on-demand predictive controlfor nonlinear hybrid systems with application to adaptive behavioralinterventions,” in Proc. 19th IEEE Conf. Decision Control, Dec. 2010,pp. 6113–6118.

[8] A. Zafra-Cabeza, D. E. Rivera, L. M. Collins, M. A. Ridao, andE. F. Camacho, “A risk-based model predictive control approach toadaptive interventions in behavioral health,” IEEE Trans. Control Syst.Technol., vol. 19, no. 4, pp. 891–901, Jul. 2011.

[9] N. N. Nandola and D. E. Rivera, “An improved formulation ofhybrid model predictive control with application to production-inventorysystems,” IEEE Trans. Control Syst. Technol., vol. 21,no. 1, pp. 121–135, Jan. 2013, doi: 10.1109/TCST.2011.2177525.

[10] S. Deshpande, N. N. Nandola, D. E. Rivera, and J. W. Younger,“Optimized treatment of fibromyalgia using system identificationand hybrid model predictive control,” Control Eng. Pract., vol. 33,pp. 161–173, Dec. 2014.

[11] R. Tibshirani, “Regression shrinkage and selection via the lasso,”J. Roy. Statist. Soc. B, Methodol., vol. 58, no. 1, pp. 267–288,1996.

[12] L. M. Collins, S. A. Murphy, and K. L. Bierman, “A conceptualframework for adaptive preventive interventions,” Prevention Sci., vol. 5,no. 3, pp. 185–196, Sep. 2004.

[13] S. A. Murphy and L. M. Collins, “Customizing treatment to the patient:Adaptive treatment strategies,” Drug Alcohol Dependence, vol. 88, no. 2,pp. S1–S3, May 2007.

[14] M. S. Glasgow, B. T. Engel, and B. C. D’Lugoff, “A controlledstudy of a standardized behavioral stepped treatment for hypertension,”Psychosomatic Med., vol. 51, no. 1, pp. 10–26, 1989.

[15] W. F. Velicer and J. O. Prochaska, “An expert system intervention forsmoking cessation,” Patient Edu. Counseling, vol. 36, pp. 119–129,Feb. 1999.

[16] A. Bemporad and M. Morari, “Robust model predictive control:A survey,” in Robustness in Identification and Control (Lecture Notesin Control and Information Sciences), vol. 245. London, U.K.: Springer,1999, pp. 207–226.

[17] H. A. Hindi and S. P. Boyd, “Robust solutions to l1, l2, and l8 uncertainlinear approximation problems using convex optimization,” in Proc.Amer. Control Conf. (ACC), vol. 6. 1998, pp. 3487–3491.

[18] L. El Ghaoui and H. Lebret, “Robust solutions to least-squares prob-lems with uncertain data,” SIAM J. Matrix Anal. Appl., vol. 18,pp. 1035–1064, Oct. 1997.

[19] M. Salhov, “A subgradient solution to structured robust least squaresproblems,” IEEE Trans. Signal Process., vol. 58, no. 9, pp. 4761–4770,Sep. 2010.

[20] K. P. Timms, D. E. Rivera, M. E. Piper, and L. M. Collins, “A hybridmodel predictive control strategy for optimizing a smoking cessationintervention,” in Proc. Amer. Control Conf., Jun. 2014, pp. 2389–2394.

[21] D. B. Marlowe, D. S. Festinger, K. L. Dugosh, K. M. Benasutti,G. Fox, and J. R. Croft, “Adaptive programming improves outcomes indrug court: An experimental trial,” Criminal Justice Behavior, vol. 39,pp. 514–532, Apr. 2012.

[22] A. J. Rush et al., “Sequenced treatment alternatives to relieve depres-sion (STAR∗D): Rationale and design,” Controlled Clin. Trials, vol. 25,pp. 119–142, Feb. 2004.

[23] L. S. Schneider et al., “National Institute of Mental Health clinicalantipsychotic trials of intervention effectiveness (CATIE): Alzheimerdisease trial methodology,” Amer. J. Geriatric Psychiatry, vol. 9, no. 4,pp. 346–360, Jan. 2001.

[24] K. Bekiroglu, C. Lagoa, S. A. Murphy, and S. T. Lanza, “A robust MPCapproach to the design of behavioural treatments,” in Proc. IEEE 52ndAnnu. Conf. Decision Control, Dec. 2013, pp. 3505–3510.

[25] S. Shiffman et al., “Dynamic effects of self-efficacy on smoking lapseand relapse,” Health Psychol., vol. 19, no. 4, pp. 315–323, Jul. 2000.

[26] M. P. Shiyko, S. T. Lanza, X. Tan, R. Li, and S. Shiffman, “Usingthe time-varying effect model (TVEM) to examine dynamic associa-tions between negative affect and self confidence on smoking urges:Differences between successful quitters and relapsers,” Prevention Sci.,vol. 13, pp. 288–299, Jul. 2012.

[27] R. M. P. Van Zundert, S. G. Ferguson, S. Shiffman, andR. C. M. E. Engels, “Dynamic effects of self-efficacy on smokinglapses and relapse among adolescents,” Health Psychol., vol. 29,pp. 246–254, May 2010.

[28] K. P. Timms, D. E. Rivera, L. M. Collins, and M. E. Piper, “A dynamicalsystems approach to understand self-regulation in smoking cessationbehavior change,” Nicotine Tobacco Res., vol. 16, pp. S159–S168,Sep. 2013.

[29] K. A. Bollen and M. D. Noble, “Structural equation models andthe quantification of behavior,” Proc. Nat. Acad. Sci. USA, vol. 108,pp. 15639–15646, Oct. 2011.


[30] C. Ramirez, V. Kreinovich, and M. Argaez, “Why �1 is a goodapproximation to �0: A geometric explanation,” J. Uncertain Syst.,vol. 7, no. 3, pp. 203–207, 2013.

[31] F. Paganini, “A set-based approach for white noise modeling,” IEEETrans. Autom. Control, vol. 41, no. 10, pp. 1453–1465, Oct. 1996.

[32] K. Gallacher et al., “Qualitative systematic reviews of treatment burdenin stroke, heart failure and diabetes—Methodological challenges andsolutions,” BMC Med. Res. Methodol., vol. 13, p. 10, Jan. 2013.

[33] A. Sav et al., “Burden of treatment for chronic illness: A conceptanalysis and review of the literature,” Health Expectations, vol. 18, no. 3,pp. 312–324, Jan. 2013.

[34] B. Spring, S. Pagoto, R. Pingitore, N. Doran, K. Schneider, andD. Hedeker, “Randomized controlled trial for behavioral smoking andweight control treatment: Effect of concurrent versus sequential inter-vention,” J. Consulting Clin. Psychol., vol. 72, no. 5, pp. 785–796,Oct. 2004.

[35] M. Grant and S. Boyd, “Graph implementations for nonsmooth convexprograms,” in Recent Advances in Learning and Control (Lecture Notesin Control and Information Sciences), V. Blondel, S. Boyd, and H.Kimura, Eds. Springer-Verlag, pp. 95–110, 2008.

[36] S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan, Linear MatrixInequalities in System and Control Theory, vol. 15. Philadelphia, PA,USA: SIAM, 1994.

Korkut Bekiroglu received the B.S. degree fromthe Department of Electrical and Electronics Engi-neering, Karadeniz Technical University, Trabzon,Turkey, in 2005, the M.S. degree from the Depart-ment of Electrical Engineering, Northeastern Uni-versity, Boston, MA, USA, in 2010, and thePh.D. degree from the Department of ElectricalEngineering, The Pennsylvania State University,University Park, PA, USA, in 2010.

He is currently a Post-Doctoral Scholar with theMethodology Center, The Pennsylvania State Uni-

versity. His current research interests include the interface of system identi-fication, longitudinal data analysis, robust control, robust model predictivecontroller design, robust interventions, adaptive treatment, discrete eventdynamical systems, and robust optimization.

Constantino Lagoa received the B.S. andM.S. degrees from the Instituto Superior Técnico,Technical University of Lisbon, Lisbon, Portugal, in1991 and 1994, respectively, and the Ph.D. degreefrom the University of Wisconsin-Madison,Madison, WI, USA, in 1998.

He joined the Electrical Engineering Department,The Pennsylvania State University, UniversityPark, PA, USA, in 1998, where he is currently aProfessor. His current research interests includerobust optimization and control, chance constrained

optimization, controller design under risk specifications, system identification,control of computer networks, and application of control approaches to theanalysis and design of adaptive behavioral interventions.

Dr. Lagoa is an Associate Editor of the IEEE TRANSACTIONS ON

AUTOMATIC CONTROL and Automatica. He is also a member of the IFACTechnical Committee on Robust Control.

Suzan A. Murphy is currently the H.E. RobbinsDistinguished University Professor of Statistics,a Professor of Psychiatry, and a Research Professorwith the Institute for Social Research, Oslo, Norway.Her current research interests include improvingsequential, individualized decision making in health,in particular on clinical trial design and data analysisto inform the development of mobile health treat-ment policies.

Ms. Susan is a fellow of the Institute of Mathemat-ical Statistics, a fellow of The College on Problems

in Drug Dependence, a Former Editor of the Annals of Statistics, a memberof the U.S. National Academy of Sciences and the U.S. National Academyof Medicine, and a 2013 MacArthur Fellow.

Stephanie T. Lanza is currently a Professor ofBiobehavioral Health and the Scientific Director ofThe Methodology Center with The PennsylvaniaState University, University Park, PA, USA. Sheis currently involved in the areas of finite mixturemodeling and time-varying effect modeling, andis devoted to disseminating cutting-edge statisticalmethods to applied researchers. Her current researchinterests include advancing analytic methods rel-evant for studying multidimensional and dynamichealth behaviors.

Ms. Lanza is a Board Member of the Society for Prevention Research andan Associate Editor of the Prevention Science.

ieee transactions on control systems ...manuscript received june 5, 2015; revised march 22, 2016;...

Documents