Contract Design for Frequency Regulationby Aggregations of Commercial Buildings

Maximilian Balandat, Frauke Oldewurtel, Mo Chen and Claire Tomlin

Abstract— We investigate the contract design problem thatan energy aggregator who participates in the wholesale mar-ket for Ancillary Services faces. Specifically, we consider asituation in which commercial buildings agree to adjust theirheating, ventilation and air conditioning power consumptionwith respect to a nominal schedule, according to a signal sentby the aggregator. This signal may vary arbitrarily withina certain band, whose (time-varying) width is part of theagreed-upon contract between aggregator and building. Inreturn, the aggregator offers monetary rewards to incentivizethe individual buildings. This allows the aggregator to bundlethe resulting capacities from different buildings and sell thetotal capacity in the spot market for frequency regulation. Theaggregator’s problem is to jointly determine nominal schedules,regulation capacities and monetary rewards to maximize itsprofit, while ensuring that the individual buildings have anincentive to participate. Assuming that there is no privateinformation, we cast the contract design problem as a bileveloptimization problem, which we in turn reformulate as a mixed-integer program. We further show that if the building does notimpose negative externalities on the aggregator, the problemreduces to a Linear Program.

I. INTRODUCTION

The essence of reliable electricity grid operation is match-ing supply and demand at all times. Today, this is commonlyachieved by having generators and load-serving entities(LSE) participate in wholesale electricity markets. Thesemarkets run on different time-scales to benefit from bettershort-term demand and supply forecasts. At some point, how-ever, running an energy spot market becomes impractical.Instead, the grid operator procures generation capacities,so-called Ancillary Services (AS), that can be dispatchedon fast time-scales. The highest quality AS is frequencyregulation (up and down), over which the operator hasnear real-time control. Traditionally, these frequency controlreserves have been provided by conventional generators. Dueto the increasing penetration of renewable energy sources,more and more generation is uncertain and intermittent,which results in an increasing need for frequency regula-tion reserves [1], [2]. Instead of providing these capacityreserves solely using conventional generators, the mitigationof frequency deviations can also be supported by activating

The authors are with the Department of Electrical Engineeringand Computer Sciences, University of California, Berkeley, USA.[balandat,mochen72,tomlin]@eecs.berkeley.eduoldewurtel@berkeley.edu

This work has been supported in part by NSF under CPS:ActionWebs(CNS-931843) and CPS:FORCES (CNS-1239166). The research of F.Oldewurtel has received funding from the European Union Seventh Frame-work Programme FP7-PEOPLE-2011-IOF under grant agreement number302255, Marie Curie project ‘Stochastic Model Predictive Control, EnergyEfficient Building Control, Smart Grid’.

reserves on the demand side. This idea is not new; largeindustrial plants such as aluminum producers [3] have beenoffering grid services for a long time. More recently, therehave also been pilot programs for demand response (DR), inwhich buildings are asked to shed load when particular gridevents occur. However, load shedding is generally consideredundesirable, as it negatively impacts user utility and comfort.

This paper addresses the problem of grouping severalmid-to-large size commercial buildings together by an entitycalled aggregator, which uses the buildings’ inter-temporalconsumption flexibility in order to provide frequency regu-lation capacity in the AS market. Buildings have flexibilityin parts of their power consumption due to their inher-ent thermal storage capacity. Intelligent control of heating,ventilation and air conditioning (HVAC) systems facilitatesflexibility in power consumption without compromising oc-cupants’ thermal and air quality comfort. Commercial build-ings seem particularly well suited for this task due to theircomparatively large power consumption, advanced buildingmanagement systems, and their HVAC systems’ potential tofollow a high-frequency regulation signal [4].

In order to avoid load shedding or violating user com-fort it is desirable to plan ahead, i.e., to back off fromcomfort and actuator constraints to increase flexibility inpower consumption. This can for example be achieved byusing Model Predictive Control (MPC), see [5], [6], [7],[8] and references therein. The idea of using MPC to havecommercial buildings provide frequency regulation has beenproposed in [9], [10]. Like most of the initial work in thisdomain, [9] assumes that the objectives of aggregator andbuildings are perfectly aligned. However, unless aggregatorand buildings are under the same ownership, this will usuallynot be the case. Indeed, having buildings provide regulationcapacity typically requires less aggressive control strategiesand hence results in a more costly operation. An aggregatortherefore will need to compensate the building for its service.So while there may be a large potential in using buildings toprovide frequency regulation, a large part of this potential canonly be tapped by providing the correct financial incentives.While [10] investigates this problem, it does so in a ratherad-hoc way, and does not provide a structured approach todetermining “the best” incentives.

In this paper, limiting ourselves to the case without infor-mation asymmetries between buildings and aggregator, weformulate the optimal contract design problem an aggregatorfaces. We propose a contract structure in which a buildingagrees to adjust its power consumption according to aregulation signal within time-varying capacity bands around

978-1-4799-8009-3/14/$31.00 ©2014 IEEE 38

Fifty-second Annual Allerton ConferenceAllerton House, UIUC, Illinois, USAOctober 1 - 3, 2014

a nominal power schedule. In exchange, the building receivesa monetary reward from the aggregator. The financial risk forthe building due to following the regulation signal can beabsorbed by the aggregator by charging the building for itsenergy consumption according to its nominal schedule. Theresulting problem is a bilevel optimization problem in whichthe aggregator jointly optimizes over nominal schedule, ca-pacities and rewards in order to maximize its expected profit,subject to the buildings’ individual rationality constraints.Using techniques from integer programming we cast thisbilevel problem as an equivalent mixed-integer program.Moreover, we show that if buildings do not impose external-ities on the aggregator, determining the resulting “first-bestcontract” [11] reduces to solving a Linear Program.

Notation: For a matrix M let (M)i: and (M):j denote thei-th row and j-th column of M , respectively. Further, defineM+:= max(M, 0) and M−:= max(−M, 0) (element-wise).By × we denote element-wise vector multiplication, and byπA(x) the Euclidean projection of x onto A.

II. PRELIMINARIES

A. Wholesale Markets for Ancillary ServicesWe consider AS markets in deregulated wholesale elec-

tricity spot markets. Specifically, we focus on the frequencyregulation capacity market run by CAISO, the CaliforniaIndependent System Operator1.

Like energy markets, frequency regulation capacity mar-kets run on different timescales, namely the Day-Ahead Mar-ket (DAM) and the Real-Time Market (RTM). Each marketoperates as follows: After submitting its bids (consistingof maximum capacities and prices for both regulation upand regulation down), a resource gets awarded a regulationcapacity schedule based on a uniform price auction con-ducted by the ISO. The market clearing process is a large-scale optimization problem, in which capacity prices aredetermined from the dual variables (commonly referred toas “shadow prices”). At run-time, the resource then receivesa Load Frequency Control (LFC) signal ω from the ISO,specifying how much it should deviate from its nominalpower schedule (which is determined in the energy market).The LFC signal is constrained to the awarded regulationcapacities. The ISO has direct access to measurements of thepower output of the resource, and hence can verify whetherit has fulfilled its obligations. If so, the resource receives acapacity payment according to the market clearing price2.

B. The Role of the AggregatorWe are interested in using the temporal flexibility that

buildings have in terms of their HVAC power consumptionin order to offer regulation capacity to the grid. The roleof the aggregator is to provide an interface between thespot market on one side and the individual buildings on theother, as illustrated in Figure 1. There are many reasons why

1The basic market setup is quite similar among different ISOs in the US.2Recently, ISOs have been ordered by the federal regulator to implement

“pay for performance” compensation [12], which rewards resources for per-formance (e.g. mileage payments or signal tracking accuracy) in addition tocapacity. For simplicity we will restrict our attention to capacity payments.

Wholesaleelectricitymarket

Aggregator

ISOmarketinterface(defined,fixed)

Bilateralcontracts(designfreedom)

bids capacitypayments

LFCsignal

flexibilityincentives

regulationsignal

financialphysical

Fig. 1: The aggregator as an interface between market and buildings

individual buildings cannot easily participate in the whole-sale market directly. Some markets have minimum bid sizerequirements [13] that often exceed a single building’s powerconsumption. Moreover, participating in the market createsadditional overhead – not only does it involve acquiring thenecessary certifications from the ISO [14], but it also requiresunderstanding market processes and bidding strategies.

An aggregator with expertise in these areas, on the otherhand, can aggregate different buildings and bundle their indi-vidual flexibilities in such a way that the resulting product issuitable for sale in the spot market. In order for the individualbuildings to be willing to participate in such a scheme,the aggregator must provide them with financial incentives.While the market interface is well defined and clearlyspecified, the aggregator has large freedom in choosing thisincentive structure. In this paper we take a contract-theoreticpoint of view and consider the problem of maximizing theaggregator’s profit over all contracts within a certain class.

C. The Basic Contract Structure

The basic structure of the bilateral contracts we proposeis the following: For a given scheduling horizon, a buildingagrees to adjust its HVAC power consumption w.r.t. a nomi-nal power schedule, based on a regulation signal w it receivesfrom the aggregator that is guaranteed to lie within certainbounds. The building is charged for its energy consumptionaccording to this nominal schedule, and the aggregator paysthe building a one-time monetary reward Rb. In addition, inexchange for paying a penalty to the aggregator, the buildingmay specify a certain amount of slack from the maximum upand down regulation signal, which it is permitted to violate.Nominal schedule, capacity bounds, slack and reward are allpart of the contract specification.

Importantly, such a contract need not directly involve thebuilding’s power provider, e.g., the local utility. In fact, theprovider simply bills the building for its actual consumptionand need not even know of the contract (here we assumethat the provider is oblivious to the “double payment”problem [15]). The difference between this amount and thehypothetical cost under the nominal schedule can then besettled ex post between building and aggregator. Chargingthe building only for its nominal consumption is an importantfeature of our contract, as it does not expose the building toany financial risk associated with the regulation signal.

39

D. Dealing With the Different Timescales of the Problem

An interesting characteristic of the multi-level problemwe consider in this paper is that it takes place on threedifferent timescales, as illustrated in Figure 2. On the market

Fig. 2: Different time scales of the aggregation problem

level, bids, prices and awarded AS capacities are determinedfor market intervals of length TM (typically TM = 1 h forDAM and TM = 15 min for RTM). On the other extreme,at the level of frequency control, the ISO sends a LFCsignal ω that is piece-wise constant over intervals of lengthTR � TM (TR = 4 s in CAISO AS regions). The controlschedule for a building is determined for an intermediateinterval length TS , which depends on the time-constants ofthe building, the frequency with which set points can bechanged, and possible computational challenges with theresulting problem. Commonly, TS will lie in the range of5− 15 min. Here we make the following Assumption:

Assumption 1: There exist constants K,K ′ ∈ N such thatTS = KTR and TM = K ′TS , respectively.

We consider a contract design problem over a schedulinghorizon of NM market periods (e.g., NM = 24 for the Day-Ahead-Market). Let NS = K ′NM , NR = KNS and

TS(t) ={τ ∈ NS : t TM ≤ τTS < (t+ 1)TM

}TR(τ) =

{θ ∈ NR : τ TS ≤ θTR < (τ + 1)TS

}We also write tτ = T−1S (τ) and τθ = T−1R (θ).

III. A BUILDING’S SCHEDULING PROBLEM

A. Building Model

Consider a building b whose HVAC dynamics for sam-pling time TS are described by the discrete-time LTI system3

xbτ+1 = Abxbτ +Bb(ubτ + wbτ ) + Ebvbτ (1a)

ybτ = Cbxbτ +Db(ubτ + wbτ ) + F bvbτ (1b)

with known initial condition xb0 ∈ Rnbx. Here ubτ ∈ Rnburepresents the control inputs to the HVAC system, ybτ ∈ Rn

by

is the system output and vbτ ∈ Rnbv is the effect of weatherand occupancy in time interval t. The signal wb

τ ∈ Rnbuis a proxy for the regulation signal wb

θ received by theaggregator4. We write ub = (ub0 , . . . , u

bNS

) and will useanalogous notation for other variables throughout the paper.

We make the following assumption:Assumption 2: For each building, the power consumption

of each actuator is linear in the associated control input.

3The assumption of time-invariance is only made for notational conve-nience; the extension to linear time-varying systems is immediate.

4Under some mild conditions, which are satisfied by most buildingmodels, one can show that robustness of (1) w.r.t. wb

τ ∈ [−rb↓τ , rb↑τ ] impliesrobustness of the “true” fast-sampled system w.r.t. wb

θ ∈ [−rb↓τθ , rb↑τθ ].

Under Assumption 2, building b’s overall HVAC energyconsumption Qb

t in interval τ can be expressed as Qbτ =

(qb)>ubτ =∑nyi=1 q

bi u

bτ,i, where qbi ≥ 0 is the per-unit energy

consumption of actuator i over an interval of length TS .As weather and occupancy effects are usually uncertain,

we need to account for this uncertainty in the schedulingproblem. There are different ways of accomplishing this;here we choose to formulate chance constraints that requirethe building’s comfort constraints to be satisfied with givenlevels of probability. The weather and occupancy effect vbτcan be split into a known forecast vbτ and the forecast errorvbτ := vbτ − vbτ . We assume that – based on the analysis ofhistorical data – a linear uncertainty model of the form

vbτ+1 = Gbvbτ +Hbebτ (2)

has been identified, where ebτd= N (0, I) are i.i.d. Gaussian

and vb0d= N (0,Σb

0) with ebτ and vb0 independent for all τ .Some algebra allows us to write the system output ybτ as

ybτ = ybτ (ub) +∑ι<τ W

bτ−1−ιw

bι +Db

τ τbτ

+ Gbτ vb0 +∑ι<τ Hb

τ,ιebι

(3)

where

ybτ (ub) = CbDbubτ + CbF bvbτ + Cb(Ab)τxb0

+∑ι<τ C

b(Ab)τ−1−ι(Bbubι + Ebvbι

) (4)

is the nominal system output, W bτ = Cb(Ab)τBb, and

Gbτ =∑ι<τ C

b(Ab)τ−1−ιEb(Gb)ι + F b(Gb)τ (5)

Hbτ,ι =

∑ι<θ<τ C

b(Ab)τ−1−θEb(Gb)θ−1−ιHb

+ F b(Gb)τ−1−ιHb (6)

B. The Robust Optimal Scheduling Problem

Denote by rb↑ = (rb↑0 , . . . , rb↑NS ) and rb↓ = (rb↓0 , . . . , rb↓NS )the sequences of non-negative vectors of “regulation up”and “regulation down” capacities for building b, respec-tively5. Suppose that at each time τ , the building receivesa regulation signal wb

τ ∈ [−rb↓τ , rb↑τ ]. For the purpose ofthis section suppose that the capacity vectors rb↑ and rb↓

are fixed and known. It may be infeasible for a buildingto tolerate all possible sequences of the regulation signalwithin [−rb↓, rb↑] without jeopardizing occupant comfort.Hence we allow the building to shrink the interval withinwhich it is required to robustly follow the regulation signalto [−rb↑τ + sb↑τ , r

b↓τ − sb↓τ ], where sb↑τ and sb↓τ with

0 ≤ sb↓τ ≤ rb↓τ , 0 ≤ sb↑τ ≤ rb↑τ ∀ τ ≤ NS (7)

are the slacks. We define the effective regulation signal

wτ = π[−rb↑τ +sb↑

τ , rb↓τ −sb↓

τ ](wbτ ) (8)

In exchange for the reduced uncertainty resulting from this,the building pays a penalty Pb(sb) to the aggregator, where

5Our convention is to use the classic point of view of generation marketsin which “regulation up” means that a generator increases its power outputand “regulation down” means that he decreases it. Thus, from the point ofview of a load, “regulation up” means a decrease while “regulation down”means an increase in power consumption w.r.t. the nominal schedule.

40

sb = (sb↑, sb↓). Given rb and sb, the building seeks thenominal schedule of minimum expected cost that ensuresconstraint satisfaction for all possible sequences of wb.

1) Input Constraints: The actual control inputs ubτ + wbτ

at time τ are restricted by physical limits on the actuators,which translate into the following constraints:

ubτ ≤ ubτ + wbτ ≤ ubτ ∀ τ ≤ NS (9)

Since the regulation signal is unknown at scheduling time,the above constraints have to hold for any regulation signalwbτ ∈ [−rb↑τ + sb↑τ , r

b↓τ − sb↓τ ]. This can be achieved by

imposing the following robustified version of (9):

ubτ + rb↑τ − sb↑τ ≤ ubτ ≤ ubτ − rb↓τ + sb↓τ ∀ τ ≤ NS (10)

Importantly, (10) deals with the uncertainty introducedby wb in a robust fashion, i.e., it accounts for the worstcase, so the building need not make any assumptions on thestatistical properties of wb.

2) Output Constraints: The building aims to ensure that,for all possible sequences of the signal wb, the output yb

satisfies the building’s comfort constraints. Since small vio-lations of comfort constraints are usually non-critical, weassume the building formulates chance constraints, whichensure that the constraints at any point in time are satisfiedwith a given level of probability. Specifically, suppose that,for all possible sequences of wb, the comfort constraintsybτ,i ≥ yb

τ,iand ybτ,i ≤ ybτ,i are to be satisfied with a

probability of at least 1−αbτ,i and 1−αb

τ,i, respectively (hereαbτ,i, α

bτ,i < 0.5 are given, and typically αb

τ,i, αbτ,i � 1).

Let Wbτ =

{(w0, . . . , wτ ) : − rb↑ι + sb↑ι ≤ wι ≤ rb↓ι −

sb↑ι , ∀ ι ≤ τ}

. The constraints on the output then read

∀ τ,∀ i,∀ w ∈ Wbτ ,

{P(ybτ,i ≥ ybτ,i

)≥ 1− αb

τ,i

P(ybτ,i ≤ ybτ,i

)≥ 1− αb

τ,i

(11)

Using techniques from robust optimization and chance-constrained programming, the above chance constraints forτ = 0, . . . , NS and 1 ≤ i ≤ nby can be expressed as

ybτ,i(ub) ≥ yb

τ,i+∑ι<τ

[(W bτ−1−ι

)+i:

(rb↑ι − sb↑ι )

+(W bτ−1−ι

)−i:

(rb↓ι − sb↓ι )]+(Db)+i:(r

b↑τ − sb↑τ )

+ (Db)−i:(rb↓τ − sb↓τ )+ σb

τ,i Φ−1(1−αbτ,i)

(12a)

ybτ,i(ub) ≤ ybτ,i−

∑ι<τ

[(W bτ−1−ι

)+i:

(rb↓ι − sb↓ι )

+(W bτ−1−ι

)−i:

(rb↑ι − sb↑ι )]−(Db)+i:(r

b↓τ − sb↓τ )

− (Db)−i:(rb↑τ − sb↑τ )− σb

t,i Φ−1(1−αbτ,i)

(12b)

where (σbτ,i)

2 = (Gbτ )i:Σb0(Gbτ )>i:+

∑ι<τ (Hb

τ )i:(Hbι )>i: and Φ

is the cdf of a standard normal random variable.Note that, importantly, both (12a) and (12b) are linear in

the control, the capacities and the slack variables.3) The Robust Scheduling Problem for Fixed Capacities:

Let cb = (cb0 , . . . , cbNS

) be the vector of electricity prices forthe building over the scheduling horizon6. As discussed in

6We assume cb is known over the whole scheduling horizon, but mayotherwise be arbitrary (i.e. cb can describe both flat and Time-of-Use tariffs).

section II-C, in the contract we propose a building pays onlyfor its power consumption according to nominal schedule.That is, the energy cost (excluding Pb) of building b is

ϕbnom(ub) =

∑NSτ=1c

bτ (qb)>ubτ

In practice the building will still pay the utility

ϕbact(u

b, wb)=∑NSτ=1c

bτ (qb)>ubτ+

∑NSτ=1c

bτ(qb)>

K

∑θ∈TR(τ)w

bθ

for its actual consumption, while the difference

∆ϕb(wb) =∑NSτ=1c

bτ(qb)>

K

∑θ∈TR(τ)w

bθ

is settled ex post with the aggregator. Here the signal wb isthe effective regulation signal received with frequency 1/TR.

The building’s overall robust scheduling problem for givenregulation capacity vectors rb↓, rb↑ is therefore

(ub∗, sb∗)(rb↓, rb↑) = arg minub,sb

ϕbnom(ub) + Pb(sb)

s.t. (7), (10), (12)(13)

We make the following assumption:Assumption 3: The building penalties Pb are of the form

Pb(sb)=∑

b

∑NSτ=1(p↑ls

↑τ + p↑q(s

↑τ )2+ p↓l s

↓τ + p↓q(s

↓τ )2)

where p↑l, p↓l ≥ 0 and p↑q, p

↓q ≥ 0 are fixed coefficients.

Under Assumption 3, (13) is a Quadratic Program (QP).

IV. THE OPTIMAL CONTRACT DESIGN PROBLEM

In this section we formulate the aggregator’s optimalcontract design problem for a given collection B of buildings.To this end we require some additional assumptions.

Assumption 4: The aggregator is risk-neutral.Assumption 5: The aggregator has no market power in the

wholesale market for frequency regulation capacity.Assumption 6: Buildings do not possess any private in-

formation. That is, for each b ∈ B the aggregator hasfull knowledge of the system model (1), constraints, energyconsumption rates qb and tariff cb. Moreover, it has accessto measurements of the initial state xb0 and the HVAC powerconsumption on the frequency 1/TR of the LFC signal ω.

Assumption 4 is standard and made primarily for mathe-matical convenience. An extension to a risk-averse aggrega-tor, while desirable, appears quite challenging at this point.

Assumption 5 holds if the total capacity that can be pro-vided by all buildings is small enough that the aggregator’sbid does not significantly affect the overall supply curve.Then the aggregator acts as a price-taker and places its bidat true cost. Due to Assumptions 2 and 4 it suffices that theaggregator has access to an unbiased estimate of the expectedcapacity prices over the scheduling horizon.

Assumption 6 is more restrictive. In practice, an aggregatorwill not know the exact building models, which causesinformation asymmetries. It would be interesting to analyzehow much of a so-called information rent a building couldearn in this case, but this is outside the scope of this paper.

41

A. Objective Function

Denote by rM↑t and rM↓t the regulation up and downcapacities offered in the market in period t, respectively.Let rM = (rM↑, rM↓) and, similarly, R = {Rb : b ∈ B},r = {(rb↓, rb↑) : b ∈ B} and s = {(sb↓, sb↑) : b ∈ B}.

The deviations εbθ := wbθ−wb

θ of building b’s control fromthe dispatch signal uτθ + wb

θ are given by

εbθ = (wbθ + rb↑τθ − sb↑τθ )− − (wb

θ − rb↓τθ + sb↓τθ )+

Observe that εbθ = 0 if the slacks sb↑τθ , sb↓τθ

are zero. With

wMθ =

{−ωθ rM↑tτθ

if ωθ ≥ 0

−ωθ rM↓tτθif ωθ < 0

(14)

denoting the aggregator’s regulation signal derived from theLFC signal ωθ ∈ [−1, 1], the deviation εMθ of the buildings’aggregate consumption from the ISO dispatch signal is

εMθ =∑

b(qb)>wbθ − wMθ +

∑b(qb)>εbθ

Using the above we can write

E[∑

b∆ϕb(wb)]=∑

b

∑NSτ=1c

bτ

∑θ∈TR(τ)

(qb)>

K E[wbθ + εbθ

]Under Assumption 4 the aggregator aims to maximize its

expected profit

ψ(rM, r, s, R)= E[∑NM

t=1

(ρ↑t r

M↑t + ρ↓t r

M↓t

)−PM (εM )

−∑b

(Rb+∆ϕb(wb)− Pb(sb)

)] (15)

which consists of the expected revenue generated in the spotmarket, a penalty P(εM ) incurred for deviations from themarket dispatch signal and, for each building, the reward Rb,the cost difference ∆ϕb and the building’s penalty Pb. Atypical market penalty would for example be

P(εM ) =∑NRθ=1 ph(εMθ )+ + pl(ε

Mθ )− (16)

where ph, pl ≥ 0 are the per-unit penalty factors7.So far we have not discussed how the aggregator would

determine the signals wbθ from the ISO’s LFC signal ωθ.

While very interesting8, this problem is beyond the scope ofthis paper. We make the following simplifying assumption:

Assumption 7: The contributions of the different buildingsto the provided overall modulation of the demand are de-termined in a “pro rata” fashion, i.e., proportional to theirrespective contracted capacities. Specifically, we assume that

wbθ =

{−ωθ rb↑τθ if ωθ ≥ 0

−ωθ rb↓τθ if ωθ < 0(17)

for all b ∈ B and θ = 0, . . . , NR.Not surprisingly, the aggregator’s payoff will depend on

the statistical properties of the LFC signal ω. Since this signaldepends directly on the mismatch of supply and demand inthe grid, which the ISO consistently tries to minimize, it isquite natural to make the following assumption:

7-PM can also be interpreted as a “pay for performance” payment [12].8The overall demand modulation must be split not only between the

different buildings but also between different actuators within the buildings.

Assumption 8: For all θ = 0, . . . , NR the LFC signal ωθis a random variable supported on [−1, 1] with a continuousdistribution that is symmetric around zero.

Using Assumptions 7 and 8 we can compute

E[wbθ

]= ζθ

(rb↓τθ − rb↑τθ

)(18a)

E[εbθ]

= E[(

(1− ωθ)rb↑τθ − sb↑τθ)−

1{ωθ≥0}]

− E[(−(1 + ωθ)r

b↓τθ

+ sb↓τθ)+

1{ωθ<0}]

(18b)

where ζθ := 0.5E[ωθ |ωθ ≥ 0]. Hence E[wbθ

]is linear in the

capacities, and this is independent of the distribution of ωθ.This is not the case for E

[εbθ], i.e., the effect of rbτθ and sbτθ

on E[εbθ]

on is distribution-dependent. E.g., if ωθ ∼ U [−1, 1]one can find that E[εbθ ] = 1

4

[(sb↑τθ )2/rb↑τθ − (sb↓τθ )2/rb↓τθ

].

From now on we will assume for simplicity that ζθ = ζ forall θ, which is for example the case if the ωθ are identicallydistributed. From (18) and Assumption 8 it follows thatE[wbθ + εbθ

]= 0 if capacities and slacks are symmetric.

B. Individual Rationality Constraints

Observe that for all buildings b ∈ B we have that

ϕb∗oo := ϕb

nom(ub∗(0 , 0)) ≤ ϕbnom(ub∗(rb↓, rb↑))

for any capacity vectors rb↓, rb↑. We refer to ϕb∗oo as the

value of the outside option of building b, since it representsthe minimum (nominal) energy cost without contract. Theaggregator must then ensure that the overall cost incurredby building b (including the monetary reward Rb) whenparticipating in the contract does not exceed ϕb∗

oo .

C. The Aggregator’s Optimization Problem

The aggregator’s optimal contract design problem can nowbe stated as the following bilevel optimization problem [16]:

maxrM,r,u,s,R

ψ(rM, r, s, R) (19a)

subject to

rM↑t ≥ 0, rM↓t ≥ 0 ∀ t≤NM (19b)

rb↑τ ≥ 0, rb↓τ ≥ 0 ∀b∈B, ∀ τ≤NS (19c)

(ub, sb) solves (13) ∀b∈B (19d)ϕb(ub, sb)−Rb ≤ ϕb∗

oo ∀b∈B (19e)

In the above problem, (19d) requires each building tobehave optimally w.r.t. its capacity vectors rb↓ and rb↑.Participation of each building is ensured by the individualrationality constraint (19e).

Observing that Rb does not affect (19d) and that (15)is strictly decreasing in Rb, it is clear that the individualrationality constraint (19e) will be tight at the optimum.Substituting the resulting equality into the objective we get

ψ(rM, r, s)= E[∑NM

t=1

(ρ↑t r

M↑t + ρ↓t r

M↓t

)−PM (εM )

−∑bϕbnom(ub)−ϕb∗

oo +∆ϕb(wb))] (20)

and with that (19) is equivalent to

maxrM,r,u,s

ψ(rM, r, s) s.t. (19b), (19c), (19d) (21)

42

If the nominal scheduling problems for all buildings arefeasible, then (19) will always have a solution: if rM = r =s = 0 and ub = ub∗(0 , 0) for all b then (19b) and (19c)hold with equality, and (19d) holds by definition of ub∗.

V. SOLUTION METHODOLOGY

The aggregator’s optimization problem (19a) is a bileveloptimization problem, for which different solution ap-proaches exist. These include vertex enumeration techniques,branch-and-bound (BNB) algorithms, penalty function meth-ods, descent algorithms and trust-region methods [16]. Wefocus on BNB techniques since they can provide certificatesfor global optimality of the solution based on duality theory.

A. The General Case

The basic idea of BNB methods for bilevel optimizationproblems is to replace the lower level problem by a set ofsuitable optimality conditions. In our case the lower levelproblem is a QP, so the corresponding Karush-Kuhn-Tucker(KKT) optimality conditions are necessary and sufficient.Hence the bilevel problem can be written equivalently as astandard optimization problem in which the KKT conditionsof the lower level problem appear as constraints9.

Since for QPs the primal feasibility, dual feasibility andLagrangian stationarity conditions are all affine constraints,the only challenge are the complementary slackness (CS)conditions. For example, the corresponding CS conditionsfor the (upper) robustified input constraints (10) are given by

λbτ ×(−ubτ + ubτ + rb↑τ − sb↑τ

)= 0, ∀ τ = 0, . . . , NS

where 0 ≤ λbτ ∈ Rnbu are the dual variables.There are two main ways of dealing with the CS condi-

tions. One is to directly apply a specialized BNB algorithm,the other one is to reformulate the problem by introducingauxiliary binary variables and bounding primal constraintsand dual variables10. Taking the latter approach, we introducevariables zbτ ∈ {0, 1}n

bu , and express primal feasibility, dual

feasibility and CS conditions jointly by

−Mubτ× zbτ ≤ −ubτ + ubτ + rb↑τ − sb↑τ ≤ 0 (22a)

0 ≤ λbτ ≤ (1− zbτ )×Mλbu,τ

(22b)

where Mubτ,Mλb

τ∈ Rn

bu

+ are sufficiently large constants. Allother CS conditions can be handled analogously.

Thus, the bilevel problem (21) is equivalent to

maxrM,r,u,s,λ,z

ψ(rM, r, s) s.t. (19b), (19c), (CS), (LS) (23)

where (CS) denotes a set of constraints of the form (22), and(LS) represents the Lagragian stationarity constraint, whichis an affine equality constraint.

In general, (23) is a Mixed-Integer optimization problemwith linear constraints and nonlinear objective, the specificform of which depends on the kind of market penalty and

9With the dual variables of the lower level problem now appearing asprimal variables of the overall problem.

10This approach is often referred to as a “big-M” formulation [17].

the shape of the distribution of the LFC signal ωθ. Notethat (23) is highly structured. In particular, the constraintsare completely decoupled. This suggests that the problemshould be amenable to decomposition techniques that aim atseparating the coupling term PM in the objective.

An important feature of this formulation is that, since theKKT conditions of the buildings’ problems appear explicitlyas constraints, any feasible solution of (23) describes asituation in which all buildings behave optimally w.r.t. theirregulation capacities. Hence it is not necessary to find theglobal optimum in order to ensure individual rationality.

B. The First-Best Contract

In general, the buildings’ decisions impose externalitieson the aggregator via their effect on the distributions of εM

and wb. If the aggregator wants to avoid being penalized fordeviations from schedule in the market and therefore insiststhat s = 0, i.e., that all buildings are robustly providing theirfull capacities rb, then the problem can be transformed intoa single Linear Program.

Indeed, if sb =0 for all b, then Pb(sb)=0, wb =wb forall b, and εM = 0. Hence we see from (20) that the onlyterm in the objective that depends on building b’s decisionvariables is ϕb

nom. Moreover, it is clearly optimal for theaggregator to commit all available capacity in the market:

rM↑t =∑

b(qb)>rb↑τrM↓t =

∑b(qb)>rb↓τ

}, ∀ t≤NM , τ ∈TS(t) (24)

Writing Ub(rb) = {ub : (10), (12) hold with sb = 0} andR = {(rM, r) : rM, r ≥ 0, (24) holds} we have

max(rM,r)∈R

E[∑NM

t=1

(ρ↑t r

M↑t + ρ↓t r

M↓t

)+∑

b ϕb∗oo

−∑b

(minub∈Ub(rb) ϕ

bnom(ub)+∆ϕb(wb)

)]= max

(rM,r)∈Rub∈Ub(rb),∀b

∑NMt=1

(E[ρ↑t ] r

M↑t + E[ρ↓t ] r

M↓t

)+∑

b ϕb∗oo

−∑b

(ϕb

nom(ub)+E[∆ϕb(wb)

]) (25)

where

E[∑

b∆ϕb(wb)]=∑

b

∑NSτ=1c

bτ

∑θ∈TR(τ)

(qb)>

K ζθ(rb↓τθ − rb↑τθ

)is linear in the decision variables. Finally R and Ub aredescribed by a set of linear constraints, so (25) is an LP.Economically speaking, there is no Moral Hazard, so theaggregator is able to implement what is known as the first-best contract [11] in contract theory (note that there is also noAdverse selection as there are no information asymmetries).Note however that the first-best contract in this setting neednot be optimal for the aggregator. This is the case if thebenefit from the buildings’ reduced costs for adjusting theirschedule under less strict robustness requirements outweighsthe expected deviation penalties incurred in the market.

VI. NUMERICAL EXAMPLE

In this section we apply our methodology for the first-best contract to a simple numerical example with fourbuildings. For each building, we use a scaled version of a

43

simple one-dimensional model identified by applying semi-parametric regression techniques on a testbed on the UCBerkeley campus [18]. We emphasize that our simulationsare primarily for illustration and for showing some importantqualitative properties of the optimal contract. In order to getmeaningful quantitative results on the economic feasibilityof such a contract scheme, a much more thorough study isnecessary. For our example we consider a scheduling horizonof TM = 24 and scheduling intervals of length TS = 15 min,so that NS = 96 and K ′ = 4.

A. Model Parameters

1) Building Models: We generate four slightly perturbedversions of the model identified in [18], where the distur-bance v is given by a weighted sum of outside temperatureand heating load from occupants, equipment, and solarheating. For each model we assume a maximum power con-sumption typical for mid-to-large size commercial buildings,from which we determine the consumption factors q. For allmodels we have C = E = 1 and D = F = 0, with theremaining parameters given in Table I. Comfort constraintsare chosen so as to have a tight temperature band duringcommon working hours, and looser band outside commonworking hours. We assume that αt,i = αt,i = 5%, Σ0 = 0.5and u = 0 for all buildings.

b A B x0 [◦C] q [kWh] u G H tariff1 0.64 −2.64 23 43.75 0.5 0.2 0.15 A10 TOU2 0.68 −2.55 22 62.5 0.75 0.25 0.2 E19 TOU3 0.635 −2.7 24 31.25 0.65 0.15 0.1 A10 TOU4 0.62 −2.65 22.5 50.0 0.5 0.1 0.15 A10

TABLE I: Building model parameters

2) Electricity Tariffs: Electricity costs cb were taken asthe energy charges11 of various PG&E commercial electricitytariffs (summer period) offered in California [19].

3) Capacity Prices and LFC signal: Figure 3 showsCAISO DAM regulation up and down prices for August2012, with the respective averages (bold) for each hour acrossthe whole month. We found that with these capacity prices

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Time of day

0

5

10

15

20

25

30CAISO DAM regulation capacity prices, August 2012 ($/MW)

reg. up

reg. down.

0

20

40

60

80

100

120

140

Fig. 3: CAISO DAM regulation capacity prices for August 2012

the optimal strategy for the aggregator is to not bid anycapacity in the market. This is primarily because of the smallthermal capacity and inefficient HVAC system of our simplemodel. As a result the increased cost for deviating from thecost-optimal power schedule outweighs the potential revenue

11In addition to the conventional energy charge, tariffs A10, A10 TOUand E19 TOU also have a monthly peak demand charge. For simplicity weassume here that this peak is not affected by the modified schedules. Pricesgiven here are for the secondary distribution voltage level.

in the capacity market. In order to still be able to illustratesome qualitative properties of the optimal contract, we havescaled the expected capacity prices ρ↑ and ρ↓ by a factor 20and 15, respectively, for our simulations. Finally, we assumethat the ISO’s LFC signals ωθ are i.i.d. uniformly on [−1, 1],which implies that ζ = 0.25.

B. Simulation Results

The optimal contract design problem for the DAM for theabove parameters was solved in 0.09 seconds on a Laptopcomputer using Gurobi [20]. The aggregator’s expected profitis $20.7, the rewards Rb are $28.4, $24.1, $9.0 and $17.5for buildings 1 − 4. Hence for the simple building modelused in this simulation, the aggregator must spend a largeshare of its revenue on incentivizing the buildings.

Figure 4 shows how the individual buildings contributeto the overall regulation capacities offered in the market.Observe how the contributions vary on the faster schedulingfrequency on the building level to provide capacity on theslower market time scale in the most cost-effective way.

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Time of day

0

20

40

60

80

100kW

Allocation of regulation capacities between buildings

rM↑

Building 1

Building 2

Building 3

Building 4

rM↓

Building 1

Building 2

Building 3

Building 4

Fig. 4: Shares of the overall regulation capacities between buildings

Temperature evolutions and control inputs for Building 1are shown in Figure 5, where uoo and u∗ are the optimalschedules of outside option and optimal contract, respec-tively. The regulation signal w has been sampled uniformlyfrom within the contracted bounds. In order to be able toclearly observe the behavior of the optimal control, we haveset v = 0 in Figure 5. The shaded areas in the lower

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 2316

18

20

22

24

26

28

◦ C

Temperature evolution - Building 1

y(uoo)

y(u∗)

y(u∗ + w)

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Time of day

0.0

0.1

0.2

0.3

0.4

0.5

Control input - Building 1

uoo

u+ r↑

u− r↓u∗

u∗ + w

Fig. 5: Temperature and control for Building 1

plot represent the amounts by which the actuator constraintshave been tightened in order to be able to ensure thatthe resulting nominal control schedule is robustly feasible

44

(observe that the control u∗ + w indeed never violates theactuator constraints).

Figure 6 shows the temperatures for Building 1 for 30randomly sampled errors sequences v, again assuming auniformly distributed LFC signal ω. The individual comfortconstraints are violated in 0.57% of all time intervals.

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Time of day

16

18

20

22

24

26

28

◦ C

Temperature evolution for 30 samples - Building 1

Fig. 6: Sample temperature trajectories for Building 1

Finally, Figure 7 shows the strong dependency of the totalamount of contracted capacity over the scheduling horizonand the aggregator’s expected profit on the parameter ζ, andhence on the shape of the distribution of ω. Noting thathigher values of ζ correspond to a high variance of ω, thisis not surprising. Indeed, when contracting regulation downcapacity this would result in the cost difference ∆ϕ(w) likelybeing large, so in expectation the aggregator would needto pay a large amount to ensure that buildings are onlycharged according to nominal schedules. For small ζ thedistribution of ω is concentrated around zero and this issue isless pronounced. The situation is mirrored for regulation upcapacity; in this case large values of ζ are beneficial for theaggregator. Figure 7 also shows that the overall amount of

0.1 0.2 0.3 0.4 0.5

ζ

0

500

1000

1500

kW

h

Effect of ζ on regulation capacities and aggregator profit

rM↑ rM↓ profit

10

20

30

40

50

60

$

Fig. 7: Effect of the parameter ζ

capacity offered is much higher for regulation down. This isdue to the fact that in many cases regulation down capacitycan be offered without having to deviate from the optimalschedule of the outside option.

VII. CONCLUSION AND FUTURE WORK

In this paper we formulated the optimal contract designproblem an aggregator faces when using the inter-temporalflexibility of buildings’ HVAC consumption in order toparticipate in the regulation capacity spot market. We showedhow to formulate the resulting bilevel optimization problemas a mixed-integer problem, and how in the absence of MoralHazard the first-best contract can be found by solving aLinear Program. Our methodology is quite general and canbe used for a wide class of related problems characterizedby the aggregation of agents with competing objectives. Weillustrated the practicality of our approach by applying it toa simple numerical example.

We believe that the use of commercial buildings to providefrequency regulation capacity can be a viable option in

certain market environments. With this work we provide aframework that can be used to analyze the feasibility ofthe business model of an aggregator. While our simulationresults do provide some interesting qualitative insights intothe contract, a more thorough quantitative study is necessaryin order to assess the economic feasibility of this scheme intoday’s market structure. In the future, we hope to extendour work to more general building models, incorporateinformation asymmetries, and take into account new marketinstruments such as “Regulation Energy Management” [14].

ACKNOWLEDGEMENT

We thank Clay Campaigne for interesting discussions.

REFERENCES

[1] D. Halamay, T. Brekken, A. Simmons, and S. McArthur, “Reserverequirement impacts of large-scale integration of wind, solar, andocean wave power generation,” Sustainable Energy, IEEE Transactionson, vol. 2, no. 3, pp. 321–328, July 2011.

[2] Y. Makarov, C. Loutan, J. Ma, and P. de Mello, “Operational impactsof wind generation on california power systems,” Power Systems, IEEETransactions on, vol. 24, no. 2, pp. 1039–1050, May 2009.

[3] D. Todd, M. Caufield, B. Helms, M. R. Starke, B. J. Kirby, and J. D.Kueck, “Providing Reliability Services through Demand Response: APreliminary Evaluation of the Demand Response Capabilities of AlcoaInc.” 2009.

[4] H. Hao, A. Kowli, Y. Lin, P. Barooah, and S. Meyn, “Ancillary servicefor the grid via control of commercial building hvac systems,” inAmerican Control Conference (ACC), 2013, June 2013, pp. 467–472.

[5] G. P. Henze, D. E. Kalz, S. Liu, and C. Felsmann, “Experimentalanalysis of model-based predictive optimal control for active andpassive building thermal storage inventory,” HVAC&R Research,vol. 11, no. 2, pp. 189–213, 2005.

[6] F. Oldewurtel, “Stochastic Model Predictive Control for Energy Effi-cient Building Climate Control,” Ph.D. dissertation, ETH Zurich, 2011.

[7] Y. Ma, F. Borrelli, B. Hencey, B. Coffey, S. Bengea, and P. Haves,“Model predictive control for the operation of building cooling sys-tems,” Control Systems Technology, IEEE Transactions on, vol. 20,no. 3, pp. 796–803, May 2012.

[8] A. Aswani, N. Master, J. Taneja, A. Krioukov, D. Culler, and C. J.Tomlin, “Energy-Efficient Building HVAC Control Using HybridSystem LBMPC,” in Proceedings of the 4th IFAC Nonlinear ModelPredictive Control Conference, Leeuwenhorst, NL, 2012, pp. 496–501.

[9] E. Vrettos, F. Oldewurtel, F. Zhu, and G. Andersson, “Robust Pro-vision of Frequency Reserves by Office Building Aggregations,” inProceedings of the 19th IFAC World Congress, August 2014.

[10] M. Maasoumy, C. Rosenberg, A. Sangiovanni-Vincentelli, and D. S.Callaway, “Model Predictive Control Approach to Online Computationof Demand-Side Flexibility of Commercial Buildings HVAC Systemsfor Supply Following,” in American Control Conference (ACC), Port-land, OR, USA, 2014.

[11] J.-J. Laffont and D. Martimort, The Theory of Incentives: ThePrincipal-Agent Model. Princenton University Press, 2002.

[12] FERC, “Order no. 755: Frequency regulation compensation in theorganized wholesale power markets,” Oct 2011.

[13] E. Cutter, L. Alagappan, and S. Price, “Impact of market rules onenergy storage economics,” in 32nd IAEE Int. Conf., vol. 24, Jun 2009.

[14] California Independent System Operator Corporation, “FifthReplacement FERC Electric Tariff,” May 2014.

[15] B. S. Black and R. J. Pierce, Jr., “The Choice between Marketsand Central Planning in Regulating the U. S. Electricity Industry,”Columbia Law Review, vol. 93, no. 6, pp. 1339–1441, 1993.

[16] B. Colson, P. Marcotte, and G. Savard, “An overview of bileveloptimization,” vol. 153, no. 1, pp. 235–256, 2007.

[17] H. P. Williams, Logic and Integer Programming, ser. OperationsResearch & Management Science. Springer, 2009.

[18] A. Aswani, N. Master, J. Taneja, V. Smith, A. Krioukov, D. Culler, andC. Tomlin, “Identifying models of hvac systems using semiparametricregression,” in American Control Conference (ACC), 2012.

[19] PG&E, “Commercial electricity tariffs,” May 2014.[20] Gurobi Optimization, “Gurobi Optimizer Reference Manual,” 2014.

45