energy packet networks with energy harvesting

SPECIAL SECTION ON SMART GRIDS: A HUB OF INTERDISCIPLINARY RESEARCH

Received January 25, 2016, accepted February 15, 2016, date of publication March 25, 2016, date of current version April 15, 2016.

Digital Object Identifier 10.1109/ACCESS.2016.2545340

Energy Packet Networks With Energy HarvestingEROL GELENBE, (Fellow, IEEE), AND ELIF TUG̃ÇE CERANIntelligent Systems and Networks Group, Electrical and Electronic Engineering Department, Imperial College, London SW7 2AZ, U.K.

Corresponding author: E. Gelenbe ([email protected])

This work was supported in part by the U.K. ERA-NET Engineering and Physical Sciences Research Council under Grant EP/K017330/1and in part by the U.K. Ministry of Defence within the Defence Science and Technology Laboratory under Grant DSTLX1000064083and Grant EESDP39623.

ABSTRACT We investigate the cooperation among energy prosumers (unified energy provider andconsumer) through the energy packet network (EPN) paradigm, which represents both the flow of workthat requires energy, and the flow of energy itself, in terms of discrete units. This paper details a stochasticmodel of EPNs, which is inspired from a branch of queuing theory called G-networks. Themodel allows us tocompute the equilibrium state of a system that includes energy storage units, energy transmission networks,and energy consumers, together with the intermittent energy sources. The model is then used to show howthe flow of work and energy in the system can be optimized for certain utility functions that consider boththe needs of the consumers, and the desire to maintain some reserve energy for potential future needs.

INDEX TERMS Renewable Energy, energy storage, energy packet networks, energy prosumers, queueingtheory, G-networks, system optimisation.

I. INTRODUCTIONIn the future, billions of computer and communicationdevices may work through the Internet of Things (IoT) tomanage and solve locally the major challenges of cities andhuman communities [1]. Applications will include energyprovisioning and the smart grid [2], physical safety andsecurity [3], health [4] and the environment [5]. In particu-lar, the smart grid will itself require innovative communica-tion solutions [6]–[10], but distributed data sources requiresynchronisation [11] while communication protocols createadditional overhead [12] and hence energy consumption.

However energy and workload monitoring and manage-ment systems will also consume energy and add to societalissues such as CO2 impact [13]. Thus in order to moveforward towards ubiquitous, self-sustainable and optimisedsystems such as smart cities, sustainable agriculture, energymicrogrids, and local energy supply systems, with desirableproperties such as reduced emissions and energy transmissionlosses, it will be necessary to power the IoT as much as possi-ble through sources of renewable and clean energy and meetthe energy needs of the communication networks, servers anddata centres that are needed to support them [14].

In the meanwhile, the worldwide increase in electricalenergy consumption for ICT [15] worldwide in the orderof 5-7% per year, has also motivated research in energyharvesting [16]–[19] for computer and communications sys-tems. Furthermore power is being increasingly generated in adistributed manner with consumers becoming ‘‘prosumers’’,

who may also be producers and/or ‘‘storers’’ of energy (e.g.in home-based battery systems, or in individual vehicles).Thus there is an increasingmotivation to dynamicallymanageelectrical energy consumption, in conjunction with its gener-ation and distribution. Such dynamic management requiresthe integration of consumers with communication networksand data centres so that energy management software maydynamically optimise the power grid.

These evolutions encourage us to study integrated modelsfor renewable energy systems in the presence of dynamic andflexible energy consumption [20]. In this respect, a varietyof emerging technologies such as Intelligent Power Switches,PowerComm Interfaces [21] andwireless energy transfer [22]can assist in dynamically managing, storing and conveyingelectrical energy. Furthermore, energy storage in conjunc-tion with harvesting, energy sharing between prosumers, andworkload migration between certain energy consumers suchas data centres, can be used to improve the economic cost andCO2 impact of the data centres and computer networks thatare required to support the energy systems of the future. Sinceenergy harvesting will in general be intermittent, it will usefulto store energy dispatch it dynamically to optimise overallsystem behaviour [23], [24].

A. CONTRIBUTIONS OF THIS PAPERThus this paper addresses the use of the ‘‘Energy PacketNetwork’’ (EPN) paradigm [25]–[27] to model and optimisethe dynamic generation, storage and consumption, as well

VOLUME 4, 20162169-3536 2016 IEEE. Translations and content mining are permitted for academic research only.

Personal use is also permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

1321

www.whitepel.com +917620593389


E. Gelenbe, E. T. Ceran: EPNs With Energy Harvesting

as the dynamic workload, of highly dynamic and distributedenergy systems. Specifically, it addresses the use of the EPN,to model and optimise the generation, storage and consump-tion, as well as the dynamic workload, of highly dynamic anddistributed energy systems, and presents several new results:• Anovel case of practical interest is first developed, whenwe take a specific ‘‘high level’’ decision regarding thedistribution of energy. In this decision, all energy storesshould, on average, contain the same amount of energy,so that if any of them fails the others can best supportthe system, and the backlog of work at each of theworkstations should on average be the same.

• Then we introduce specific utility functions for thegeneral case, that take into account both the needs ofthe consumers, and the desire to maintain some reserveenergy for future needs.

• The control variables considered are the rates at whichwe can move energy packets from energy stores toworkstations, and also move energy between differentenergy stores. The model also allows us to representthe movement of jobs between workstations. The utilityfunctions contain contradictory terms, so that as we varythe control variables, some of these terms increase whileothers decrease, leading to clear optima.

• In order to optimise these utility functions, we developan algebraic formulation of the partial derivatives ofthe quantities of interest with respect to the controlvariables. This algebraic formulation is used to derivethe computational algorithms that obtain the optimumoperating points of the utility functions.

• A significant numerical example is also presented.It concerns a remote sensing/processing centre operatinga (radio) telescope or radar system, together with asensing and life support system, which constiute three‘‘workstations’’. Electrical energy is provided by a pho-tovoltaic source and by wind. Within the context ofthis system, the previously defined utility functions areconsidered and detailed optimisation results are illus-trated numerically in several tables and figures, regard-ing the optimum flow of energy to each of the threeworkstations.

The paper ends with conclusions including suggestion forfurther work.

II. ENERGY PACKET NETWORKSThe EPNmodel for networks of energy prosumers combines:• The intermittent flow of harvested of energy, repre-sented by a random flow of arriving ‘‘energy pack-ets (EPs)’’. An energy packet is a discrete unit of energy,e.g. X Joules, which is represented as arriving in ‘‘onechunk’’. It is simply a simplified version of the actualflow of energy which is continuous.

• The storage of energy (say in batteries), again in discreteunits of EPs. An energy storage unit (ES) such as a bat-tery is modelled as a queue of EPs that are waiting to beused. The ES is replenished by a flow of EPs from some

external source including an energy harvesting unit, andit can be depleted both when energy is forwarded to aconsumer, and through losses that represent leakage orline losses.

• The sources of power, the ESs and the consumers areinterconnected by Power Switches (PSs) which dynam-ically connects the sources of power to the ESs, and thesources of power and ESs to the consumers.

• The consumers will request for EPs from either a PS orESs, and these requests will typically be intermittent,since they are a function of the work that these con-sumers accomplish with the energy.

A typical example of such consumers are ICT sys-tems which intermittently receive computational work toaccomplish (for instance in terms of programs that needto be executed, or in terms of data packets that needto be transmitted), and which in turn require energy toaccomplish this work. Thus a consumer is also repre-sented by:• A queue of work that it has to accomplish which mayoriginate from some outside source or from some otherconsumer of energy which accomplished a prior workstep, and

• One or more servers that have job execution times whichhave a random duration which may depend on the natureof the job and the speed of the work unit, and also on theflow of power to the server.

An EPN will contain multiple work units with externalarrivals of jobs or data packets, the jobs or data packets maymove from one consumer queue to the other, and there will bemultiple queues of EPs, with external arrivals of renewable orsteadily flowing energy, and energy itself may flow betweenES, or it may leak from the energy stores, or move to con-sumers where it is used to execute jobs or transmit packets.Such EPNs can be analysed using a branch of queueingnetwork theory called G-Networks [28], [29]. Typically inqueueing theory, jobs move around a set of servers wherethey queue up to receive service until they complete theirneeds and leave the system. In G-Networks on the other hand,some of the flows of ‘‘customers’’, in this case the EPs, canqueue up (as in an ES), but they can move to act as servers oractivators for other customers, in this case the jobs that needto be executed or the data packets that need to be transmitted,that queue up at the servers where work is accomplished.Thus the G-Network can be used to represent job flowsthat consume energy and energy flows that allow jobs to beexecuted.

Electronic systems that capture a similar physicalbehaviour have been recently built and experimentallytested under the name of ‘‘power packet’’ systems.They are designed for the smart dispatching of electricalenergy [30], [31], and in some approaches the forwarding ofan EP takes the form of a pulse of current with a fixed voltageand duration, and each EP is equippedwith an encoded headercompromising the destination information for the EP [32] thatallows PSs to determine where the energy needs to be sent.

1322 VOLUME 4, 2016




In previous work [33] we have used the EPN model tostudy the architectures that interconnect energy prosumersystems so that energy losses and the response time to servicerequest are minimised.

In this paper we develop optimisation algorithms that opti-mise utility functions that include a linear combination ofthe throughput or useful, and the the probability that thesystem does not run out of energy. Thus the type of utilitythat we seek to maximise includes two sets of terms: the sumof the job processing rates of the system, and the sum ofthe probabilities that there is reserve energy in the system.Clearly if we try to just maximise the work done we may runout of energy, while if we try to just save energy then thesystem may delay the job execution times.

III. THE MATHEMATICAL MODELWe study an EPN consisting of a finite set of nodes. It con-tains m energy storage (ES) nodes denoted El , 1 ≤ l ≤ mstore and dispatch EPs, while n workstation (WS) nodes Wi,1 ≤ i ≤ n are ‘‘places of work’’ where jobs are executed.The system diagram is described schematically in Figure 1.Energy packets enter the energy store El at rate γl fromexternal renewable sources. Also, the energy stored in El maybe lost at a rate πl due to leakage and standby losses, and Eisends energy packets out of the store at rate rl either to otherenergy stores or to workstations.

The external arrival rate of jobs to the work stationWi is λi,while the workstation may also receive jobs from anotherworkstation Wj with probability Pji after workstation Wj fin-ishes processing that job. Moreover, uncompleted jobs storedat the work stationWi might also be lost at a time-out rate βi.However processing at Wi can only occur if an energy

packet arrives at the workstation from some ES El . This canhappen in two ways:• With probability s(l, i), El sends an energy packet toWi,and it may send EPs to Ek with probability S(l, k), with∑n

l=1 s(l, i)+∑m

k=1 S(l, k) = 1 for any l. We will writew(l, i) = rls(l, i) and W (l, k) = rlS(l, k) so that rl =∑n

l=1 w(l, i)+∑m

k=1W (l, k).• With probabilityPij the workstationWi sends the job thathas just completed to node Wj for more processing, or

with probability di = 1 −∑n

j=1 Pij the workstation Wiwill simply remove the job that terminates atWi from thesystem without forwarding it to another workstation.

In fact a more complex model in which a single job canconsume more than one energy packet at a time can also beused, and will be considered in future work.

As a consequence of the above assumptions, usingG-Network theory [28], [34] the probabilities Qi that Wi hasat least one job in its queue, and ql that El has at least one EPin its ES, will satisfy:

Qi =λi +

∑nj=1

∑ml=1 qlw(l, j)QjPji∑m

l=1 qlw(l, i)+ βi, (1)

where the rate of service at Wi is determined by the rate atwhich it receives energy, and ql is the probability that energy

FIGURE 1. Schematic representation of an EPN with two types of nodesand ‘‘queues’’: the ‘‘positive’’ nodes contain the work to be done whilethe ‘‘negative’’ nodes are the ESs which store energy and are replenishedby harvesting. ESs provide power in the form of EPs to the ‘‘positive’’nodes. Work can move from one positive node to another where they areprocessed when energy becomes available, and finally the work leavesthe network after completing a certain number of work steps. EPs canleak from the ESs (negative nodes), or be transferred to another ES orthey can be transferred to a positive node to accomplish work.

storage station El has at least one EP in store:

ql =γl +

∑mk=1 qkW (k, l)rl + πl

. (2)

The above equations also allow us to state the followingresult [34]:Theorem 1: Assume that the external arrivals of jobs,

and the arrivals of EPs from renewable energy sources, areindependent Poisson processes with rates λi, and γl . Supposethat the rl , πl and βi are the parameters of mutually inde-pendent exponentially distributed random variables. Thenif (K1, . . . ,Kn) ≥ (0, . . . , 0) and (k1, . . . , km) ≥(0, . . . , 0) represent the backlogs of jobs to be executed atthe workstations, and the number of energy packets stored atthe ESs, respectively. Then

limt→∞

P[K1(t) = K1, . . . ,Kn(t) = Kn;

k1(t) = k1, . . . , km(t) = km]

=

n∏i=1

QKii (1− Qi)m∏i=1

qkll (1− ql).

Corollary 1: Because in steady-state the joint probabilitydistribution of the number of jobs waiting, and energy packets

VOLUME 4, 2016 1323




stored, at each of the WS and ES, is the product of themarginal distributions, we have:

limt→∞

P[Ki(t) ≥ Ki] = QKii , limt→∞

P[kl(t) ≥ kl] = qkll .

A. VECTOR REPRESENTATIONDenote by Q and q, respectively, the row vectors whoseelements are (1) and (2). We can then write:

Q = 3+ QwqP; q = 0 + qW , (3)

where

• wq is the n×n diagonal matrix whose diagonal elementsare

∑ml=1 qlw(l, j),

• P is the n× n matrix

P = [Pij∑m

l=1 qlw(l, j)+ βj]n×n,

• W is the m× m matrix

W = [W (k, l)rl + πl

]m×m,

• and the3, 0 are, respectively, the n and m dimensionalvectors of elements

3i =λi∑m

l=1 qlw(l, i)+ βi, 0l =

γl

rl + πl,

The expression (3) can then be readily written as:

q = 0[1−W ]−1, Q = 3[1− wqP]−1. (4)

IV. AN INTERESTING SPECIAL CASEA case of practical interest arises when we take a specific‘‘high level’’ optimisation decision, namely:

• All energy stores should, on average, contain the sameamount of energy, so that if any of them fails the otherscan best support the system, so that:

q∗ ≡γl +

∑mk=1 q∗W (k, l)rl + πl

, ∀l,

• The backlog of work at each of the workstations shouldon average be the same, i.e.

Q∗ ≡λi +

∑nj=1

∑ml=1 q∗w(l, j)Q∗Pji∑m

l=1 q∗w(l, i)+ βi, ∀i.

Let us denote W+(l) =∑m

k=1W (l, k), and w+(i) =∑ml=1 w(l, i). Then we have:

q∗ =γl

rl + πl −W+(l), (5)

Q∗ =λi

βi + q∗[w+(i)−∑n

j=1 w+(j)Pji]

,

=λi

βi +γl

rl+πl−W+(l)[w+(i)−

∑nj=1 w

+(j)Pji], (6)

and we know that w+(i)−∑n

j=1 w+(j)Pji ≥ 0, and we recall

that Q∗, q∗ must satisfy 0 < Q∗ < 1 and 0 < q∗ < 1.If q∗ and Q∗ are fixed, we obtain:

w+(i)−n∑j=1

w+(j)Pji =1q∗

[λiβ1

λ1− βi]

+λi

q∗λ1[λ1

Q∗− β1] =

1q∗

[λi

Q∗− βi], (7)

so that we must have:

Q∗ <λi

βi, ∀ i, or Q∗ < min

1≤i≤n[λi

βi]. (8)

Then writing the m-vector c with elements:

ci =1q∗

[λi

Q∗− βi], (9)

we have:

w+ = w+P+ c, (10)

= c[1− P]−1,

which has a unique solution because the Markov chain P istransient. Since all the qk = q∗ are identical, we set

w(k, i) =w+(i)m

, (11)

so that all the w(k, i) are now determined.As a consequence, for each 1 ≤ k ≤ m we have:

n∑i=1

w(k, i) =

∑ni=1 w

+(i)m

≤ rk . (12)

Turning now to the set of weights W (k, l) we have:

W+(l) = rl + πl −γl

q∗, 1 ≤ l ≤ m, (13)

where:

q∗ ≥ min1≤l≤m

[γl

rl + πl], (14)

and for 1 ≤ k ≤ m we have:

W (l, k) =1m[rl + πl −

γl

q∗]. (15)

V. UTILITY FUNCTIONS AND OPTIMISATIONSimpler EPN optimisation problems, than the ones studiedhere were considered earlier in [35]. The optimisation wouldbe conducted on the assumption that:• The rl are variable, but they have an upper bound thatrepresents the maximum amount of power that can bedelivered by the energy stores.

• The Pij are fixed and represent the sequence of job stepsthat the jobs have to go through.

• The S(l, j) and s(l, k) are the ‘‘control variables’’: theyare modified or selected so as to maximise U .

• Cases can be considered where the S(l, k) are also fixed.Obviously, we wish to limit the backlog of work at the

workstations, while we also want to have some reserve of

1324 VOLUME 4, 2016




energy since there may be unpredictable needs. Thus a sensi-ble cost or utility function would have the form:

U1 =

n∑i=1

aiQKii +

m∑l=1

bl(1− qkll ), (16)

which is the weighted probability that the backlogs ofwork exceed the values Ki at each workstation i, and theweighted probabilities that there are not at least kl energypackets at ES l, where the ai and bl are non-negativereal numbers (the weights). A simpler utility that may beminimised is the weighted probabilities that there is somebacklog at the workstations, plus that there is no energyin reserve, which is U1 when Ki = 1 and kl = 1 forall i, l:

U01 =

n∑i=1

aiQi +m∑l=1

bl(1− ql). (17)

A cost function other than U1 defined above that we maywish to minimise is:

U2 =

n∑i=1

ai[∑m

l=1 qlw(l, i)+ βi]−1

1− Qi+

m∑l=1

bl(1− qkll ),

(18)

which differs from U1 only in the first sum which is simplytheweighted sum of the average response time of jobs waitingto be served at each of the workstations.

On the other hand, the following utility function U3 needsto be maximised since it establishes a balance between thethroughput of the system (the first term) and the probabilitythat some energy is kept in reserve:

U3 =

m∑l=1

[n∑i=1

aiqlw(l, i)Qi + blqkll ]. (19)

VI. COMPUTING PARTIAL DERIVATIVESOF THE UTILITIESThe optimisation of the such utilities, where the utilities arecontinuous and differentialble functions of the control vari-ables, will require the computation of the derivatives of theutility functions with respect to the control variables. In thiscase, the control variables are the routing probabilities s(l, k)and S(l, i) for the EPs. In fact because it is easier to work withnon-negative real numbers of arbitrary size rather than withprobabilities, we will consider that the control variables arethe quantities w(l, k) and W (l, i).Thus, for some real valued function V of the w(l, i) and

W (k, l), l, k ∈ {1, . . . ,m} and i ∈ {1, . . . , n}, let us usethe notation:

Vw(l,i)≡

∂V∂w(l, i)

, VW (k,l)≡

∂V∂W (k, l)

. (20)

A. THE CASE WHERE THE rl ARE CONSTANTIn a certain number of circumstances, we can consider thatthe rl are constant, for instance when each of the ESs has a

fixed and constant rate at which each of them can be emptiedof its EPs. Here we will first focus on this case.Remark 1: Since rw(a,b)l = rW (a,b)

l = 0 for any l, a, b, itis easy to see that:

w(l, j)z(k,i) = 1, if 1[z = w, l = k, i = j] = 1,

− 1, if 1[z = w, l = k, i 6= j] = 1,

− 1, if 1[z = W , l = k, i 6= j] = 1,

0, if 1[z = w or W , l¬k] = 1,

and

W (l, j)z(k,i) = 1, if 1[z = W , l = k, i = j],

− 1, if 1[z = W , l = k, i 6= j] = 1,

− 1, if 1[z = w, l = k] = 1,

0, if 1[z = w or W , l 6= k] = 1.

As a direct application of Remark 1, we construct thefollowing lemmas that will be used later in the optimisingalgorithms.Lemma 1: The derivative of ql with respect to any w(a, b)

for l, a ∈ {1, . . . ,m}, b ∈ {1, . . . , n} is given by:

qw(a,b)l =

∑mk=1 q

w(a,b)k W (k, l) − qarl + πl

,

or in vector form:

qw(a,b) = qw(a,b)W − qaδ = qaδ[1−W ]−1, (21)

where

• qw(a,b) is the m-vector each whose elements are thederivatives of the ql ,

• W is the m × m matrix of elements W (l, k)/(rk + πk )and

• δ is the m-vector δ = [ 1r1+π1

, . . . , 1rm+πm

].

Lemma 2: The derivative of ql with respect to anyW (a, b)for l, a, b ∈ {1, . . . ,m} is given by:

qW (a,b)l =

∑mk=1 q

W (a,b)k W (k, l) + qa(1[l = b]− 1[l 6= b])

rl + πl,

qW (a,b)= qa.[2δb − δ][1−W ]−1,

where δb = [0, . . . , 0, 1rb+πb

, 0, . . . , 0] is the m-vectorwith 0 elements everywhere except in the b-th position whichhas the value (rb + πb)−1.Lemma 3: The derivative of Qi with respect to any w(a, b)

for a, b ∈ {1, . . . ,m}, i, b ∈ {1, . . . , n} is given by:

Qw(a,b)i =

∑nj=1

∑ml=1[Q

w(a,b)j ql + Qjq

w(a,b)l ]Pjiw(l, j)∑m

l=1 qlw(l, i)+ βi

+qa[QbPbi −

∑nj 6=b,j=1 QjPji]∑m

l=1 qlw(l, i)+ βi

−Qi

∑ml=1 q

w(a,b)l w(l, i)+qa(1[b = i]− 1[b 6= i])∑m

l=1 qlw(l, i)+ βi.

VOLUME 4, 2016 1325




Lemma 4: The derivative of the Qi with respect to anyW (a, b) for a, b ∈ {1, . . . ,m} is given by:

QW (a,b)i =

∑nj=1

∑ml=1[Q

W (a,b)j ql + Qjq

W (a,b)l ]Pjiw(l, j)∑m

l=1 qlw(l, i)+ βi

−

∑nj=1 qaQjPji∑m

l=1 qlw(l, i)+ βi

−Qi

∑ml=1 q

W (a,b)l w(l, i)− qa∑m

l=1 qlw(l, i)+ βi.

B. THE CASE WHERE THE rl ARE NOT CONSTANTIn other circumstances the rl will not be constant, so thatwhen we modify any of the w(l, i) orW (l, k) we also changethe rl without affecting the other parameters.

This may occur for instance when the ESs are organised asa stack of batteries in parallel so that their output power flowcan be varied as a function of the number of storage units thatare switched to the power output bus.

However in practical circumstances there will be a maxi-muml value of rMl ≥ rl which cannot be exceeded because ofthe physical limitations of the devices that are used to storeenergy and of the power circuits that carry the power flows.Remark 2: Since rw(a,b)l = rW (a,b)

l = 0 for any l, a, b, itis easy to see that:

w(l, j)z(k,i) = 1, if 1[z = w, l = k, i = j],

− 1, if z = w, l = k, i 6= j; rl = rMl ,

− 1, if z = W , l = k, i 6= j; rl = rMl ,

0, if z = w, l = k, i 6= j; rl <Ml

0, if z = W , l = k, i 6= j; rl < rMl ,

0, if z = w or W , l¬k,

and

W (l, j)z(k,i) = 1, if z = W , l = k, i = j,

− 1, if z = W , l = k, i 6= j; rl = rMl ,

− 1, if z = w, l = k, rl = rMl ,

0, if z = W , l = k, i 6= j; rl < rMl ,

0, if z = w, l = k, rl =<Ml ,

0, if z = w or W , l 6= k.

As a consequence, when the rl < rMl , l = 1, . . . ,m wehave the following lemmas.Lemma 5: The derivative of ql with respect to any w(a, b)

for l, a ∈ {1, . . . ,m}, b ∈ {1, . . . , n} is given by:

qw(a,b)l =

∑mk=1 q

w(a,b)k W (k, l)

rl + πl

−ql

rl + πl,

or in vector form:

qw(a,b) = qw(a,b)W − q1l = −q1l[1−W ]−1, (22)

where as before W is the m × m matrix of elementsW (l, k)/(rk + πk ), and 1l is the m × m matrix which is

zero everywhere except in the diagonal terms which have thevalue 1

rl+πl.

Lemma 6: The derivative of ql with respect to anyW (a, b)for l, a, b ∈ {1, . . . ,m} is given by:

qW (a,b)l =

∑mk=1 q

W (a,b)k W (k, l)+ qa(1[l = b]− ql)

rl + πl,

qW (a,b)= −[qaδb − ql1l][1−W ]−1,

where δb = [0, . . . , 0, 1rb+πb

, 0, . . . , 0] is the m-vectorwith 0 elements everywhere except in the b-th position whichhas the value (rb + πb)−1.Lemma 7: The derivative of Qi with respect to any w(a, b)

for a, b ∈ {1, . . . ,m}, i, b ∈ {1, . . . , n} isgiven by:

Qw(a,b)i =

∑nj=1

∑ml=1[Q

w(a,b)j ql + Qjq

w(a,b)l ]Pjiw(l, j)∑m

l=1 qlw(l, i)+ βi

+qaQbPbi∑m

l=1 qlw(l, i)+ βi

−Qi

∑ml=1 q

w(a,b)l w(l, i)+ qa1[i = b]∑ml=1 qlw(l, i)+ βi

Lemma 8: The derivative of the Qi with respect to anyW (a, b) for a, b ∈ {1, . . . ,m} is given by:

QW (a,b)i =

∑nj=1

∑ml=1[Q

W (a,b)j ql + Qjq

W (a,b)l ]Pjiw(l, j)∑m

l=1 qlw(l, i)+ βi

−Qi

∑ml=1 q

W (a,b)l w(l, i)∑m

l=1 qlw(l, i)+ βi.

VII. OPTIMAL SOLUTION WITH GRADIENT DESCENTSince the optimisation of the EPN optimisation can beexpressed as the minimization or maximisation of utilityfunctions such as those defined in Section V, it can beachieved by selecting the appropriate EP flow rates w(a, b)and W (a, b) when other system parameters are fixed. Sincewe are delaing with continuous and differentiable utilityfunctions, the gradient descent algorithm is a useful tool inthis case.

At a given operating point of the EPN X =

[λ, γ, r,P, π, β], the gradient descent algorithm at its t th

computational step is:

wt+1(a, b) = wt (a, b)+ η Uw(a,b)|w(a,b)=wt (a,b), (23)

Wt+1(a, b) = Wt (a, b)+ η UW (a,b)|W (a,b)=Wt (a,b), (24)

where |η| is the rate of the gradient descent, we setη < 0 fo rthe minimisation of utility functionsU1,U0

1 ,U2, while η > 0 forthe maximisation of utilityfunction U3.

In practice, we will be interested in a gradual optimisationof the system, where we modify parameters progressively, ina system that should operate normally and adapt the ongoingwork work and energy flows.

1326 VOLUME 4, 2016




Thus we compute the partial derivative of the utility func-tions as:

Uw(a,b)1 =

n∑i=1

aiKiQKi−1i Qw(a,b)i −

m∑l=1

blkl(1−qkl−1l )qw(a,b)l ,

(25)

similarly,

U01w(a,b)

=

n∑i=1

aiQw(a,b)i −

m∑l=1

blqw(a,b)l , (26)

another cost function which we would like to minimise is:

Uw(a,b)2 =

n∑i=1

aig(i)[1

1− QiQw(a,b)i

−

∑ml=1(w(l, i)q

w(a,b)l + qlw(l, i)w(a,b))

[∑m

l=1 qlw(l, i)+ βi]]

−

m∑l=1

blkl(1− qkl−1l )qw(a,b)l , (27)

where

g(i) =[∑m

l=1 qlw(l, i)+ βi]−1

1− Qi. (28)

Moreover, the partial derivative of the utility functionwhich needs to be maximised is computed:

Uw(a,b)3 =

m∑l=1

[n∑i=1

ai[qw(a,b)l w(l, i)Qi + qlw(l, i)w(a,b)Qi

+ qlw(l, i)Qw(a,b)i ]+ blklq

kl−1l qw(a,b)l ]. (29)

The partial derivative with respect to the W (a, b) canbe computed similarly. The w(l, i)w(a,b), W (k, l)w(a,b) ,Qw(a,b), qw(a,b), w(l, i)W (a,b), W (k, l)W (a,b), QW (a,b), qW (a,b)

are detailed, for both constant and variable rl , in the previoussection. Thus, the steps of the gradient algorithm are:• First initialize all the values w0(a, b), W0(a, b) andchoose η,

• Solve the system of non-linear equations givenin (1) and (2) to obtain the steady state probabilities,

• Calculate the partial derivatives as given in Section VIusing the steady state probabilities computed in the pre-vious step.

• Using these partial derivatives and the chain rule, com-pute the value of the relevant utility function from (25),(26), (27) or (29), and

• Finally update the values wt+1(a, b),Wt+1(a, b) accord-ing to (24).

VIII. A SET OF NUMERICAL EXAMPLESTo illustrate the proposed approach, we have constructed anumerical example for a facility, such as a remote sensing andmonitoring facility such as an astronomical observatory on amountain peak, or a border control station in a remote partof the world, that is powered by energy harvesting devices

such as solar panels and wind turbines. The arrival rate to theworkstations is in units, each of which can be executed orcarried out with exactly one EP. Thus the relative workloadfor the different workstations is expressed by the arrival rateof work to each workstations. In the example wewill considerthere are three distinct workstations carrying out distinct workso that they can transfer work from oneworkstation to anotherand the W (a, b) = 0 for all a 6= b.The work that uses energy is carried out in the three

workstations:• W1 is meant to model a radio (or optical) telescope orradar station that is operating continuously and requiressubstantial processing, represented by a flow of workarriving at rate λ1. We assign to it a priority a1 ≥ 0.

• W2 represents the infrastructure energy needs suchas lighting, air-conditioning etc., that receives workrequests at rate λ2, with a2 > 0. Since this is a life-support system we would expect it to have high priority.

• W3 is mean to model the perimeter security and surveil-lance, and the monitoring of weather conditions, withworkload arriving at rate λ3. Since this represents thelife-support with a priority a3 > 0, which may becomparable to that of a2, while the priority a1 may belower.

Radars or telescopes provide a substantial amount of imag-ing data per unit time to be processed and transmitted, thus λ1can be considerably larger than λ2 and λ3. Moreover, secu-rity and atmospheric monitoring devices typically forwardlimited motion sensing, video camera, temperature and winddata, thus λ1 > λ2 > λ3. Likewise, the priorities of these dif-ferent work stations are chosento be a3 ≥ a2 > a1. To reflectdifferent forms of energy harvesting, such as photovoltaicand wind, we have two ESs, E1 and E2 which store energyfrom two different renewable sources at a rate γ1 and γ2EPs per unit time. The system model for this example issummarised in Figure 2. To obtain significant results and toremain within the stability region for the model, we mustoperate with parameters that represent a balance between theenergy flow and the flow of workload. Also, the impatienceof jobs, represented by the parameters βi, can help avoidinstability that occurs when the job queues increase in sizeand the waiting time of jobs at the execution queues becomevery large. Similarly, the energy leakage will also reduce theamount of energy that is stored when there is no work toperform. Note, however, that we will typically have πi � rMito represent the fact that leakage rates are only a small fractionof the maximum power (energy/time) that the ESs can offer.

A plausible scenario is constructed with a set of numericalparameters shown in Table 1 in order to represent a plausiblescenario. We first consider the case where the rl values arenot constant, and the case where they are constant.

A. WHEN r1, r2 ARE NOT CONSTANTConsider the case where the rl values are not constant, but amaximum practical value rMl is chosen as in Table 1. We setP(a, b) = 0 for a, b ∈ {1, 2, 3} so that jobs are not moved

VOLUME 4, 2016 1327




FIGURE 2. We use the EPN to model the energy and work flows in aremotely located sensing facility, such as an astronomical observatoryor radar station, with two energy storage units, two renewable energysources (e.g. one photovoltaic and one wind) and three main energyconsuming workstations: the major instrument (e.g. a radar), the lifesupport systems (e.g. air conditioning, lighting), and the atmosphericsensing and perimeter security sensors.

TABLE 1. Parameters used in the numerical examples.

between the different workstations. Also in the numericalexamples we set s(l, i) ≥ 0.05 , i = {1, . . . , 3}, l = {1, 2}and

∑3i=1 s(l, i) ≤ 1, l = {1, . . . , 2} to make sure that

each WS receives some minimum amount of energy to avoid‘‘starvation’’ that would lead to an infinite backlog of work atsome workstations, and

∑ni=1 w(l, i) ≤ r

Ml , l = {1, . . . ,m}.

The unconstrained optimisation problem is solved numer-ically for the four utilities U1,U0

1 ,U2,U3 of Section V, andthe resulting values for the optimum power flows are given inTables 2, 3, 4, 5 for different b1, b2 values.Table 5 shows that in order to maximise U3 and minimise

U1,U01 ,U2, the ESs should be providing energy at their

maximum total rate, i.e.∑

i w(l, i)∗= rMl , when b1, b2

TABLE 2. Optimised EP flow rates for the different utility functions whenb1, b2 = 100, the r1, r2 are not constant and the parameters are givenin Table 1.

TABLE 3. Optimised EP flow rates for the different utility functions whenb1, b2 = 10, the r1, r2 are not constant and the other numericalparameters are in Table 1.

TABLE 4. Optimised EP flow rates for the different utility functions whenb1, b2 = 1, the r1, r2 are not constant and the other numericalparameters are given in Table 1.

TABLE 5. Optimised EP flow rates for the different utility functions whenb1, b2 = 0.1, the r1, r2 are not constant and the other numericalparameters are given in Table 1.

are considerably smaller than a1, a2, a3. However when theb1, b2 are larger, we observe that the optima occur when thesum

∑i w(l, i)

∗ lies between 0.15 ∗ rMl and rMl depending

1328 VOLUME 4, 2016




TABLE 6. Optimised EP flow rates for the different utility functions whenb1, b2 = 100, r1, r2 are constant and the other numerical parameters aregiven in Table 1.



on the values of b1, b2. Note that the second terms in theutility functions concern the reserve energy contained inthe ESs.

B. WHEN r1, r2 ARE CONSTANTWhen the r1 and r2 are constant, energy leaves each of the ESsat a constant rate. However not to waste energy, we will allowsome of it to be sent to another ES rather than to the WS, ifthe optimisation requires it, i,e, the P(a, b), a, b ∈ {1, 2} arenow unconstrained and can be positive.

TABLE 9. Optimised EP flow rates for the different utility functions whenb1, b2 = 0.1, rl are constant and the other numerical parameters aregiven in Table 1.

This is illustrated in Tables 6, 7, 8, 9 where we see thatdepending on the requirements of optimisation, a fractionof the energy sent out by the ESs can be forwarded forstorage to the other ES. Note however than in this example,the energy loss during transfer, i.e. the ohmic loss, is nottaken into account. Clearly, if energy transfer losses becomesignificant then an optimised system will shy away frommaking additional energy transfers which do not lead to directconsumption.

IX. CONCLUSIONSIn this paper, we have discussed the main properties of theEnergy Packet Network model as a framework for a sys-temwhere intermittent distributed workloads and intermittentsources of energy operate jointly with multiple energy con-suming workstations, and with interconnected energy stor-age units, where both work and energy circulate. We obtainthe equilibrium probability distribution for both the backlogof work and the backlog of energy throughout the system.An interesting special case is also considered where the workand energy flows have been designed so that all energy storesand work backlogs are the same. Other stochastic models thatare more adapted to the study of wireless sensors with smallscale energy harvesting [6], [16], [17], [36] are consideredelsewhere [37].

Here we have studied several utility functions that describethe overall system performance, and an algorithmic methodbased on gradient descent (or ascent) of these utility functionsis developed to seek energy and work distribution mecha-nisms that optimise the utility functions. Also, the gradientdescent approach we have described requires matrix inver-sions to compute Qw(a,b) and qw(a,b) which are of compu-tational complexity O(n3) and O(m3), respectively. Simplernumerical schemes, such as power expansions of the matri-ces, can result in useful approximations, especially when theenergy and work flow rates vary frequently with time, or areimprecisely known and will be studied in future work.

Another direction that will be considered includes morecomplex models in which a single job can consume morethan one energy packet at a time, or where an energy packet

VOLUME 4, 2016 1329




may service several jobs at a time. The approach studiedin this paper can also be extended to multiple classes ofEPs, with power colouring which can be used to representthe origin of the EPs, for instance different types of har-vested energy from wind or photovoltaic, as well as fromnon-renewable or fossil sources, or with different economiccost.

Multiple classes of workloads depending on their energyneeds, or their priorities, or their importance with regard tothe income they produce, can also be considered. We willalso examine ‘‘good’’ but suboptimal solutions which canbe obtained at a lower computational cost. Finally, we willalso study the dynamic behaviour of such systems when theflow of work and of energy is regulated in real-time, usingtechniques such as reinforcement learning.

ACKNOWLEDGEMENTThe first author also thanks the Académie des Technologies,the National Academy of Technologies of France, for encour-aging his initiative to examine the links between Energyand ICT.

REFERENCES[1] J. Gubbi, R. Buyya, S. Marusic, and M. Palaniswami, ‘‘Internet of

Things (IoT): A vision, architectural elements, and future directions,’’Future Generat. Comput. Syst., vol. 29, no. 7, pp. 1645–1660, 2013.

[2] A. R. Al-Ali and R. Aburukba, ‘‘Role of Internet of Things in the smart gridtechnology,’’ J. Comput. Commun., vol. 3, no. 5, pp. 229–233, May 2015.

[3] E. Gelenbe and F.-J. Wu, ‘‘Large scale simulation for human evacua-tion and rescue,’’ Comput. Math. Appl., vol. 64, no. 12, pp. 3869–3880,Dec. 2012.

[4] T. J. McCue. (Apr. 2015). $117 Billion Market for Internet of Things inHealthcare by 2020, Forbes Tech. [Online]. Available: http://www.forbes.com/sites/tjmccue/2015/04/22/117-billion-market-for-internet-of-things-in-healthcare-by-2020/2715e4857a0b2536043d2471

[5] (Dec. 2015). Environmental Leader. [Online]. Available: http://www.environmentalleader.com/tag/internet-of-things

[6] E. Gelenbe, D. Gesbert, and D. Gündüz, H. Külah, and E. Uysal-Biyikog̃lu,‘‘Energy harvesting communication networks: Optimization and demon-stration (the E-CROPS project),’’ in Proc. 24th Tyrrhenian Int. WorkshopDigit. Commun.–Green ICT (TIWDC), Genoa, Italy, Sep. 2013, pp. 1–6.

[7] F. Akhtar and M. H. Rehmani, ‘‘Energy replenishment using renewableand traditional energy resources for sustainable wireless sensor networks:A review,’’ Renew. Sustain. Energy Rev., vol. 45, pp. 769–784, May 2015.

[8] A. A. Khan, M. H. Rehmani, and M. Reisslein, ‘‘Cognitive radio forsmart grids: Survey of architectures, spectrum sensing mechanisms, andnetworking protocols,’’ IEEE Commun. Surveys Tuts., vol. 18, no. 1,pp. 860–898, Oct. 2016.

[9] R. C. Qiu et al., ‘‘Cognitive radio network for the smart grid: Experimentalsystem architecture, control algorithms, security, and microgrid testbed,’’IEEE Trans. Smart Grid, vol. 2, no. 4, pp. 724–740, Dec. 2011.

[10] Y. Zhang, R. Yu, M. Nekovee, Y. Liu, S. Xie, and S. Gjessing, ‘‘Cognitivemachine-to-machine communications: Visions and potentials for the smartgrid,’’ IEEE Netw., vol. 26, no. 3, pp. 6–13, May/Jun. 2012.

[11] E. Gelenbe and K. Sevcik, ‘‘Analysis of update synchronization for multi-ple copy data bases,’’ IEEE Trans. Comput., vol. C-28, no. 10, pp. 737–747,Oct. 1979.

[12] F. Baccelli, E. Gelenbe, and B. Plateau, ‘‘An end-to-end approach to theresequencing problem,’’ J. ACM, vol. 31, no. 3, pp. 474–485, Jul. 1984.

[13] E. Gelenbe and Y. Caseau, ‘‘The impact of information technologyon energy consumption and carbon emissions,’’ Ubiquity, vol. 2015,pp. 1:1–1:15, Jun. 2015. [Online]. Available: http://doi.acm.org/10.1145/2755977

[14] A. Berl et al., ‘‘Energy-efficient cloud computing,’’ Comput. J., vol. 53,no. 7, pp. 1045–1051, 2010.

[15] B. Lannoo et al., ‘‘Overview of ICT energy consumption (deliver-able 8.1),’’ Eur. Netw. Excellence Internet Sci., Thessaloniki, Greece,Tech. Rep. EU FP7 Project FP7-2888021, 2013.

[16] J. Yang and S. Ulukus, ‘‘Optimal packet scheduling in an energy har-vesting communication system,’’ IEEE Trans. Commun., vol. 60, no. 1,pp. 220–230, Jan. 2012.

[17] A. Rahimi, Ö. Zorlu, A. Muhtaroglu, and H. Külah, ‘‘Fully self-poweredelectromagnetic energy harvesting system with highly efficient dual railoutput,’’ IEEE Sensors J., vol. 12, no. 6, pp. 2287–2298, Jun. 2012.

[18] E. Gelenbe, ‘‘A sensor node with energy harvesting,’’ SIGMETRICSPerform. Eval. Rev., vol. 42, no. 2, pp. 37–39, Sep. 2014.

[19] E. Gelenbe and Y. M. Kadioglu, ‘‘Energy loss through standbyand leakage in energy harvesting wireless sensors,’’ in Proc. 20thIEEE Int. Workshop Comput. Aided Modelling Design Commun. LinksNetw. (CAMAD), Guildford, U.K., Sep. 2015, pp. 231–236. [Online].Available: http://dx.doi.org/10.1109/CAMAD.2015.7390515

[20] I. Anghel et al., ‘‘GEYSER: Enabling green data centres in smart cities,’’in Energy Efficient Data Centers (Lecture Notes in Computer Science),vol. 8945, S. Klingert, M. Chinnici, and M. R. Porto, Eds. Springer, 2015,pp. 71–86.

[21] M. M. He et al., ‘‘An architecture for local energy generation, distribution,and sharing,’’ in Proc. IEEE Energy Conf. (ENERGY), Nov. 2008, pp. 1–6.

[22] B. Gurakan, O. Ozel, J. Yang, and S. Ulukus, ‘‘Energy cooperationin energy harvesting communications,’’ IEEE Trans. Commun., vol. 61,no. 12, pp. 4884–4898, Dec. 2013.

[23] K. M. Chandy, S. H. Low, U. Topcu, and H. Xu, ‘‘A simple optimal powerflow model with energy storage,’’ in Proc. 49th IEEE Conf. DecisionControl (CDC), Dec. 2010, pp. 1051–1057.

[24] H.-I. Su and A. El Gamal, ‘‘Modeling and analysis of the role of fast-response energy storage in the smart grid,’’ in Proc. 49th Annu. AllertonConf. Commun., Control, Comput. (Allerton), Sep. 2011, pp. 719–726.

[25] E. Gelenbe, ‘‘Energy packet networks: ICT based energy allocation andstorage (invited paper),’’ in Proc. GreeNets, 2011, pp. 186–195.

[26] E. Gelenbe, ‘‘Energy packet networks: Adaptive energy manage-ment for the cloud,’’ in Proc. 2nd Int. Workshop Cloud Comput.Platforms (CloudCP), 2012, Art. no. 1.

[27] E. Gelenbe, ‘‘Synchronising energy harvesting and data packets in a wire-less sensor,’’ Energies, vol. 8, no. 1, pp. 356–369, Jan. 2015. [Online].Available: http://www.mdpi.com/journal/energies

[28] E. Gelenbe and A. Labed, ‘‘G-networks with multiple classes of sig-nals and positive customers,’’ Eur. J. Oper. Res., vol. 108, no. 2,pp. 293–305, Jul. 1998. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0377221797003718

[29] E. Gelenbe, ‘‘The first decade of G-networks,’’ Eur. J. Oper.Res., vol. 126, no. 2, pp. 231–232, 2000. [Online]. Available:http://dx.doi.org/10.1016/S0377-2217(99)00475-0

[30] R. Takahashi, T. Takuno, and T. Hikihara, ‘‘Estimation of power packettransfer properties on indoor power line channel,’’ Energies, vol. 5, no. 7,pp. 2141–2149, 2012.

[31] R. Takahashi, S.-I. Azuma, K. Tashiro, and T. Hikihara, ‘‘Design and exper-imental verification of power packet generation system for power packetdispatching system,’’ in Proc. Amer. Control Conf. (ACC), Jun. 2013,pp. 4368–4373.

[32] B. Kreiner and J. Reeves, ‘‘Packetized power,’’U.S. Patent 2014 0 013 146 A1, Jan. 9, 2014. [Online]. Available:https://www.google.com/patents/US20140013146

[33] E. Gelenbe and E. T. Ceran, ‘‘Central or distributed energy storagefor processors with energy harvesting,’’ in Proc. Sustain. Internet ICTSustainability (SustainIT), Madrid, Spain, Apr. 2015, pp. 1–3.

[34] E. Gelenbe, ‘‘G-networks with triggered customer movement,’’ J. Appl.Probab., vol. 30, no. 3, pp. 742–748, Sep. 1993. [Online]. Available:http://www.jstor.org/stable/3214781

[35] E. Gelenbe and C. Morfopoulou, ‘‘A framework for energy-aware routingin packet networks,’’ Comput. J., vol. 54, no. 6, pp. 850–859, Jun. 2011.

[36] O. Orhan, D. Gündüz, and E. Erkip, ‘‘Throughput maximization for anenergy harvesting communication system with processing cost,’’ in Proc.IEEE Inf. Theory Workshop (ITW), Lausanne, Switzerland, Sep. 2012,pp. 84–88.

[37] E. Gelenbe and A. Marin, ‘‘Interconnected wireless sensors with energyharvesting,’’ in Analytical and Stochastic Modelling Techniques and Appli-cations (Lecture Notes in Computer Science), vol. 9081, M. Gribaudo,D. Manini, and A. Remke, Eds. Springer, 2015, pp. 87–99.

1330 VOLUME 4, 2016




EROL GELENBE (F’86) received his PhD fromthe Polytechnic Institute of NYU and the Doctorof Science degree from University of Paris VI.He was awarded Honoris Causa Doctorates fromthe University of Liége (Belgium), the Univer-sitá di Roma II (Italy), and Bogaziçi University(Istanbul). He is a Professor and the Head of Intel-ligent Systems and Networks in the Departmentof Electrical and Electronic Engineering at Impe-rial College London, working on energy savings,

system security, and network quality of service. He has founded the Sys-tem Performance Evaluation Groups at INRIA and in French universities,chaired the ECE Department, Duke University, and created the School ofElectrical Engineering and Computer Science at the University of CentralFlorida. His engineering impact has included founding the team that designedthe commercial queueing network analysis package QNAP at INRIA, anddeveloping the flexible manufacturing system simulator FLEXSIM, theSYCOMORE multiprocessor switch, the XANTHOS fiber optics local areanetwork, and the cognitive packet network routing algorithm CPN. In hismore theoretically oriented work, he invented mathematical models, suchas G-Networks for computer and network performance, and the randomneural network to mimic spiked neuronal behavior and learning in bio-logical systems. He was elected a Fellow of ACM in 2001. He receivedthe In Memoriam Dennis Gabor Award of the Hungarian Academy ofSciences in 2013, the Oliver Lodge Medal of the IET (London) in 2010, theACM-SIGMETRICS Lifetime Achievement Award in 2008, and the GrandPrix France Telecom in 1996. He is a Fellow of the FrenchNational Academyof Technologies, the Hungarian, Polish and Turkish Science Academies, andthe Royal Academy of Sciences, Arts and Letters of Belgium. He is a Knightof the Légion d’Honneur (France), a Commander of Merit and Grand Officerof the Order of the Star of Italy.

ELIF TUG̃Çe CERAN received the B.S. andM.S. degrees in electrical and electronics engi-neering from Middle East Technical University,Ankara, Turkey, in 2012 and 2014, respectively.She is currently pursuing the Ph.D. degree withthe Intelligent Systems and Networks ResearchGroup, Department of Electrical and ElectronicEngineering, Imperial College London. She isa Research Assistant with the Intelligent Sys-tems and Networks Research Group, Department

of Electrical and Electronic Engineering, Imperial College London. Herresearch interests include energy harvesting wireless networks, energypacket networks, and the modeling and performance evaluation of computerand communication systems powered by renewable energy sources. In 2015,she co-authored two conference papers in these areas.

VOLUME 4, 2016 1331



energy packet networks with energy harvesting

Education