energy cost optimization in water distribution systems using markov decision processes
DESCRIPTION
University of Sao Paulo Department of Electrical and Computer Engineering Intelligent Techniques Laboratory. Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes Paulo T. Fracasso , Frank S. Barnes and Anna H. R. Costa. Agenda. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/1.jpg)
Energy Cost Optimization in Water Distribution Systems Using Markov Decision ProcessesPaulo T. Fracasso, Frank S. Barnes and Anna H. R. Costa
Energy Cost Optimization in Water Distribution Systems Using Markov Decision ProcessesPaulo T. Fracasso, Frank S. Barnes and Anna H. R. Costa
University of Sao PauloDepartment of Electrical and Computer Engineering
Intelligent Techniques Laboratory
University of Sao PauloDepartment of Electrical and Computer Engineering
Intelligent Techniques Laboratory
![Page 2: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/2.jpg)
AgendaAgenda
• Anatomy of Water Distribution Systems
• Problem relevancy
• Markov Decision Process
• Modeling a Water Distribution System as an MDP
• Monroe Water Distribution System
• Experiment results
• Conclusions
2
![Page 3: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/3.jpg)
Water distribution system Water distribution system • It is a complex system composed by pipes, pumps and other
hydraulic components which provide water supply to consumers.
Focus of my work
3
![Page 4: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/4.jpg)
Problem relevancyProblem relevancy• About 3% of US energy consumption (56 billion kWh) are used
for drinking water (Goldstein and Smith, 2002).
$2 billion/year
Source: Electric Power Research Institute,1994.4
![Page 5: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/5.jpg)
MDP is a model for sequential decision making in fully observable environments when outcomes are uncertain.
Advantages of MDP compared to other techniques: Real world – operates in uncertain and dynamic domains Planning – generates control policies to sequential decisions Optimal solution – guarantees to achieve a higher future payoff
Disadvantages of MDP: Discrete domains (state and action) Course of dimensionality
Markov Decision Process - MDPMarkov Decision Process - MDP
5
![Page 6: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/6.jpg)
MDP is defined as a tuple where: S is a discrete set of states (can be factored in Nv features):
A is a discrete set of actions:
T is a transition function where
R is a reward function where
Markov Decision Process - MDPMarkov Decision Process - MDPRTAS ,,,
ttt assPT ,|'',, 1
S
AA
6
),...,(),...,,...,(,..., 11
111
V
SS
V
S
NNN
NNS
ttt asrR ,|,
ASR :
1,0: SAST
![Page 7: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/7.jpg)
Solving an MDP consists in finding a policy , which is defined as a mapping from states to actions, s.t.
Bellamn’s equation allows to break a dynamic optimization problem into simpler sub-problems:
The optimal value of the utility is:
The optimal policy are the actions obtained from :
Markov Decision Process - MDPMarkov Decision Process - MDP
S
AVTR
'
** '',,,maxarg
S
VTRV'
'',,,
AS :
S
AVTRV
'
** '',,,max
*V
![Page 8: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/8.jpg)
Water Distribution System modeled as an MDPWater Distribution System modeled as an MDP
8
HN
HHTS ,...,, 1
maxmin ,TTT
maxminmin ,...,, TTTTT
maxmin ,HHH
maxminmin ,...,, HHHHH
![Page 9: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/9.jpg)
Water Distribution System modeled as an MDPWater Distribution System modeled as an MDP
9
UN
UUA ,...,1
1,0U
)(),(),(1 tAtDtHftH
1,...,,0 UU
1,0U
DC CCR
BC FP
FP
OP
OPC PTtPwPTtPwC )()(
DMBC
D PtPwxmaC )(
![Page 10: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/10.jpg)
MarkovDecision
Processes
Constraints
Control policy
Dem
and
Ele
ctric
al p
ower
Final result:
Energy priceschema
10
Water Distribution System modeled as an MDPWater Distribution System modeled as an MDP
![Page 11: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/11.jpg)
Understand MDP resultsUnderstand MDP results
Control policy: Maps state variables into a set
of actions
States variables: everything that is important to control (tank level and time)
Set of actions: what you can manipulate (pumps)
Indicates controllability (avoid black region)
Correlated to demand curve
Tan
k le
vel
Time
11
![Page 12: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/12.jpg)
Understand MDP resultsUnderstand MDP results
Controller: Uses control policy map to
produce actions
Actions are based just on tank level and time
Easy to implement and fast to run in PLC (lookup table)
Tan
k le
vel
Time
Pum
p tr
igge
r
12
![Page 13: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/13.jpg)
Monroe Water Distribution System
Characteristics: 11 pumps
1 storage tank
4 pressure monitoring
40k people served
182 miles of pipes
Diameters varyingfrom 2 to 42 inches
13
![Page 14: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/14.jpg)
Monroe Water Distribution System
14
Demand curve (during summer season):
Average: 6 700 GPM
Minimum: 4 188 GPM
Maximum: 8 389 GPM
Pressure restrictions (in PSI): J-6: 65 ≤ P ≤ 70 ▪ J-131: 45 ≤ P ≤ 55 J-36: 50 ≤ P ≤ 60 ▪ J-388a: 40 ≤ P ≤ 90
![Page 15: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/15.jpg)
Monroe Water Distribution System
15
Pumps (E2, E3, E4, E5, E6, E7, W8, W9, W10, W11 and W12):
Energy price schema: On-peak (09:00 – 20:59): $0.04014/kWh Off-peak (21:00 – 08:59): $0.03714/kWh Demand (monthly): $13.75/kW
![Page 16: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/16.jpg)
MDP apply to Monroe WDSMDP apply to Monroe WDS
Mathematical model:
Set of states: where and
Set of actions:
Transition function:
Reward function:
Data flux diagram:
HTS ,
)(),(),()1( tAtdtHftH
demandt
peakofft
peakon tPwtPwtPwTtR $)(max$)($)(30)(00:9
00:21
00:21
00:9
EPANETDLL.INP FILE MATLAB
)(
)1(
tPw
th
16
24,0T 25.33,1H
111,...,UUA
![Page 17: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/17.jpg)
MDP results in Monroe WDSMDP results in Monroe WDS
Expected electrical power :
17
E5 and E7 consume 144.3kW W11, E2 and E6 consume 320.4kW
![Page 18: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/18.jpg)
MDP results in Monroe WDSMDP results in Monroe WDS
Number of activated pumps (27 possibilities):
18
on={E2,E6} on{E5,E7}
on={W12,E3,E4,E5} on={E2,E3,E4,E5}
![Page 19: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/19.jpg)
MDP results in Monroe WDSMDP results in Monroe WDS
SCADA records: obtained from historical data (July 6th, 2010) 75% of WTP consumption is considered to be used in pump
One day is extrapolated to one billing cycle (30 days) Both approaches started in the same level (19.3 ft)
Energy expenses SCADA records MDP Difference Off-peak energy [$/month] 3 210.57 2 608.32 -23.1% On-peak energy [$/month] 3 750.78 3 768.51 +0.5% Demand [$/month] 3 836.25 3 603.67 -6.5% Total [$/month] 10 797.60 9 980.50 -8.2%
19
![Page 20: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/20.jpg)
ConclusionsConclusions
MDP avoids restrictions (level, pressure, and pumps) and reduces expenses with energy
To reduce energy consumption is different to reduce expenses with energy (demand is the biggest villain)
Summer season imposes small quantity of feasible actions Verify if it is possible to reduce the number of pump combination MDP policy is easy to implement in a non-intelligent device (PLC)
20
![Page 21: Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes](https://reader034.vdocument.in/reader034/viewer/2022051517/5681591e550346895dc6478f/html5/thumbnails/21.jpg)
ContactContact
Thank you for your attention
PAULO THIAGO FRACASSO [email protected]
Av. Prof. Luciano Gualberto, trav.3, n.158, sala C2-50CEP: 05508-970 - São Paulo, SP - Brazil
Phone: +55-11-3091-5397
21