renewable energy & electricity markets
DESCRIPTION
Renewable energy & Electricity markets. Be careful what you wish for. Adam Wierman , Caltech. Joint work with Sachin Adlakha , Subhonmesh Bose, Desmond Cai , John Ledyard, Steven Low, and Jayakrishnan Nair. here!. Renewable energy is coming!. MW. Wind:. Worldwide. MW. China. - PowerPoint PPT PresentationTRANSCRIPT
Renewable energy & Electricity markets
Adam Wierman, CaltechJoint work with Sachin Adlakha, Subhonmesh Bose, Desmond Cai, John Ledyard, Steven Low, and Jayakrishnan Nair.
Be careful what you wish for
MW
WorldwideWind:
MW
Europe
AmericasChina
Solar PV:
Renewable energy is coming!
here!
Renewable energy is coming!…but incorporation into the grid isn’t easy
They are typically Uncontrollable (not available “on demand”) Intermittent (large fluctuations) Uncertain (difficult to forecast)
Each line is wind generation over 1 day
here!
Key Constraint: Generation = Load(at all times)
low uncertainty
Today’s grid
GenerationLoad
Key Constraint: Generation = Load(at all times)
low uncertaintycontrollable(via markets)
Today’s grid
GenerationLoad
Key Constraint: Generation = Load
less controllablehigh uncertainty
(at all times)
Tomorrow’s grid
low uncertainty
Key Constraint: Generation = Load
less controllablehigh uncertainty
low uncertainty
(at all times)
1) Huge price variability, leading to generators opting out of markets!2) More conventional reserves needed, countering sustainability gains!
“ON JUNE 16th something very peculiar
happened in Germany’s electricity market. The
wholesale price of electricity fell to minus €100
per megawatt hour (MWh). That is, generating
companies were having to pay the managers of
the grid to take their electricity.”
“Energiewende has so far
increased, not decreased,
emissions of greenhouse
gases.”
What can be done?Reduce the uncertainty
Design for the uncertainty
•Better prediction• “Aggregation” … in time (storage) … in space (distributed generation) … in generation (heterogeneous mix)
•Redesign electricity markets• Increase amount of demand response
our focus at Caltech
This talk: Two electricity market design challenges1) How many markets should there be? and when should they occur?2) The nasty economic consequences of Kirchhoff's laws
The newsvendor problemNetworked Cournot competitionstochastic
networks
Forget about energy for a second…This section is really about the role of uncertainty in newsvendor problems
Forget about energy for a second…This section is really about the role of uncertainty in newsvendor problems
Estimate demand,
Purchase,
Demand is realized
lost revenue wasted inventory
uncertainty
“You have to decide today how many newspapers you want to sell tomorrow…”
Forget about energy for a second…
“You have to decide today how many newspapers you want to sell tomorrow…”seasonal productsperishable goods
compute instancesenergy
…
This section is really about the role of uncertainty in newsvendor problems
timeint. /day
ahead
realtime
longterm
Utility buys power to
meet demand
Electricity marketsmarkets
timeint. /day
ahead
realtime
longterm
markets
PIRP
timeint. /day
ahead
realtime
longterm
markets
What is the impact of long term wind contracts?As renewable penetration increases: 1)Should markets be moved closer to real-
time? 2)Should markets be added?
4 hr market
How should utilities procure electricity in the presence of renewable energy?First step:
As renewable penetration increases: 1)Should markets be moved closer to real-
time? 2)Should markets be added?
What is the impact of long term wind contracts?
int. /day
ahead
realtime
longterm
price↑
𝑝𝑙𝑡 𝑝𝑖𝑛 𝑝𝑟𝑡
int. /day
ahead
realtime
longterm
price volatility↑
𝑝𝑙𝑡 𝑝𝑖𝑛 𝑝𝑟𝑡
𝑞𝑙𝑡 𝑞𝑖𝑛 𝑞𝑟𝑡
𝐸 [𝑝𝑖𝑛 ]>𝑝𝑙𝑡 𝐸 [𝑝𝑟𝑡|𝑝𝑖𝑛 ]>𝑝𝑖𝑛
int. /day
ahead
realtime
longterm
price↑
𝑝𝑙𝑡 𝑝𝑖𝑛 𝑝𝑟𝑡
𝑞𝑙𝑡 𝑞𝑖𝑛 𝑞𝑟𝑡
�̂�𝑙𝑡 �̂�𝑖𝑛 𝑤
𝜀1=�̂�𝑙𝑡− �̂�𝑖𝑛 𝜀2=�̂�𝑖𝑛−𝑤
Assumption: and are independent(A generalization of the martingale model of forecast evolution)
wind uncertainty ↓
𝑞𝑙𝑡+𝑞𝑖𝑛+𝑞𝑟𝑡+𝑤≥𝑑Key Constraint: Generation = Load
int. /day
ahead
realtime
longterm
price↑wind uncertainty ↓
𝑝𝑙𝑡 𝑝𝑖𝑛 𝑝𝑟𝑡
𝑞𝑙𝑡 𝑞𝑖𝑛 𝑞𝑟𝑡
�̂�𝑙𝑡 �̂�𝑖𝑛 𝑤
(we ignore network constraints for now)
int. /day
ahead
realtime
longterm
price↑wind uncertainty ↓
𝑝𝑙𝑡 𝑝𝑖𝑛 𝑝𝑟𝑡
𝑞𝑙𝑡 𝑞𝑖𝑛 𝑞𝑟𝑡
�̂�𝑙𝑡 �̂�𝑖𝑛 𝑤
Utility goal:min𝐸 [𝑝𝑙𝑡 𝑞𝑙𝑡+𝑝𝑖𝑛𝑞𝑖𝑛+𝑝𝑟𝑡𝑞𝑟𝑡 ]Subject to causality constraints
int. /day
ahead
realtime
longterm
𝑝𝑙𝑡 𝑝𝑖𝑛 𝑝𝑟𝑡
𝑞𝑙𝑡 𝑞𝑖𝑛 𝑞𝑟𝑡
�̂�𝑙𝑡 �̂�𝑖𝑛 𝑤
Utility goal:min𝐸 [𝑝𝑙𝑡 𝑞𝑙𝑡+𝑝𝑖𝑛𝑞𝑖𝑛+𝑝𝑟𝑡𝑞𝑟𝑡 ]Subject to causality constraintsVariant of the newsvendor problem
[Arrow et. al. ’51], [Silver et. al. ’98], [Khouja ’99], [Porteus ’02], [Wang et. al. ’12].
Theorem:The optimal procurement strategy is characterized by reserve levels and such that
where
and uniquely solves
int. /day
ahead
realtime
longterm �̂�𝑙𝑡 �̂�𝑖𝑛 𝑤
𝜀1=�̂�𝑙𝑡− �̂�𝑖𝑛 𝜀2=�̂�𝑖𝑛−𝑤
baseline, e.g., average output of a wind farm scale, e.g., number of wind farms
Scaling regime
aggregation, e.g., degree of correlation between wind farms
𝒘 𝒍𝒕 (𝜸 )=𝜸𝜶 𝜺𝟐 (𝜸 )=𝜸𝜽𝜺𝟐𝜺𝟏 (𝜸 )=𝜸𝜽𝜺𝟏
baseline, e.g., average output of a wind farm scale, e.g., number of wind farms
Scaling regime
aggregation, e.g., degree of correlation between wind farms
Theorem:
Procurement with zero uncertainty
Extra procurementdue to uncertainty
baseline, e.g., average output of a wind farm scale, e.g., number of wind farms
Scaling regime
aggregation, e.g., degree of correlation between wind farms
Theorem:
Depends on markets & predictions - prices - forecasts
Depends on wind aggregation - =1/2 (independent) - =1 (correlated)
baseline, e.g., average output of a wind farm scale, e.g., number of wind farms
Scaling regime
aggregation, e.g., degree of correlation between wind farms
Theorem:
This form holds more generally than the model studied here:
-- more than three markets: [Bitar et al., 2012]-- when prices are endogenous: [Cai & Wierman, 2014]-- when small-scale storage is included: [Hayden, Nair, & Wierman, Working paper]
timeint. /day
ahead
realtime
longterm
markets
Electricity markets
As renewable penetration increases: 1)Should markets be moved closer to real-
time? 2)Should markets be added?
No!
What is the impact of long term wind contracts?
timeint. /day
ahead
realtime
longterm
markets
Electricity markets
As renewable penetration increases: 1)Should markets be moved closer to real-
time? 2)Should markets be added?
4 hr ahead marke
t?What is the impact of long term wind contracts?
realtime
longterm v/s int. real
timelongterm
What happens to if a market is added?
What happens to if a market is added?
6 6.5 7 7.5 8 8.5 9 9.5 10
int. /day
ahead
realtime
longterm
𝜀2 Gaussian
𝑝𝑙𝑡=6 6<𝑝𝑖𝑛<10 𝑝𝑟𝑡=10
𝑝𝑖𝑛
]
2 markets
3 markets
3 markets are always better!
When does this happen?
Theorem:If is increasing for , decreasing for , and satisfies:
is decreasing for is decreasing for
then the expected procurement is lower with 3 markets than with 2 markets.
Satisfied by the Gaussian distribution
int. /day
ahead
realtime
longterm
𝜀2 Weibull
𝑝𝑙𝑡=6 6<𝑝𝑖𝑛<10 𝑝𝑟𝑡=10
6 6.5 7 7.5 8 8.5 9 9.5 10𝑝𝑖𝑛
]
2 markets
3 markets
3 markets can be worse!
When does this happen?
Theorem:If satisfies the condition:
=0 , then there exist prices such that the expected procurement is higher with 3 markets than with 2 markets.
Estimation errors are heavy-tailed(specifically, long-tailed)
timeint. /day
ahead
realtime
longterm
markets
As renewable penetration increases: 1)Should markets be moved closer to real-
time? 2)Should markets be added?
No! It depends, Gaussian or heavy-tailed?
4 hr market
What is the impact of long term wind contracts?
timeint. /day
ahead
realtime
longterm
marketsmarkets
PIRP
How should wind be incorporated into the markets?
What is the impact of long term wind contracts?
This talk: Two electricity market design challenges1) How many markets should there be? and when should they occur?2) The nasty economic consequences of Kirchhoff's laws
The newsvendor problemNetworked Cournot competition
Forget about energy for a second…This section is really about intermediaries & competition in networked markets
Forget about energy for a second…This section is really about intermediaries & competition in networked markets
Rarely is competition in a single, well defined market… firms typically compete across a variety of markets
Firms Markets
Gas pipelines in the US
Rarely is competition in a single, well defined market… firms typically compete across a variety of markets
Examples: gas, airlines, construction, … , energy
Forget about energy for a second…This section is really about intermediaries & competition in networked markets
Key Constraint: Generation = Load(at all times)
G
G
G
GG
(at all times)
L L
L
Key Constraint: Generation = Load
controllable(via markets)
G
G
G
GG
L L
L
cost
quantity
Market run by the Independent System Operator (ISO)Determines the quantity to procure and price to charge each generator in order to meet the load s.t. network constraints.
A toy example
𝐺1
𝐺2
Load = 6
capa
city
= 1
cost
quantity
cost
quantity
𝐺1
𝐺2
Load = 6
capa
city
= 1
2
1
3
𝐺1
𝐺2
Load = 6
capa
city
= 1
2
1
3
3 2
1
But what if is strategic?
𝐺1
𝐺2
Load = 6
capa
city
= 1
cost
quantity
2
1
3
Kirchhoff's laws create a hidden monopoly!
“…supply-demand imbalance, flawed market design and inconsistent rules made possible significant market manipulation” -- FERC
Kirchoff’s laws can have nasty market consequences…
“JPMorgan Chase & Co. will pay $410 million to settle U.S. Federal Energy Regulatory Commission allegations that the bank manipulated power markets, enriching itself at the expense of consumers in California and the Midwest from 2010 to 2012.”
Kirchoff’s laws can have nasty market consequences…
“Energy Capital Partners … paid $650 million last year to acquire three generating plant complexes, including the second largest electric power plant in New England, Brayton Point. Five weeks after the deal closed, Energy partners moved to shutter Brayton Point. Why would anyone spend hundreds of millions of dollars to buy the second largest electric power plant in New England and then quickly take steps to shut it down?
Kirchoff’s laws can have nasty market consequences…
How can “market power” be identified and quantified? Can markets be designed to mitigate market power?
G
G
G
GG
L
L
cost
quantity
Market run by the Independent System Operator (ISO)
Networked Cournot competition
Determines the quantity to procure and price to charge each generator in order to meet the load s.t. network constraints.
Market maker / Intermediary (ISO)
GeneratorsBid: quantityQuadratic Costs: Profit: Load
Linear demand function
Networked Cournot competition
Stochastic?
Determines the quantity to procure and price to charge each generator in order to meet the load s.t. network constraints.
ISO behavior is typically regulatedOften forced to maximize one of :1) Social welfare: Consumers’ utility – generation costs2) Residual social welfare: Consumers’ utility – generator profits3) Consumer surplus: Consumers’ utility – consumer payments
Market maker / Intermediary (ISO)Determines the quantity to procure and price to charge each generator in order to meet the load s.t. network constraints.
Choose “rebalancing quantities” to
Shift factor matrix line constraints(Kirchhoff’s Laws)
Market maker / Intermediary (ISO)Determines the quantity to procure and price to charge each generator in order to meet the load s.t. network constraints.
Choose “rebalancing quantities” to &
GeneratorsBid: quantityQuadratic Costs: Profit: Load
Linear demand function
Market maker / Intermediary (ISO)
[Barquin & Vasquez 2005, 2008], [Iklic 2009], [Neuhoff et at, 2005], [Yao, Oren, Adler, 2005, 2007] …
Networked Cournot competition
Existence?
TheoremA generalized Nash equilibrium always exists when the ISO maximizes social welfare or residual social welfare.
However, a generalized Nash equilibrium may not exist if the ISO maximizes consumer surplus.
very susceptible to market power manipulations
A toy example: “Path 15”
linear demand
]
𝐺1
𝐺2
𝐿2
𝐿1
A toy example: “Path 15”
linear demand
quadratic cost
quadratic cost 𝑐1=𝑐2
𝑐1=𝑐2
linear demand
]
𝐺1
𝐺2
𝐿2
𝐿1
A toy example: “Path 15”
linear demand
quadratic cost
quadratic cost 𝑐1=𝑐2
𝑐1=𝑐2
consumer surplus
social welfareresidual social welfare 𝐺1
𝐺2
𝐾Profi
t
Even without line constraints the 2-node network is not
equivalent to an aggregated market!
Line expansion has very different impact depending
on the market objective
linear demand
𝑟 ∈(−∞ ,∞)
𝐺1
𝐺2
𝐿2
𝐿1
A toy example: “Path 15”
linear demand
quadratic cost
TheoremA generalized Nash equilibria exist for all three objectives, but the equilibria differ considerably:- For social welfare, - For residual social welfare, .- For consumer surplus,
quadratic cost 𝑐1=𝑐2
𝑐1=𝑐2
Even without line constraints the 2-node network is not
equivalent to an aggregated market!
Line expansion has very different impact depending
on the market objective
How can “market power” be identified and quantified? Can markets be designed to mitigate market power?
What is the “right” market objective?
This talk: Two electricity market design challenges1) How many markets should there be? and when should they occur?2) The nasty economic consequences of Kirchhoff's laws
The newsvendor problemNetworked Cournot competition
Many other rich, challenging stochastic networks problems…
Renewable Energy & Electricity MarketsAdam Wierman, CaltechJoint work with Sachin Adlakha, Subhonmesh Bose, Desmond Cai, John Ledyard, Steven Low, and Jayakrishnan Nair.
Be careful what you wish for
Subhonmesh Bose, Desmond Cai, Steven Low and Adam Wierman. “The role of a market maker in networked Cournot competition.” Under submission.
Chenye Wu, Subhonmesh Bose, Adam Wierman and Hamed Mohsenian-Rad. “A unifying approach for assessing market power in deregulated electricity markets.” Proceedings of IEEE PES General Meeting, 2013. ``Best Paper on System Operations and Market Economics'' award recipient.
Jayakrishnan Nair, Sachin Adlakha and Adam Wierman. “Energy procurement strategies in the presence of intermittent sources.” Proceedings of ACM Sigmetrics, 2014.
Desmond Cai and Adam Wierman. “Inefficiency in Forward Markets with Supply Friction.” Proceedings of IEEE CDC, 2013.
This talk: 3 electricity market design challenges1) How many markets should there be? and when should they occur?2) The nasty economic consequences of Kirchhoff's laws3) Who should have control: the engineer or the economist?
the newsvendor problemnetworked Cournot competitionshadow pricing vs. VCG
Key Constraint: Generation = Load
less controllablehigh uncertainty
low uncertainty
(at all times)
Tomorrow’s grid
Demand must follow Generation(to some extent)
Grid needs huge growth in Demand Response
Demand must follow Generation(to some extent)
News articles
[ADD REFS TO DEMOS, ETC]
The big debate for demand response: The economist vs. The engineer
Prices to devices, a.k.a. “let there be markets”Send nodal price signals to consumers and let consumer devices respond
The big debate for demand response: The economist vs. The engineer
Prices to devices, a.k.a. “let there be markets”Send nodal price signals to consumers and let consumer devices respond
+ Prices can be designed so that, at equilibrium, social optimality is achieved- Consumer response is uncertain- Markets do not equilibrate instantaneously, and convergence is likely unstable
[ADD CITATIONS]
The big debate for demand response: The economist vs. The engineer
[ADD REFS TO DEMOS, ETC]
Direct control, a.k.a. “Hand over the keys”Give the utility control over consumer devices
The big debate for demand response: The economist vs. The engineer
Direct control, a.k.a. “Hand over the keys”Give the utility control over consumer devices
+ Response is fast and guaranteed- Computational demands on utility are extreme- Utility does not know customer preferences, so control is not socially optimal
The big debate for demand response: The economist vs. The engineer
How can we combine these perspectives?
“Mechanisms for control”Idea: price control policies rather than consumption
𝑟 ∈[−𝐾 ,𝐾 ]
G
G
L
L
A toy example: “Path 15”
consumerswith utility
generators w/ quadratic cost
generators w/ quadratic cost
consumerswith utility
Social objective:
“Mechanism for control”1. Consumers report 2. Utility computes allocation and prices
Social objective:
The challenges:1. Communication: Can the consumers describe their utilities?2. Incentives: Will the consumer be truthful?3. Computation: Can the utility compute the prices efficiently?