electricity spot price forecasting in two swedish …876430/fulltext01.pdfdenition 1. a market...

97
Master project Electricity spot price forecasting in two Swedish regions Analysis of factors which cause price differences between SE3 (Stockholm) and SE4 (Malmö) price regions Authors: Anna Sedova, Ekaterina Shcheglova Supervisor: Roger Pettersson Course Code: 5MA11E Subject: Mathematics and Modeling Level: Master

Upload: others

Post on 31-Aug-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Master project

Electricity spot price

forecasting in two Swedish

regions Analysis of factors which cause price differences

between SE3 (Stockholm) and SE4 (Malmö) price

regions

Authors: Anna Sedova, Ekaterina

Shcheglova

Supervisor: Roger Pettersson

Course Code: 5MA11E

Subject: Mathematics and Modeling

Level: Master

Page 2: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

AbstractFactors which effect the differences and similarities between spot prices

in the region SE3 (Stockholm) and the region SE4 (Malmo) are investi-gated. Predictors, which may influence the spot prices, are electricityconsumption, electricity production, electricity flow, electricity capac-ity, wind power production and other electricity indexes. The signifi-cance of a statistical analysis crucially depends on the choice of an ade-quate model. In this thesis the focus is on regression building. Well es-tablished candidates are the Generalized Linear Model (GLM), the Gen-eralized Additive Model (GAM) and the Autoregressive Integrated Mov-ing Average with Explanatory (ARIMAX). Preference should be givento non-parametric methods in case the data apparently cannot be ex-plained by any model at hand. For parameter estimation the methodsare the maximum likelihood method, the least-squares method and theLocal Scoring procedure. Selected models are tested on energy datafor the regions SE3 (Stockholm) and SE4 (Malmo). The models pro-vide quite good results, and allow to distinguish the degree to whicheach factor influences on the spot price. It is established that electricityconsumption, electricity production and wind power production havea major influence on the spot price formation. Besides, import/exportflows of SE3/SE4 regions partially explain the differences in the spotprice between these regions.

2

Page 3: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Abbreviations

GLM Generalized Linear ModelGAM Generalized Additive ModelARIMA Auto Regressive Integrated Moving Average

modelMCP market clearing priceMCV market clearing volumeMLE maximum likelihood estimatesIRLS iteratively-reweighed least-squaresARMA Auto Regressive Moving Average modelARIMAX Auto Regressive Integrated Moving Average

model with ExplanatoryACF autocorrelation functionPACF partial autocorrelation functionAIC Akaike information criterionBIC Schwarz Bayesian criterionTFM transfer function modelDK1, DK2 Denmark areasFI Finland areaSE1, SE2, SE3, SE4 Swedish areasNO1, NO2, NO3, NO4 Norwegian areasPL Poland area

3

Page 4: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Contents

1 Introduction 6

2 Theoretical background 102.1 The Linear model . . . . . . . . . . . . . . . . . . . . . . . 102.2 The Generalized Linear Model . . . . . . . . . . . . . . . . 11

2.2.1 The model . . . . . . . . . . . . . . . . . . . . . . . 112.2.2 The Exponential family . . . . . . . . . . . . . . . . 12

2.3 The maximum likelihood estimation . . . . . . . . . . . . 162.4 Estimation of model adequacy . . . . . . . . . . . . . . . . 20

2.4.1 The residual estimate. Adjusted coefficient of de-termination R2 . . . . . . . . . . . . . . . . . . . . 22

2.5 The Generalized Additive Model . . . . . . . . . . . . . . 222.5.1 The model . . . . . . . . . . . . . . . . . . . . . . . 222.5.2 The Local Scoring procedure . . . . . . . . . . . . . 232.5.3 Scatterplot Smoothers . . . . . . . . . . . . . . . . 23

2.6 The Auto Regressive Moving Average, the Auto Regres-sive Integrate Moving Average and the Seasonal Auto Re-gressive Integrate Moving Average models . . . . . . . . . 25

2.7 The Auto Regressive Integrated Moving Average modelwith Explanatory . . . . . . . . . . . . . . . . . . . . . . . 27

2.8 The Akaike’s Information Criterion and the Bayesian In-formation Criterion . . . . . . . . . . . . . . . . . . . . . . 29

3 Price modeling 303.1 Data and notation . . . . . . . . . . . . . . . . . . . . . . . 303.2 Data preprocessing . . . . . . . . . . . . . . . . . . . . . . 31

3.2.1 Spot prices . . . . . . . . . . . . . . . . . . . . . . . 313.2.2 Flow and capacity . . . . . . . . . . . . . . . . . . . 323.2.3 Net capacity utilization . . . . . . . . . . . . . . . . 41

3.3 Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433.4 Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

3.4.1 The Generalized Linear Model . . . . . . . . . . . . 443.4.2 The Generalized Additive Model . . . . . . . . . . 45

4 The statistical estimates of the models and predictors relativeimportance values 484.1 Comparison of the models . . . . . . . . . . . . . . . . . . 52

4.1.1 The Auto Regressive Integrated Moving Averagemodel with Explanatory . . . . . . . . . . . . . . . 54

4

Page 5: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

4.2 Analysis of predictors . . . . . . . . . . . . . . . . . . . . . 55

5 User interface 585.1 Additional modules. Implementation. . . . . . . . . . . . 585.2 The user interface usage. . . . . . . . . . . . . . . . . . . . 62

6 Conclusion 63

7 Appendix 657.1 Spot price prognosis . . . . . . . . . . . . . . . . . . . . . . 65

7.1.1 Region SE3 . . . . . . . . . . . . . . . . . . . . . . . 657.1.2 Region SE4 . . . . . . . . . . . . . . . . . . . . . . . 74

7.2 Tables of correlation coefficients r . . . . . . . . . . . . . . 837.3 Statistical estimates and p-values . . . . . . . . . . . . . . 847.4 Source code . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

References 94

5

Page 6: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

1 Introduction

In this chapter the Swedish electricity market is considered. The prob-lem is to point out factors which influence price differences between theregions SE3 (Stockholm) and SE4 (Malmo). Let us introduce some ex-plaining definitions.

Definition 1. A market clearing price (MCP) is the price of a good (forexample, electricity) when supply and demand are in equilibrium (MCP isalso called equilibrium price).

Definition 2. A market clearing volume (MCV) is the quantity of a good(for example electricity) when supply and demand are in equilibrium.

Definition 3. A supply curve is a relation between the price and the quan-tity of a good which can be traded at that price.

Definition 4. A demand curve is a relation between the price of a good andthe quantity of this good which consumers would like and be able to buy atthat price.

The main principles of power pool and power exchange are illustratedin Figure 1.

Figure 1: Power pool vs. power exchange price formation mechanism

A power pool is a market for electricity. Power pools mainly were cre-ated by governments to introduce competition in power generation. Inthis case the market clearing price (MCP) is establishing through the in-tersection of the supply curve and the estimated demand (which auto-matically defines the market clearing volume, MCV). The participantscan only be power generators. By power generators are meant powerproducers.

6

Page 7: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Definition 5. A power exchange is a market for electricity. Power ex-changes mainly launch on a private initiative. In this case the market clear-ing price (MCP) is establishing through the intersection of the supply curveand the demand curve. The participants, for instance, can be a combinationof power generators, distributors and traders.

The Nord Pool (The Nordic commodity market for electricity) was es-tablished in 1992 in Norway [[1], [2]]. Despite of the name Nord Pool, itis a power exchange, not a power pool. Nord Pool is the oldest and one ofthe most mature power exchanges in the world. Today there are over 300market participants active on the Nord Pool. These include generators,suppliers/retailers, traders, large customers and financial institutionsfrom over 10 countries including Sweden (from 1996).

Sweden is divided into several price areas: SE1 (Lulea), SE2 (Sundsvall),SE3 (Stockholm), SE4 (Malmo) from year 2011 [3] (see Figure 2).

Figure 2: Swedish bidding areas. In northern Sweden there is a surplus of electricity production com-pared with the demand for electricity. In southern Sweden, the circumstances are the opposite

The main purposes for this partition were:

• to indicate where it is necessary to extend the electricity grid;

7

Page 8: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

• to indicate where in the country it is necessary to increase the gen-eration of electricity;

• to decrease the need for long-distance electricity transmission.

Definition 6. A spot price is a one-day ahead price, which is settled before12 a.m. by specialists of the Nord Pool.

Bixia is Sweden’s fourth largest electricity supplier and pursues powertrading on Nord Pool since 1999 [4]. Through Bixia many small electriccompanies operate in their community and provide personalized ser-vice. The suppliers of Bixia produce electricity from renewable sourcessuch as solar, wind, biomass and small hydro.

One field of interest for Bixia is understanding and managing the rela-tionships between the daily values of prices (see Definition 6) in the SE3and SE4 areas (see Figures 3, 4), using explanatory variables to measuretime and seasonal effects and the wind power production via historicaldata. The data we have analyzed is from November 1st, 2011 to Decem-ber 31st, 2012.

Price forecasts are necessary to develop bidding strategies in order tomaximize profits. Factors which have large impact on electricity priceformation are important for spot price forecasting in order to indicatewhere to extend the electricity grid, increase the generation of electricityand to decrease outlays correlated with long-distance electricity trans-mission.

Figure 3: Spot prices of SE3 and SE4. Note that the spot price of SE4 is always higher than the SE3 spotprice

The wind power production of the Denmark areas (DK1, DK2) wasused, because there are no free Swedish wind power production data.Data of DK1 were used as data for SE3 and data of DK2 were used as

8

Page 9: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 4: Daily SE3/SE4 spot price rations. The relationship between prices all over the year

data for SE4, because they are closely located (geographically) to eachother (see Figure 2).

Definition 7. An Nord Pool electricity capacity (trading capacity) is aone-day ahead capacity, which is published by the transmission system oper-ators on Nord Pool Spot’s web site before 10:00. Import/export of electricitycan not be higher than the trading capacity between each bidding area.

Factors which may influence the spot prices are electricity consump-tion, electricity production, electricity flow, electricity capacity (see Def-inition 7), wind power production and other electricity indexes. Thiswill be investigated in the thesis. It is not evident that the response vari-able and the predictors have linear dependence. The Generalized Lin-ear Model (GLM), which can handle non-linear dependencies betweenthe predictors and the response variable, can be viewed as a model-ing approach. Moreover, if it is impossible or very complicated to es-tablish dependence analytically one can use the Generalized AdditiveModel (GAM). Furthermore, there are well-known methods for forecast-ing stochastic processes, for which the first and the second moments de-pend on time, for example linear regression where the residuals may bedependent and therefore are often modeled by ARIMA (Auto RegressiveIntegrated Moving Average Model).

9

Page 10: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

2 Theoretical background

Recall that the problem is to investigate which variables influence theprice differences between SE3 (Stockholm) and SE4 (Malmo) regions ap-peared in the data.

The problem is to build a model. To solve it we need to model the de-pendence of the electricity prices from a number of factors, such as thecapacity, the flow, the electricity consumption, the electricity produc-tion, the wind power production and the net capacity utilization. In thisthesis the Generalized Linear Model (GLM), the Generalized Additivemodel (GAM) and Auto Regressive Integrated Moving Average modelwith Explanatory (ARIMAX) will be used. Parameter estimates for themodels will be obtained by the Maximum Likelihood method, the least-squares method and the Local Scoring procedure.

2.1 The Linear model

Assume, for instance, the multiple linear regression model

Y = X ′β + ε, (1)

with X = (1,X1, · · · ,Xk) as predictor variable for a response variable (ordependent variable) Y , with parameter β = (β0,β1, · · · ,βk), ε ∼ N (0,σ 2)[5] .

Here we note that Y given (1,X1 = x1, · · · ,Xk = xk) is N (x′β,σ 2), wherex = (1,x1, · · · ,xk)′.

In particular we have

E[Y |X = x] = x′β,

where xi for i = 1, · · · , k are not correlated. The reason we let the firstcomponent of X and x to be one is that then we include an intercept β0

into the model, i.e.

Y = β0 + β1X1 + · · ·+ βkXk + ε.

Let us provide some examples.

Example 1: An example of a linear relationship between y and x.

E[Y |X = x] = β0 + β1x,

whereβ = (β0,β1)′,

x = (1,x)′.

10

Page 11: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

The vector x is here typographed in bold to avoid ambiguity.Example 2: An example of a cubic model of a relationship between

y and x.

E[Y |X = x] = β0 + β1x+ β2x2 + β3x

3,

whereβ = (β0,β1,β2,β3)′,

x = (1,x,x2,x3)′.

2.2 The Generalized Linear Model

In this chapter the Generalized Linear Model (GLM) and the Exponen-tial family of distributions will be introduced. The Generalized LinearModel (GLM) was formulated by Nelder and Wedderburn in 1972 [6].

2.2.1 The model

For the Generalized Linear Model (GLM) we assume

g(E[Y |X = x]) = x′β (2)

for some injective function g, named link function .It means that

E[Y |X = x] = g−1(x′β),

since the link function is one-to-one.By denoting

µ = E[Y |X = x]

andη = x′β

we haveη = g(µ). (3)

For usual linear regression the link function g is the identity function,g(y) = y, so we have

E[Y |X = x] = x′β. (4)

It should be noted that the Generalized Linear Model (GLM) is

g(E[Y |X = x]) = x′β

which, if g is non-linear, is not the same as

E[g(Y )|X = x] = x′β.

11

Page 12: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

In linear modeling Y is typically assumed to be normally distributed.One more way to generalize the usual linear model is to assume Y to

be a more general random variable than a normally distributed one.

2.2.2 The Exponential family

The Generalized Linear Model (GLM) is a powerful generalization oflinear regression to more general Exponential family. The response vari-able Y in the Generalized Linear Model (GLM) may have any distribu-tion from the Exponential family which includes the Normal, binomial,Poisson, exponential, gamma and inverse Gaussian.

A random variable Y is said to belong to the Exponential family ifits probability density depending on the parameter θ ∈ Rk, where θ =(θ1, · · · ,θk), can be written in the form

f (y) = exp(C(θ)′T (y)−A(θ) +B(y)),

where C(θ) = (C1(θ), · · · ,Cd(θ))′, T (y) = (T1(y), · · · ,Td(y))′. Note that dmay not be equal to k.

A distribution is said to belong to the overdispersed exponential familyif its probability density depending on the parameter

(θ,ϕ) = (θ1, · · · ,θk,ϕ1, · · · ,ϕl)can be written in the form

f (y) = exp{yθ − b(θ)a(ϕ)

+ c(y,ϕ)}, (5)

where b(θ), a(ϕ) and c(y,ϕ) are known functions that not the same fromone Exponential family to another [5].

Note that if a distribution belongs to the overdispersed exponentialfamily it may not belong to the Exponential family, but if ϕ is fixed orknown then the distribution belongs to the Exponential family. The pa-rameter θ is interpreted as a location parameter andϕ a scale parameter.

Note that in GLM literature a density of the form (5) is said to belongto the Exponential family which may confuse readers somewhat.

The mean and the variance of a distribution from the Exponentialfamily are the following [7]:

E(Y ) = b′(θ), V ar(Y ) = a(ϕ)b′′(θ), (6)

which can be seen by considering the moment generating function

M(t) = E[etY ] =∫etyf (y)dy.

12

Page 13: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

For purpose of self-containment a proof of (6) is included.Since

M ′(t) = E[Y etY ], M ′′(t) = E[Y 2etY ],

we obtain the well-known relations:

E[Y ] =M ′(0), V ar[Y ] = E[Y 2]− (E[Y ])2 =M ′′(0)− (M ′(0))2.

Since

1 =∫f (y)dy =

∫eθy−b(θ)a(ϕ) +c(y,ϕ)

dy,

we have

eb(θ)a(ϕ) =

∫eθya(ϕ) +c(y,ϕ)

dy.

Hence

M(t) =∫ety+θy−b(θ)

a(ϕ) +c(y,ϕ)dy =

= e−b(θ)a(ϕ)

∫eθ+ta(ϕ)a(ϕ) y+c(y,ϕ)

dy =

= e−b(θ)a(ϕ)e

b(θ+ta(ϕ))a(ϕ) =

= eb(θ+ta(ϕ))−b(θ)

a(ϕ) .

HenceM ′(t) = b′(θ + ta(ϕ))M(t),

M ′′(t) = a(ϕ)b′′(θ + ta(ϕ))M(t) + (b′(θ + ta(ϕ))2M(t)

andM ′(0) = b′(θ)M(0),

M ′′(0) = a(ϕ)b′′(θ)M(0) + (b′(θ))2M(0)

and consequently, since M(0) = 1,

E[Y ] =M ′(0) = b′(θ),

V ar[Y ] =M ′′(0)− (M ′(0))2 = a(ϕ)b′′(θ).

An alternative proof of (6) is shortly described in [8]. Here we present amore explicit version

l = logf .

13

Page 14: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

We have∂l∂θ

=∂f∂θ

f,

∂2l∂θ2 =

∂2f

∂2θ1f−(∂f

∂θ

)2 1f 2 .

Hence

E

[∂l∂θ

]=

∫ ∂f∂θ

ff dy =

∫∂f

∂θdy =

∂∂θ

∫f dy =

∂∂θ

1 = 0,

E

[∂2l∂θ2

]=

∫ ∂2f

∂2θ1f−(∂f

∂θ

)2 1f 2

f dy =∫∂2f

∂θdy −

∫ ∂f∂θf

2

f dy =

=∂2

∂θ2

∫f dy −E

( ∂l∂θ)2 = −E

( ∂l∂θ)2 .

In our case withf = e

θy−b(θ)a(ϕ) +c(y,ϕ)

,

l =θy − b(θ)a(ϕ)

+ c(y,ϕ),

∂l∂θ

=y − b′(θ)a(ϕ)

,

∂2l∂θ2 = −b

′′(θ)a(ϕ)

.

0 = E[∂l∂θ

]=E[Y − b′(θ)]

a(ϕ)⇒ E[Y ] = b′(θ)

0 = E[∂2l∂θ2

]+E

( ∂l∂θ)2 = −b

′′(θ)a(ϕ)

+1

a(ϕ)2E[(Y − b′(θ))2

]⇒ V ar[Y ] = a(ϕ)b′′(θ)

Different choices of the function b(θ) generate different distributionsin the Exponential family.

Example 1.Let us consider the Gaussian distribution. This distribution has the

density function

f (y) =1

√2πσ 2

exp{−

(y −µ)2

2σ 2

}. (7)

14

Page 15: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Let us show that the Gaussian distribution is from the Exponentialfamily. Transforming (7) we can see that the Gaussian distribution isfrom the Exponential family:

f (y) =1

√2πσ 2

exp{−

(y −µ)2

2σ 2

}=

exp{−y2 + 2yµ−µ2

2σ 2 − log(σ√

2π)}

=

exp{yµ−µ2/2

σ 2 −y2

2σ 2 − log(σ√

2π)}.

So, we obtain (5) with θ = µ, b(θ) = θ2/2 = µ2/2, a(ϕ) = ϕ = σ 2, c(y,ϕ) =−y2/(2ϕ)− log(

√ϕ2π) = −y2/(2σ 2)− log(σ

√2π).

According to (6), the Gaussian distribution has the mean

E(Y ) = b′(θ) = θ = µ

and the varianceV ar(Y ) = a(ϕ)b

′′(θ) = σ 2.

Example 2.Let us consider the example of the standard linear model. This model

can be described as the GLM with normal errors and identity link func-tion g(µ) = µ, so that

η = µ.

For the Generalized Linear Model (GLM) the equation linking eachvalue of the response variable Y and a set of n predictors that take valuesx = (1,x1,x2, · · · ,xk)′ has the following form [6]:

g(µ) = β0 + β1x1 + · · ·+ βxk, (8)

where β is a vector of unknown parameters, µ = E[Y |X = x].Thus, the Generalized Linear Model (GLM) has the following three

components [6]:

• A set of parameters β and explanatory variables (predictors), x =(1,x1,x2, . . . ,xk)′, where xi for i = 1, · · · , k are not correlated;

• A response variable Y whose density for all outcomes of the predic-tors x = (1,x1,x2, · · · ,xk) have the same form

f (y) = exp(θy − b(θ)a(ϕ)

+ c(y,ϕ))

15

Page 16: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

but where the parameter θ may depend on x;

• A monotone link function g(µ).

2.3 The maximum likelihood estimation

Estimation for Generalized Linear Models (GLM)(2) is based on the the-ory of maximum likelihood estimation.

The maximum likelihood estimation begins with writing the likeli-hood function of the given data. This function includes the unknownparameters. The values of the parameters that maximize the likelihood,are known as the maximum likelihood estimates (MLE) [9] .

The maximum likelihood estimate β of β = (β0,β1, · · · ,βk) is given bya system of equations. These equations were shortly motivated in [8].Here we give a more explicit motivation. First assume we have only onepair of observation (x,y) where y is an outcome of Y , x = (1,x1, · · · ,xk) anoutcome of X = (1,X1, · · · ,Xk).

Let in GLM (see (2))µ = E[Y |X = x]

and with notation as in (3), section 2.2.1

η = g(µ).

Assumeη = x′β

andfY |X=x(y) = e

yθ−b(θ)a(ϕ) +c(y,ϕ)

.

The maximum likelihood method is to find β such that it maximizesfY |X=x(y) i.e., minimizes

l = logfY |X=x(y) =yθ − b(θ)a(ϕ)

+ c(y,ϕ).

This means that we want to find β such that

∂l∂βj

= 0, j = 1, · · · , k,

assuming there is no restriction of β, else it is more complicated.Now

∂l∂βj

=∂l∂θ

dθdµ

dβj,

dldθ

=y − b′(θ)a(ϕ)

16

Page 17: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

dθdµ

=1dµdθ

= [µ = E[Y |X = x] = b′(θ)] =1

db′(θ)dθ

=1

b′′(θ)

dβj=dx′β

dβj= xj ,

where xj is the jth component of x.Now recall that V ar[Y |X = x] = a(ϕ)b′′(θ).Hence

∂l∂βj

=y − b′(θ)a(ϕ)

1b′′(θ)

dηxj =

y −µV ar[Y |X = x]

dηxj ,

where xj is the jth component of x. Now consider ∂l∂βj

for n pairs of (x,y)-observations. The y-observations are denoted by (y1, · · · , yn). For conve-nience denote now the x-observations by x1, · · · ,xn where

x1 = (x10, · · · ,x1k) . . .xn = (xn0, · · · ,xnk)

where xij is the jth component of the ith observation. Each yi is an obser-vation of Yi which is interpreted as the conditional variable of Y givenX = xi, ”Yi = Y |X = xi” or more exact

P (Yi ≤ y) = P (Y ≤ y|X = xi), ∀ y.

Letµi = E[Y |X = xi], ηi = x′iβ,

where we recall that now the (k+1) vector xi is the ith x-observation.Assume ηi = g(µi) and

fYi (y) = eyθi−b(θi )a(ϕ) +c(y,ϕ)

,

it means θ depends on i but not ϕ.Now

l = logfY1(y1) · · · ·fYn(yn) =

n∑i−1

[yθi − b(θi)a(ϕ)

+ c(yi ,ϕi)]

=n∑i−1

li

and∂l∂βj

=n∑i=1

∂li∂βi

,

where∂li∂βj

=yi −µiV ar[Yi]

dµidηi

xij .

17

Page 18: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Hence the maximum likelihood estimate of β = (β0,β1, · · · ,βk) should sat-isfy the equations

n∑i=1

yi −µiV ar[Yi]

dµidηi

xij = 0, j = 0, · · · , k. (9)

Again, if there are restrictions on β then it is more complicated.To solve these equations the Fisher scoring procedure [10] can be

used.The adjusted dependent variable regression is an equivalent proce-

dure which has the form of iteratively-reweighed least-squares (IRLS).The system of equations (9) can be numerically solved by an iterative

procedure. Let us also here, for the purpose of self-containment brieflyaccount for the procedure.

The procedure takes into consideration the uncertainty of g(Yi) by theweigh

wi =1

V ar[Yi](dηidµi

)2. (10)

The particular choice for the weight can be motivated by the first orderTaylor expansion of the function g in (2) where we expand at the point

g(Yi) ≈ g(µi) + (Yi −µi)g ′(µi),

where dew to (3)

g ′(µi) =dg(µi)dµi

=dηidµi

.

Hence

V ar[g(Yi)] ≈ V ar[Yi](dηidµi

)2

.

We can rewrite

yi −µiV ar[Yi]

dµidηi

xij =1

V ar[Yi](dηidµi

)2 (yi −µi)(dηidµi

)2dµidηi

xij = wi(yi −µi)dηidµi

xij .

Hence we want to find β that solves ∂l∂βj

= 0, j = 0,1, · · · , k, where

∂l∂βj

=n∑i=1

wi(yi −µi)dηidµi

xij .

The procedure is as a variant of Newton’s optimization method namedas Fisher’s scoring method [10]. Given an estimate β(N ), of β, an estimateβ(N+1) of β is obtained as

β(N+1) = β(N ) + I−1uN ,

18

Page 19: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

where

uN = ∇βl =(∂l∂β0

, · · · , ∂l∂βn

)′evaluated at β = β(N ),

I = E[− ∂2l∂βi∂βj

]is the matrix evaluated at β = β(N ) [10].

Let us see how this algorithm can be implemented.

Iij = E[− ∂∂βi

(∂l∂βj

)]= E

− ∂∂βi n∑r=1

wr(yr −µr)dηrdµr

xrj

,

where

∂∂βi

(wr(yr −µr)

dηrdµr

xrj

)= (yr −µr)

∂∂βi

(wrdηrdµr

xrj

)+wr

dηrdµr

xrj∂∂βj

(yr −µr).

The first term has zero mean and the second equals

−wrdηrdµr

xrj∂µr∂βi

== −wrxrj∂ηr∂βi

= −wrxrj∂x′rβ

∂βi= −wrxrjxri .

Hence

Iij =n∑r=1

wrxrixrj

Nowβ(N+1) = β(N ) + I−1u(N )

can be written asIβ(N+1) = Iβ(N ) +u(N ).

Now

(Iβ(N ))j =k∑j=0

Iijβ(N )i =

k∑j=0

n∑r=1

wrxrixrjβ(N )i =

n∑r=1

wrxrj

k∑j=0

xriβ(N )j =

n∑r=1

wrxrjηr ,

where ηr = x′rβ is evaluated at β = β(N ).

(Iβ(N+1))j =n∑r=1

wrxrj

k∑i=0

xriβ(N+1)i

u(N )j =

∂l∂βj

=n∑r=1

wr(yr −µr)dηrdµr

xrj .

HenceIβ(N+1) = Iβ(N ) +u(N )

19

Page 20: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

meansn∑r=1

wrxrj

k∑i=0

xriβ(N+1)i =

n∑r=1

wr

(ηr + (yr −µr)

dηrdµr

)xrj ,

which, by

zr = ηr + (yr −µr)dηrdµr

,

z = (z1, · · · , zn)′,can be written in the form

X ′WXβ(N+1) = X ′WZ.

Thenβ(N+1) = (X ′WX)−1X ′Wz, (11)

where W is the diagonal matrix of the weights wi(10) and X the designmatrixX = (xij). So, we can obtain the improved estimate of β regressingthe variable z on the predictors x using the weights w.

Example.Most statistical packages that include procedures for fitting the GLM

have an efficient algorithm based on (11). They begin by using someinitial approximation β(0) to evaluate z and W , then (11) is solved togive β(1) which in turn is used to obtain better approximations for z andW (11), and so on until adequate convergence is achieved.

Let us consider the case with the identity link of the usual linear model.Then η = µ, and ∂η/∂µ is equal to 1 and the variable z is y itself.

The weights are then constant and no iteration is required. Startingvalues may be obtained by applying the link to the data, namely, µ = yand η = g(µ) = µ are taken.

2.4 Estimation of model adequacy

If g(y) = y, i.e. the usual linear regression

E[Y |X = x] = x′β,

then with y = xβ and with the identity matrix W = I ,∑(yi − y)2 =

∑(yi − yi)2 +

∑(yi − y)2,

i.e.T SS = ESS +RSS, (12)

where TSS is a total sum of squares, ESS is an explained sum of squaresand RSS is a residual sum of squares [11] .

20

Page 21: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Note that if the link function g(y) , y or ifW , I (W is not the identitymatrix) then, generally,

T SS , ESS +RSS.

For our data we assume for simplicity the Gaussian distribution withthe unit link. Then W is proportional to the identity matrix. Hence wecan apply the coefficient of determination R2 to measure adequacy ofthe regression.

The Generalized Linear Model opens up for more general models, forinstance, by replacing the Gaussian distribution by some other distribu-tion or replacing the unit link function by another link.

For comparison of the ARIMAX, the GAM and the GLM models weuse the AIC information criterion, which will be considered below.

The coefficient of determination R2 using the notation given above isdefined as

R2 = 1− ESST SS

=RSST SS

. (13)

The coefficient of determination R2 is one of the most well-knownways to measure adequacy of the regression. In regression, this coeffi-cient R2 is the statistical measure of how well the regression model ap-proximates the real data points. If the R2 is close to 1 (or 100%), then theregression model to a large extent describes the relationship between theresponse Y and predictor variables, at least for the GLM with the Gaus-sian distribution and the unit link, it means the usual linear regression.R2 is often interpreted as the proportion of response variation ”ex-

plained” by the regressors in the model. For instance, the value of thecoefficient R2 = 0.8 can be interpreted as follows:

”Eighty percent of the variation in the response variable can be ex-plained by the explanatory variables”.

Note that R2 will never decrease if a variable is added to the model,whether or not it helps to explain the response variable Y . When a newvariable is added to the model, the ESS is calculated over a larger setof variables, and ESS will be greater than or equal to what was before.This will cause that the coefficient R2 to increase, even if the addition ofnew variable does not influence the quality of the model.

There is a tool to fix this problem. R2 is replaced with the adjusted R2

which adjusts it for the added degrees of freedom.

21

Page 22: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

2.4.1 The residual estimate. Adjusted coefficient of determinationR2

The adjusted coefficient of determination of a multiple linear regressionmodel is defined in terms of the coefficient of determination R2 as fol-lows:

R2 = 1− (1−R2)n− 1

n−m− 1, (14)

where m is the number of predictors, and n is the sample size [11].The coefficient R2 is equal or less than the R2. The adjusted coefficient

of determination R2 can take values less than 0.Recall that the coefficient of determinationR2 will increase if any vari-

able is added to the regression, while R2 will increase if and only if theindependent variables, which are added to the regression help to explainthe response better.

Obviously, it is impossible to add an infinity quantity of variables tothe model because of multicollinearity effects.

Multicollinearity occurs when the model includes multiple predictorsthat are strongly correlated not just to the response variable, but also toeach other.

For the GLM with other distributions than the Gaussian and otherlinks than the unit link there is a wide variety of so called pseudo R2,see e.g. [12].

2.5 The Generalized Additive Model

The Generalized Additive Model (GAM) was formulated by Trevor Hastieand Robert Tibshirani in 1986 to denote additive extensions of the classof Generalized Linear Models (GLM) [9].

The central idea of GAM is to replace the linear form of a predictorwith a specified smooth function.

2.5.1 The model

For one predictor variable X, the model will be

g(E[Y |X = x]) = s(x), (15)

where s is an unspecified smooth function.For multiple predictor x = (x1,x2, · · · ,xn)′, the function s(x) is assumed

to be

s(x) = s0 +n∑i=1

sj(xj). (16)

So, the GAM consists of a sum of smooth functions.

22

Page 23: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Introducing the same notation as in (3) we can then write g(µ) = η,where

η = s0 + s1(x1) + s2(x2) + · · ·+ sn(xn), (17)

s0 is a constant, µ = E[Y |X1 = x1,X2 = x2, · · · ,Xn = xn] is the conditionalexpectation, sj(xj), for j = 1,2, · · · ,n, are smooth functions standardizedso that E(sj(xj)) = 0 and estimated one at a time. Estimation of eachsj(xj) is achieved through a scatterplot smoother (see 2.5.3). The Gen-eralized Additive Model (GAM) is nonparametric in that one does notimpose a parametric form on the functions but instead estimates themin an iterative manner through the use of scatterplot smoothers whichwill be described below.

2.5.2 The Local Scoring procedure

We recall that the response Y is assumed to have a distribution given byequation (15), with the mean µ = E(Y |X1 = x1, · · · ,Xn = xn) linked to thepredictors through

g(µ) = s0 + s1(x1) + · · ·+ sn(xn). (18)

Estimation of s0, s1, · · · , sn is accomplished by replacing the weighted lin-ear regression in the adjusted dependent variable regression by an ap-propriate one [9].

The Local Scoring procedure has the advantage of being completelyautomatic [9]. The name Local Scoring comes from the fact that a localaveraging is used to generalize the Fisher scoring procedure.

2.5.3 Scatterplot Smoothers

A smoother is a tool for summarizing the trend of a response Y as a func-tion of predictors X1,X2, · · · ,Xn. It produces an estimate of the trend thatis less variable than Y itself, therefore it is named a smoother. An impor-tant property of a smoother is its nonparametric nature. For this reasona smoother is often referred to as a tool for nonparametric regression.

The smoother is mainly used for description (it helps the eyes to pickout the trend in the plot) and estimation of the dependence of the meanof Y given the predictors.

There are two main decisions to be made in scatterplot smoothing [9]:

• how to average the response values in each neighborhood;

• how large the neighborhoods should be.

23

Page 24: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Let us for simplicity assume that the predictor is one-dimensional andits observations x1, · · · ,xn are sorted x(1) < x(2) < · · · < x(k). Let y1, · · · , ykbe the corresponding response observations. For convenience let x =(x1, · · · ,xk) and y = (y1, · · · , yk).

Examples of smoothers are:the running-mean and running-line smoothers, bin smoothers, ker-

nel smoothers, running median, running least squares line or regressionspline [5].

The running-mean smoothersTo estimate a smooth function x 7→ s(x) from data. For that we need

to determine a symmetric nearest neighborhood [9] of a data point xi.A simple way is to choose the k points to the left and k points to the

right of xi that are closed to xi.Let us denote the indexes of these points by N s(xi), it means

N s(xi) = {max(i − k,1), · · · , i − 1, i, i + 1, · · · ,min(i + k,n)}. (19)

Then the running-mean smoother

s(xi) = Averagej∈N s(xi )(yj).

This simple smoother is also called a moving average [5].The running− line smoother is a simple generalization of the run-

ning mean defined bys(xi) = β0 + β1xi , (20)

where β0, β1 are least-squares estimates for the data points in N s(xi).The parameter k controls the appearance of the running-line smoother.Large values of k tend to produce smooth curves while small values gen-erally produce more jagged curves.

A span is the proportion of points in each neighborhood [9]. Let usdenote the span as

w = (2k + 1)/n.

If the span w is equal to 2, then each neighborhood includes all data. Ifw is equal to 1/n, it means that each neighborhood consists only of onedata point and therefore the smoother interpolates the data.

In the Local Scoring procedure a running-line smoother is used, whichrequires a choice of span size w. Note that other smoothers can be alsoused. For more details see [9].

The running-line smoother typically produces reasonable results. Oneconvenient property is that the estimate in a neighborhood can be foundby updating the estimate of the previous neighborhood. This smootherrelieves the problem of large biases at the end points.

24

Page 25: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

The Local Scoring procedure is equivalent to another method of es-timating, the Local likelihood estimation [9]. In the local likelihoodestimation it is assumed that s(x) is locally linear, which means it fitsa line in a neighborhood around each xi value. These procedures givevery similar results, but for large data sets, the Local Scoring procedureis faster.

2.6 The Auto Regressive Moving Average, the Auto Regressive In-tegrate Moving Average and the Seasonal Auto Regressive Inte-grate Moving Average models

The Auto Regressive Moving Average (ARMA) model is one of the mostused time series models developed by Box and Jenkins [13] and is there-fore also referred to as the Box-Jenkins model. This method uses histor-ical data of univariate time series to analyze its own trend and forecastfuture cycles.

One can incorporate one or more time series in a model to predict thevalue of another series using a transfer function. These functions can beused both to model and forecast the response series and to analyze theimpact of the intervention.

The ARMA model is suitable to analyze univariate stochastic time se-ries. For the ARMA approach the analyzed time series must be station-ary, i.e. the mean, variance and covariance of the series are all con-stant over time. Stationarity is important because, if the series is non-stationary, all the typical results of the classical regression analysis arenot valid. For ARIMA modeling, differences of data are considered inorder to obtain stationary data. The original data are finally integrated,which explains the I in ARIMA.

When the ARIMA model includes other time series as input variables,the model is sometimes referred to as an ARIMAX model [13]. The ARI-MAX model is somewhat more detailed described in Section 2.9.

The SARIMA model, the ARIMA with seasonal terms, can be writtenas follows [14]:

yi = φ1yi−1 +φ2yi−2 + · · ·+φpyi−p +Φ1yi−s +Φ2yi−2s + · · ·+ΦP yi−P s+

+at −θ1ai−1 −θ2ai−2 − · · · −θqai−q −Θ1ai−s −Θ2ai−2s − · · · −ΘQai−Qs.(21)

We can rewrite (21) using the backshift operator (or lag) B,Byi = yi−1[14]:

φp(B)Φp(Bs)zi = Θq(B)ΘQ(BS)ai , (22)

where

25

Page 26: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

zi = (1−B)d(1−Bs)Dyi ,φp(B) is a nonseasonal autoregressive AR(p) polynomial in B,Θq(B) is a nonseasonal moving average MA(q) polynomial in B,Φp(Bs) is a seasonal autoregressive AR(P ) polynomial in Bs,ΘQ(Bs) is a seasonal moving average MA(Q) polynomial in Bs,ai is an error term (white noise, i.e. a sequence of independent equally

distributed random variables),s is a seasonal order (Bsyi = yi−s),d,D are nonseasonal and seasonal orders of differencing (integration).Then we can rewrite (22) using a more parsimonious notation as fol-

lows:SARIMA(p,d,q)(P ,D,Q)s, (23)

wherep,P are the numbers of autoregressive parameters,q,Q are the numbers of moving average parameters.For the ARMA model, d = D = 0, Θq = ΘQ = I , and for the ARIMA

model, D = 0, θq = θQ = I .The Box-Jenkins approach is an iterative four-stage modeling approach

which includes [14]:

• identification;

• estimation;

• diagnostic checking;

• finally forecasting.

Let us look at each of the steps separately.

• identification stage;In this stage the researcher visually examines the time plot of theseries autocorrelation function (ACF) and the partial autocorrelationfunction (PACF). Plotting each observation of the series against timei provides useful information concerning outliers, missing valuesand structural breaks in the data. The analyzed time series must bestationary. Once stationarity has been achieved (as taking logarithmand/or differences), then one needs to identify the parameters of themodel, i.e. AR and MA orders examining the ACF and the PACF.

• estimation stage;In this stage, each of the tentative models are estimated and the var-ious coefficients are examined. The estimated models are compared

26

Page 27: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

using the Akaike information criterion(AIC)([13]) and the SchwarzBayesian criterion (BIC)([15]). The main approaches of fitting Box-Jenkins models are non-linear least squares and maximum likeli-hood estimation.

• diagnostic checking stage;In this stage the goodness of fit of the model is examined. Residualsshould meet Gaussian white noise assumptions, i.e. autocorrelationand normality is tested. If these assumptions are not satisfied, oneneeds to fit a more appropriate model. Care must here be taken toavoid overfitting.

• forecasting stage;The main function of ARIMA models is forecasting. Their forecast-ing ability can be considered when forecasts are compared to actualtime series.

2.7 The Auto Regressive Integrated Moving Average model with Ex-planatory

Here we give a brief introduction to the ARIMAX model.Let us assume two time series which are denoted yi and xi, respec-

tively, which are both stationary.Then, the transfer function model (TFM) can be written as follows [13]:

yi = C + ν(B)xi +Ni , (24)

where yi is the output series (or dependent variable), xi is the input se-ries (or independent variable), C is a constant term, Ni is a stochasticdisturbance, i.e. a noise series of the system that is independent of theinput series .ν(B)xi is a transfer function (or impulse response function), which al-

lows X to influence Y through a distributed lag. B is again a backshiftoperator Bxi = xi−1. The function ν is a polynomial ν(x) = ν0 + ν1x + · · ·and hence

ν(B)xi = (ν0 + ν1B+ ν2B2 + · · · )xi = ν0xi + ν1xi−1 + ν2xi−2 + · · · . (25)

When xi and Ni are assumed to follow ARIMA models, the transferfunction model (24) is known as an ARIMAX model. This ARIMAXmodel is quite different from an ARIMA model, because there are twodifferent series, the input series xi and the output series yi.

The coefficients νj are called impulse response weights, which could bepositive or negative. The larger the absolute value of any weight νj is,the larger is the response of yi to a change in xi−j . Output series might

27

Page 28: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

not react immediately to a change in input series, thus some initial νweights may be equal to zero. The number of initial ν weights equal tozero is called dead time and is here denoted by b [14].

Theoretically, the transfer function ν(B)xi has an infinite number ofcoefficients. We can however write the transfer function as a rationalpolynomial distributed model of lag of finite order

ν(B)xi =ωh(B)Bb

δr(B)xi (26)

andωh(B) = ω0 +ω1B+ · · ·+ωhBh,δr(B) = 1− δ1B− · · · − δrBr ,

where h is the number of terms plus one of the included independentvariables, r is the number of terms of the included dependent variable[14].

The disturbance series Ni can be written in the form of an ARIMAmodel as follows [14]:

Ni =θ(B)Θ(Bs)

φ(B)Φ(Bs)(1−B)d(1−Bs)Dai , (27)

where ai is a zero mean and normally distributed white noise.Then, substituting (25) with a maximum lag denoted by K and (27)

into (24), one has the transfer function model in the full formula [14]:

yi = C + ν0xi + ν1xi−1 + ν2xi−2 + · · ·+ νKxi−K+

+θ(B)Θ(Bs)

φ(B)Φ(Bs)(1−B)d(1−Bs)Dai .

(28)

To construct a TFM is a similar iterative process as to construct a uni-variate Box-Jenkins ARIMA model.

After checking that there is no feedback from earlier values of theoutput to current values of the input, one can start with a linear transferidentification method (LTF) to find out the orders (b,r,h) of a rationalform transfer function [17]. First, we specify the free-form distributedlag model in which K is chosen according to the analyst’s judgment andthen we specify low orders for the disturbance series Ni. A nonlinearleast square method can be used to estimate parameters.

After estimation of the model, one has to check the estimated dis-turbance series for stationarity by means of the sample autocorrelationfunction and the sample partial autocorrelation function. If the distur-bance series is not stationary, then it is necessary to differentiate input

28

Page 29: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

and output accordingly. If the disturbance is stationary, then one cango to the stage 2 where one may use preliminary estimated impulse re-sponse weights to choose the orders (b,r,h) of one/few tentative rationalform transfer function(s) to represent ν(B). One can identify the orders(b,r,h) by visually comparing the estimated impulse response functionwith some common theoretical functions. If the linear transfer functionmodel is adequate then one can compute forecasts. There are severaldiagnostic checks to decide whether the model is adequate based on theresiduals which should be independent of each other as well to the in-put series, for example cross-correlation check and/or autocorrelationcheck. It is a good practice to build ARIMA models for both the out-put and the input series before attempting to build a transfer functionmodel [16].

2.8 The Akaike’s Information Criterion and the Bayesian Informa-tion Criterion

To select the orders p and q, the Akaike’s Information Criterion (AIC) orthe Bayesian Information Criterion (BIC) could be used [13], [15],

AIC = −2log(L) + 2n, (29)

BIC = −2log(L) +nlog(m), (30)

where

• L is the likelihood of the data with a certain model;

• m is the number of observations;

• n is the number of parameters, which is equal to (p + q) for theARMA(p,d,q) model.

The orders p and q are selected so that AIC and BIC, respectively, areminimized. Note that a model typically fits better as a model gets morecomplicated. It means that −2log(L) gets smaller, but n gets bigger. Apreferred best model is a model, which achieves a compromise betweencomplexity and quality of the model.

It is crucial to evaluate the forecast accuracy, because it is importantto look at how well the model fits the new data. The accuracy of theforecast can only be determined by considering how well the modelperforms on the new data that wasn’t used when the model was con-structed.

29

Page 30: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

3 Price modeling

The problem is to find reasons for differences between spot prices in thetwo regions SE3 (Stockholm) and SE4 (Malmo). Regression models suchas the Generalized Linear Model, the Generalized Additive Model andThe Auto Regressive Integrated Moving Average model with Explana-tory are used to predict spot prices and investigate factors which impacton the prices.

Values of ”electricity consumption” (orange line) and ”electricity pro-duction” (blue line) of SE3 doesn’t differ so much as ”electricity con-sumption” (light blue line) and ”electricity production” (pink line) ofSE4 as shown in Figure 5. Apparently, ”electricity consumption” and”electricity production” have a great impact on the spot price formation.

Figure 5: Electricity consumption and production of SE3/SE4

Furthermore, export/import of electricity probably have significantinfluence on the SE3 spot price. Region SE4 has experienced a shortageof electricity as shown in Figure 5.

3.1 Data and notation

Bixia provided part of the data on energy consumption, production andspot prices for the regions SE3 and SE4. Data on energy flow, capacity,and wind power were derived from the site Nord Pool Spot [18]. Datainclude:

• hourly recordings of the spot prices of SE3 and SE4 (EUR);

30

Page 31: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

• hourly recordings of capacities of SE3 and SE4 (MWh);

• hourly recordings of flows to and out of SE3, to and out of SE4(MWh);

• daily recordings of consumption of SE3 and SE4 (MWh);

• daily recordings of production of SE3 and SE4 (MWh).

Wind power production of the Denmark areas (DK1, DK2) was used,because there are no free data of Swedish wind production. Data ofDK1 were used as data for SE3 and data of DK2 were used as data forSE4. The wind production was daily recorded.

Data were measured from November 1st, 2011 to December 31th,2012 for both regions.

All data were in .xls format (Microsoft Excel).

3.2 Data preprocessing

The R Project for Statistical Computing was used for modeling. Thereare methods for importing data of .xls format to R, but it causes a lotof errors. Therefore data were converted in .csv format from MicrosoftExcel files and then imported from .csv files to R using the standardprocedure. One example for implementing spot prices of SE3 and SE4into R is:

1 # s e t v a r i a b l e for data2 data<−03 # write data from Spotpr ices . csv to v a r i a b l e ” data ”4 data<− read . csv ( ” Spotpr ices . csv ” , header =FALSE ,5 sep = ” ; ” , dec = ” . ” , as . i s =T)6 # def ine length of the ” data ” ( number of rows )7 N<− length ( data [ , 1 ] )8 # s e t matrix for spot p r i c e s and f i l l up i t by data9 s p o t p r i c e s<−matrix ( data = NA, nrow = N, ncol = 2)

10 s p o t p r i c e s [ 1 :N, 1 ]<−as . double ( data [ 1 :N, 2 ] )11 s p o t p r i c e s [ 1 :N, 2 ]<−as . double ( data [ 1 :N, 3 ] )

3.2.1 Spot prices

Daily records of data were used for the analysis. Daily spot prices aredefined by:

priceSE3d,mean =

24∑h=1

priceSE3d,h

24,

31

Page 32: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

priceSE4d,mean =

24∑h=1

priceSE4d,h

24,

where priced,h is the spot price value for every hour h on day d (whered is a serial number of the day in the datasets), priced,mean is an averagedaily spot price value of the SE3 and SE4 regions, respectively.

3.2.2 Flow and capacity

The flow to SE3 describes the quantity of electricity that comes to SE3from neighboring areas.

The flow from SE3 describes the quantity of electricity that comes tothe neighboring areas from SE3. Similarly for the SE4 area.For SE3 the neighboring regions are Sweden (SE2, SE4), Norway (NO1),Denmark (DK1), Finland (FI).

For SE4 the neighboring regions are Sweden (SE3), Denmark (DK2),Poland (PL).

Figure 6: Electricity flows from/to SE3/SE4

Figure 7: Illustration of the Nord Pool tariff system

The Nord Pool tariff system is illustrated in Figure 7. The aquamarine

32

Page 33: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

color in the vessels illustrates electrical power and the walls of the tanksillustrate the transmission grid. The power generators pay a fee to thegrid owner for every kWh, which they infuse into the grid. By powergenerators are meant power producers. Accordingly, end-consumerspay for every kWh received from the grid. In addition, kilowatt-hoursare also free to be sold/bought in all the regions of the Nord Pool market[19].

This means that a retailer in Southern Sweden may buy electricityfrom a northern Sweden generator. Of course, such a transaction willnot lead to the fact that the electricity goes all the way from North toSouth of Sweden. The principle is that for every hour of work in a cer-tain place the manufacturer must pour infuse to the power system theamount of electricity which corresponds to the amount received fromthe system by the consumers of the retailer.

In addition, there is one more factor, which is called trading capacity.The trading capacity is an upper limitation of the flow. For instance,the sum of import from/export to SE3 equals the physical flow on theinterconnectors SE3-SE2, SE3-SE4, SE3-NO1, SE3-DK1, SE3-FI. The im-port/export cannot be higher than the capacity limitation. Hence theflow to/from SE3 on each of the interconnector can be restricted by thecapacity on each single interconnector.

Our approach is that regional price differences can be explained withthe help of capacity and flow investigation. Flow and capacity are mea-sured in MWh.

Hourly values of the flow and capacity for the SE3 and SE4 regionswhich are connected with neighboring price regions are obtained.

Let f lowto SE3h and capto SE3

h denote, the hourly flow and capacity fromneighboring areas to SE3 respectively. Similarly, f lowto SE4

h and capto SE4h

denote, respectively the hourly flow and capacity from neighboring ar-eas to SE4. Analogously, f lowf rom SE3

h and capf rom SE3h denote the hourly

flow and capacity from SE3 to neighboring areas. Similarly f lowf rom SE4h

and capf rom SE4h denote the hourly flow and capacity from SE4 to neigh-

boring areas.

Define f lowto SE3h , f lowf rom SE3

h , f lowto SE4h , f lowf rom SE4

h as

f lowto SE3h = f lowto SE3

h,SE2 + f lowto SE3h,SE4 +

+ f lowto SE3h,NO1 + f lowto SE3

h,DK1 + f lowto SE3h,FI ,

where

33

Page 34: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

• f lowto SE3h,SE2 is the hourly flow from SE2 to SE3;

• f lowto SE3h,SE4 is the hourly flow from SE4 to SE3;

• f lowto SE3h,NO1 is the hourly flow from NO1 to SE3;

• f lowto SE3h,DK1 is the hourly flow from DK1 to SE3;

• f lowto SE3h,FI is the hourly flow from FI to SE3,

f lowf rom SE3h = f lowf rom SE3

h,SE2 + f lowf rom SE3h,SE4 +

+ f lowf rom SE3h,NO1 + f lowf rom SE3

h,DK1 + f lowf rom SE3h,FI ,

where

• f lowf rom SE3h,SE2 is the hourly flow from SE3 to SE2;

• f lowf rom SE3h,SE4 is the hourly flow from SE3 to SE4;

• f lowf rom SE3h,NO1 is the hourly flow from SE3 to NO1;

• f lowf rom SE3h,DK1 is the hourly flow from SE3 to DK1;

• f lowf rom SE3h,FI is the hourly flow from SE3 to FI,

f lowto SE4h = f lowto SE4

h,SE3 + f lowto SE4h,DK2 + f lowto SE4

h,P L ,

where

• f lowto SE4h,SE3 is the hourly flow from SE3 to SE4;

• f lowto SE4h,DK2 is the hourly flow from SE2 to SE4;

• f lowto SE4h,P L is the hourly flow from NO1 to SE4,

f lowf rom SE4h = f lowf rom SE4

h,SE3 + f lowf rom SE4h,DK2 + f lowf rom SE4

h,P L ,

where

• f lowf rom SE4h,SE3 is the hourly flow from SE4 to SE3;

• f lowf rom SE4h,DK2 is the hourly flow from SE4 to DK2;

• f lowf rom SE4h,P L is the hourly flow from SE4 to PL.

34

Page 35: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

The capacity measures capto SE3h , f lowf rom SE4

h , capto SE4h , capf rom SE4

h arecalculated analogously. The total hourly flow between SE3 and neigh-boring areas is:

f lowSE3h = f lowf rom SE3

h − f lowto SE3h .

A negative total flow (f lowSE3h ) indicates a net import from the neigh-

boring areas to area SE3. A positive total flow (f lowSE3h ) indicates a net

export from SE3 to the neighboring areas. The notation for the capacityis similar.

Analogously for the SE4 area:

f lowSE4h = f lowf rom SE4

h − f lowto SE4h .

A negative total flow (f lowSE4h ) indicates a net import from the neigh-

boring areas to SE4. A positive total flow (f lowSE4h ) indicates a net export

from SE4 to neighboring areas. The notation for the capacity is similar.After that, daily values can be obtained. For this purpose the total net

hourly flow (f lowSE3nh ) and capacity to/from SE3 (capSE3

nh ) is calculated:

f lowSE3nh =

24∑h=1

(f lowf rom SE3h − f lowto SE3

h )

and

capSE3nh =

24∑h=1

(capf rom SE3h − capto SE3

h ).

Note that the left side index nh represents net hourly, while in theright hand side h is a number of hour in a day.

Now some measures of daily flow and capacity can be constructed:

f lowSE3d,mean =

24∑h=1

f lowSE3nh

24,

f lowSE3d,max =max{f lowSE3

nh ,nh = 1, . . . ,24},

f lowSE3d,min =min{f lowSE3

nh ,nh = 1, . . . ,24}.

The flow measures for SE4 (f lowSE4d,mean, f low

SE4d,max, f low

SE4d,min), capacity

measures for SE3 (capSE3d,mean, cap

SE3d,max, cap

SE3d,min) and SE4 (capSE4

d,mean, capSE4d,max,

capSE4d,min) are defined analogously.

”Mean”, ”Min”, ”Max” flows (f lowto SE3d,mean, f low

to SE3d,max , f lowto SE3

d,min , f lowf rom SE3d,mean ,

f lowf rom SE3d,max , f lowf rom SE3

d,min ) values can be easily obtained.

35

Page 36: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

For example,

f lowto SE3d,mean =

24∑h=1

f lowto SE3h

24,

where

f lowto SE3h =

24∑h=1

f lowto SE324.

Analogically, can be calculated:

• f lowto SE4d,max , f lowto SE4

d,min , f lowf rom SE4d,mean , f lowf rom SE4

d,max , f lowf rom SE4d,min ;

• capto SE3d,max , capto SE3

d,min , capf rom SE3d,mean , capf rom SE3

d,max , capf rom SE3d,min ;

• capto SE4d,max , capto SE4

d,min , capf rom SE4d,mean , capf rom SE4

d,max , capf rom SE4d,min .

The amount of electricity flows and capacity between the region SE3/SE4 and the neighboring regions are represented in Figures 8, 9, 10,11. It may be noted that certain periods of time with large differencesin price are a reflection of periods with high levels of capacity and lowflow of electricity to the region SE3/SE4 and low trade capacity and highflow of electricity from the region SE3/SE4.

36

Page 37: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 8: Top: Daily total capacity SE3. Center: Daily capacity out of SE3. Bottom: Daily capacity toSE3. Explanation is given on the page 36 37

Page 38: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 9: Top: Daily total capacity SE4. Center: Daily capacity out of SE4. Bottom: Daily capacity toSE4. Explanation is given on the page 36 38

Page 39: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 10: Top: Daily total flow SE3. Center: Daily flow out of SE3. Bottom: Daily flow to SE3. Expla-nation is given on the page 36 39

Page 40: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 11: Top: Daily total flow SE4. Center: Daily flow out of SE4. Bottom: Daily flow to SE4. Expla-nation is given on the page 36 40

Page 41: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

3.2.3 Net capacity utilization

It is reasonable to believe that trading capacity is one of the importantfactors which impact on the spot price. In fact, the sum of import/ex-port SE3 is equal to the physical flow on the interconnectors SE3-SE2,SE3-SE4, SE3-NO1, SE3-DK1, SE3-FI. Similarly for SE4. The import/ex-port can not be higher than the trading capacity between each biddingregion, which is defined by the transmission system operators. Hencethe flow to/from SE3 on each of the interconnector can be limited bythe capacity on each single interconnector. Trading capacities for thenext day are published on Nord Pool Spot’s web site before 10:00 a.m.Therefore the available capacity survey can be useful.

It is good to know how export or import influence the spot price at aspecific moment of time. To obtain a value of the available capacity forthe SE3 region the net capacity utilization between the SE3 region andneighboring region called nrSE3 (which may be SE2, SE4, NO1, DK1,FI)is defined as the following variable:

InrSE3h = knrSE3max

f lowto nrSE3h

capto nrSE3h

,f low

f rom nrSE3h

capf rom nrSE3h

, (31)

where

• InrSE3h is the indicator, which shows whether the electricity export or

the electricity import is more important for price formation;

• knrSE3 = 1 if the maximum isf low

f rom nrSE3h

capf rom nrSE3h

. This means that the maxi-

mum is generated by the export of electricity;

• knrSE3 = −1 if the maximum isf lowto nrSE3

h

capto nrSE3h

. This means that the maxi-

mum is generated by the import of electricity;

• f lowto nrSE3h is the hourly flow from neighboring areas to SE3;

• capto nrSE3h is the hourly capacity limit of the flow from neighboring

areas to SE3;

• f lowf rom nrSE3h is the hourly flow from SE3 to neighboring areas;

• capf rom nrSE3h is the hourly capacity limit for the flow from SE3 to

neighboring areas.

The maximum value of the ratios is considered because the capacitycan restrict the flow.

Example.

41

Page 42: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Assume nrSE3 = SE2, f lowf rom nrSE3h = 2000, capf rom nrSE3

h = 2000,f lowto nrSE3

h = 500, capto nrSE3h = 1000.

ThenInrSE3h = knrSE3max

{ 5001000

,20002000

}.

From the previous step InrSE3h is:

InrSE3h = knrSE3max{0.5,1}.

The maximum value is equal to 1. Hence the maximum is generatedby export from SE3 to a neighboring region. Therefore knrSE3 = 1 andInrSE3h = 1. For these values the export larger influences the price than

the import. If say the flow to SE3 from SE2 would increase from 500to 550 the spot price would increase not so much. If the ratios in thebrackets are equal, the first ratio is chosen. The first ratio is the importratio. Import, after all, allows to reduce the price in the region, whichimports the electricity. The spot price is not linearly dependent on flows.

The net capacity utilization between SE4 and its neighboring areas isdefined analogously.

To obtain a daily measure of the net capacity utilization in region SE3say, with standardized weights, which are fractions of the capacity ofan hour in a specific neighboring region related to the capacity in allregions, it can be calculated as:

wnrSE3h =

w′ nrSE3h

5∑nrSE3=1

w′ nrSE3h

, (32)

where w′ nrSE3h = capto nrSE3

h + capf rom nrSE3h and similarly for SE4.

Hence, if, for example, the capacity from SE3 to SE2 is 7300 and fromSE2 to SE3 is 6600, while the capacity from SE3 to SE4 is 5100 andfrom SE4 to SE3 is 2000, the highest weight is given by the SE3-SE2connection. The hourly net utilizations is defined as (31,32):

netnrSE3h =

5∑nrSE3=1

InrSE3h wnrSE3

h , (33)

netnrSE4h =

3∑nrSE4=1

InrSE4h wnrSE4

h . (34)

A daily measure of the net capacity utilization for SE3 is:

netmean SE3d =

∑h

nnrSE3h /24

42

Page 43: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

andnetmedian SE3

d =median{nnrSE3h ,h = 1, ...,24}.

Similarly, the daily measure of the net capacity utilization for SE4 canbe calculated with the help of the formula (34).

3.3 Factors

Our discussion will center on a response variable, which is the spotprice, and supposedly a set of predictors such as:

• capacity;

• flow;

• electricity consumption;

• electricity production;

• wind power production;

• net capacity utilization.

Below is a list of possible predictors for the SE3 area.The spot price will be included to the model as the variable priced,mean

(an average daily spot price value).There are several values of the ”capacity” which can be included to

the model (MWh):

• capto SE3d,mean, daily average capacity if the electricity transfers from neigh-

boring areas to SE3;

• capto SE3d,max , daily maximum capacity if the electricity transfers from

neighboring areas to SE3;

• capto SE3d,min , daily minimum capacity if the electricity transfers from

neighboring areas to SE3;

• capf rom SE3d,mean , daily average capacity if the electricity transfers from

SE3 to neighboring areas;

• capf rom SE3d,max , daily maximum capacity if the electricity transfers from

SE3 to neighboring areas;

• capf rom SE3d,min , daily maximum capacity if the electricity transfers from

SE3 to neighboring areas;

• capSE3d,mean, daily average capacity;

• capSE3d,max, daily maximum capacity;

43

Page 44: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

• capSE3d,min, daily minimum capacity.

Furthermore, there are several values of the ”flow” which can be in-cluded to the model (MWh):

• f lowto SE3d,mean, daily average flow if the electricity transfers from neigh-

boring areas to SE3;

• f lowto SE3d,max , daily maximum flow if the electricity transfers from neigh-

boring areas to SE3;

• f lowto SE3d,min , daily minimum flow if the electricity transfers from neigh-

boring areas to SE3;

• f lowf rom SE3d,mean , daily average flow if the electricity transfers from SE3

to neighboring areas;

• f lowf rom SE3d,max , daily maximum flow if the electricity transfers from

SE3 to neighboring areas;

• f lowf rom SE3d,min , daily maximum flow if the electricity transfers from

SE3 to neighboring areas;

• f lowSE3d,mean, daily average flow;

• f lowSE3d,max, daily maximum flow;

• f lowSE3d,min, daily minimum flow.

Moreover, variables consumSE3d (daily value of the electricity ”con-

sumption” in SE3, MWh), prodSE3d (daily value of the electricity ”pro-

duction” in SE3, MWh) and windprodSE3d (”wind power production”, %

of total production) can be also added to the model.”Net capacity utilization” can be represented in the model by:

• netmean SE3d , average daily value of the net capacity utilization,

• netmedian SE3d , median daily value of the net capacity utilization.

For the SE4 region the possible predictors are instead indexed by SE4.Obviously, it is not wise to add all variables to model at the same time

because of multicollinearity appearance.

3.4 Modeling

3.4.1 The Generalized Linear Model

The GLM is implemented in R by the function glm() [20] (see page 11).The function summary() [21] allows to represent statistical results of the

44

Page 45: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

modeling. The function calc.relimp() [[22], [23]] allows to find out sharesof influence from the predictors on the response variable. The functionpredict() [24] permits to build forecasts for the spot price.

Example of implementation for SE3 (with windprodSE3d , consumSE3

d aspredictors) is:

1 # pr ice modeling with the help of GLM2 glm f i t <−glm ( avgdayprice ˜ wind SE3+consumption SE3 ,

family=gaussian ( ) , data=d )3 summary ( glm f i t )4 # r e s u l t s of summary ( glm f i t )5 Formula :6 avgdayprice ˜ wind SE3 + consumption SE37

8 #determine the proportion of inf luence f a c t o r s onthe dependent v a r i a b l e

9 c a l c . relimp ( glm f i t , type = c ( ”lmg” ) , r e l a = TRUE)10 # r e s u l t of c a l c . relimp ( glm f i t , type = c (” lmg ” ) ,

r e l a = TRUE)11 Response v a r i a b l e : avgdayprice12

13 2 Regressors :14 wind SE3 consumption SE315 Metrics are normalized to sum to 100% ( r e l a=TRUE) .16

17 R e l a t i v e importance metr ics :18 wind SE3 0.0395283919 consumption SE3 0.9604716120

21 # pr ice prognosis22 pred glm<−pred ic t ( glm f i t , type=” response ” ) #value

3.4.2 The Generalized Additive Model

The GAM is implemented in R by the function gam()(see page 23) [[5],[25], [26]]. The function summary() [21] allows to represent statisticalresults of the modeling. The function calc.relimp() [[22], [23]] allows tofind out shares of influence from the predictors on the response variable.The function predict() [24] permits to build forecasts for the spot price.

1 # pr ice modeling with the help of GLM

45

Page 46: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

2 gam f i t <−gam( avgdayprice ˜ s ( wind SE3 )+s ( consumptionSE3 ) , family=gaussian ( ) , o f f s e t =NULL, method=”ML” ,data=d )

3 summary (gam f i t )4 # r e s u l t s of summary (gam f i t )5 Formula :6 avgdayprice ˜ s ( wind SE3 ) + s ( consumption SE3 )7 R−sq . ( adj ) = 0.6868 #determine the proportion of inf luence f a c t o r s on

the dependent v a r i a b l e9 c a l c . relimp (gam f i t , type = c ( ”lmg” ) , r e l a = TRUE)

10 # r e s u l t of c a l c . relimp (gam f i t , type = c (” lmg ” ) ,r e l a = TRUE)

11 Response v a r i a b l e : avgdayprice12

13 2 Regressors :14 wind SE3 consumption SE315 Metrics are normalized to sum to 100% ( r e l a=TRUE) .16

17 R e l a t i v e importance metr ics :18 wind SE3 0.0395283919 consumption SE3 0.9604716120

21 # pr ice prognosis22 pred gam<−pred ic t (gam f i t , type=” response ” ) #value

Models were realized with different sets of predictors. During theinvestigation it was revealed that the factors such as:

• capto SE3d,mean, cap

to SE3d,max , capto SE3

d,min ;

• capf rom SE3d,mean , capf rom SE3

d,max capf rom SE3d,min ;

• capSE3d,mean, cap

SE3d,max, cap

SE3d,min

are significantly correlated (see Appendix 7.2, Tables of correlation co-efficients r, Tables 7, 8, 9).

Analogically, factors:

• f lowto SE3d,mean, f low

to SE3d,max , f lowto SE3

d,min ;

• f lowf rom SE3d,mean , f lowf rom SE3

d,max , f lowf rom SE3d,min ;

• f lowSE3d,mean, f low

SE3d,max, f low

SE3d,min

46

Page 47: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

are significantly correlated as well (see Appendix 7.2, Tables of correla-tion coefficients r, Tables 10, 11, 12).

Furthermore, it was noted, the efficiency of the ”mean” and ”max”variables in the model are quite similar, while models that include ”min”variables don’t provide good statistical results (larger p-values for theexplaining variables indicating non-significance, and smaller R2). More-over, import/export flows have big impacts on the electricity spot pricein SE3/SE4.

Including the variable ”net utilization” to the model it was found outthat netmedian SE3

d provides a better fit than netmean SE3d .

We construct the models below for the evaluation.

The GLM for the SE3 area:1. yd = β0 + β1windprod

SE3d + β2consum

SE3d + εd ;

2. yd = β0 + β1windprodSE3d + β2consum

SE3d + β3cap

to SE3d,max + εd ;

3. yd = β0 + β1windprodSE3d + β2consum

SE3d + β3net

median SE3d + εd ;

4. yd = β0 + β1consumSE3d + β2cap

to SE3d,max + β3prod

SE3d + εd ;

5. yd = β0 + β1windprodSE3d + β2consum

SE3d + β3net

median SE3d + β4prod

SE3d + εd ;

6. yd = β0 + β1windprodSE3d + β2consum

SE3d + β3net

median SE3d + β4f low

f rom SE3d,max + εd ;

7. yd = β0 + β1windprodSE3d + β2consum

SE3d + β3net

median SE3d + β4f low

to SE3d,max + εd ;

8. yd = β0 + β1windprodSE3d + β2consum

SE3d + β3cap

f rom SE3d,max + β4prod

SE3d + εd ;

9. yd = β0+β1windprodSE3d +β2consum

SE3d +β3cap

f rom SE3d,max +β4net

median SE3d +β5prod

SE3d +β6f low

f rom SE3d,max +εd .

The GAM for the SE3 area:1. yd = s1(windprodSE3

d ) + s2(consumSE3d ) + εd ;

2. yd = s1(windprodSE3d ) + s2(consumSE3

d ) + s3(capto SE3d,max ) + εd ;

3. yd = s1(windprodSE3d ) + s2(consumSE3

d ) + s3(netmedian SE3d ) + εd ;

4. yd = s1(consumSE3d ) + s2(capto SE3

d,max ) + s3(prodSE3d ) + εd ;

5. yd = s1(windprodSE3d ) + s2(consumSE3

d ) + s3(netmedian SE3d ) + s4(prodSE3

d ) + εd ;

6. yd = s1(windprodSE3d ) + s2(consumSE3

d ) + s3(netmedian SE3d ) + s4(f low

f rom SE3d,max ) + εd ;

7. yd = s1(windprodSE3d ) + s2(consumSE3

d ) + s3(netmedian SE3d ) + s4(f lowto SE3

d,max ) + εd ;

8. yd = s1(windprodSE3d ) + s2(consumSE3

d ) + s3(capf rom SE3d,max ) + s4(prodSE3

d ) + εd ;

9. yd = s1(windprodSE3d )+s2(consumSE3

d )+s3(capf rom SE3d,max )+s4(netmedian SE3

d )+s5(prodSE3d )+s6(f low

f rom SE3d,max )+

+εd .

The GLM for the SE4 area:1. yd = β0 + β1windprod

SE4d + β2consum

SE4d + εd ;

2. yd = β0 + β1windprodSE4d + β2consum

SE4d + β3cap

to SE4d,max + εd ;

3. yd = β0 + β1windprodSE4d + β2consum

SE4d + β3net

median SE4d + εd ;

4. yd = β0 + β1consumSE4d + β2cap

to SE4d,max + β3prod

SE4d + εd ;

5. yd = β0 + β1windprodSE4d + β2consum

SE4d + β3net

median SE4d + β4prod

SE4d + εd ;

6. yd = β0 + β1windprodSE4d + β2consum

SE4d + β3net

median SE4d + β4f low

f rom SE4d,max + εd ;

7. yd = β0 + β1windprodSE4d + β2consum

SE4d + β3net

median SE4d + β4f low

to SE4d,max + εd ;

47

Page 48: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

8. yd = β0 + β1windprodSE4d + β2consum

SE4d + β3cap

f rom SE4d,max + β4prod

SE4d + εd ;

9. yd = β0+β1windprodSE4d +β2consum

SE4d +β3cap

f rom SE4d,max +β4net

median SE4d +β5prod

SE4d +β6f low

f rom SE4d,max +εd .

The GAM for the SE4 area:1. yd = s1(windprodSE4

d ) + s2(consumSE4d ) + εd ;

2. yd = s1(windprodSE4d ) + s2(consumSE4

d ) + s3(capto SE4d,max ) + εd ;

3. yd = s1(windprodSE4d ) + s2(consumSE4

d ) + s3(netmedian SE4d ) + εd ;

4. yd = s1(consumSE4d ) + s2(capto SE4

d,max ) + s3(prodSE4d ) + εd ;

5. yd = s1(windprodSE4d ) + s2(consumSE4

d ) + s3(netmedian SE4d ) + s4(prodSE4

d ) + εd ;

6. yd = s1(windprodSE4d ) + s2(consumSE4

d ) + s3(netmedian SE4d ) + s4(f low

f rom SE4d,max ) + εd ;

7. yd = s1(windprodSE4d ) + s2(consumSE4

d ) + s3(netmedian SE4d ) + s4(f lowto SE4

d,max ) + εd ;

8. yd = s1(windprodSE4d ) + s2(consumSE4

d ) + s3(capf rom SE4d,max ) + s4(prodSE4

d ) + εd ;

9. yd = s1(windprodSE4d )+s2(consumSE4

d )+s3(capf rom SE4d,max )+s4(netmedian SE4

d )+s5(prodSE4d )+s6(f low

f rom SE4d,max )+

+εd .

Additionally, multicollinearity test was implemented with the help ofthe function vif() [[27], [28]]. One example of multicollinearity pres-ence appears if variables that characterize the consumption and pro-duction are both included in the model. The value of vif exceeding 5and the corresponding p-value (see Appendix 7.3, Statistical estimatesand p-values) exceeding 0.05 are considered evidence of collinearity[28]. As can be seen from the table below, in the first case windprodSE3

d ,consumSE3

d , netmedian SE3d and prodSE3

d are checked with the help of vif().There are values of vif which are larger than 5 and p-value larger than0.05. That proves multicollinearity in this case. Then we exclude prodSE3

dand the remaning predictors values are in the absence of the multi-collinearity.

Table 1: Results of vif for the GLM model

Predictors windprodSE3d consumSE3

d netmedian SE3d prodSE3

dvif-value 1.054052 6.916041 3.982179 4.990643p-value 3.06e − 06 < 2e − 16 0.0885 3.05e − 10

vif-value 1.001299 1.431150 1.431488p-value 1.14e-07 < 2e − 16 < 2e − 16

4 The statistical estimates of the models and predictorsrelative importance values

The statistical estimates and the predictors relative importance valuesof modeling are represented in Tables 2, 3, 4, 5 below. The numbers inthe first column of the tables corresponds to the model numbers above.In the second column of the tables there are the values of the adjustedcoefficient of determination R2 (see page 22). This ratio characterizes the

48

Page 49: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

proportion of variation of the dependent variable due to the variabilityof the regression predictors.

For the adjusted R2 also the danger of taking too many predictors(multicollinearity) is considered. The p-values of the predictors can befound in Appendix 7.3, Statistical estimates and p-values.

The bigger the R2 is (closer to 1 or 100%), the smaller the total varia-tion will be and the better the regression will describe the relationshipbetween the dependent and explanatory variables. The total variationwith the number of predictors is taken into consideration.

In subsequent columns there are the R2 values expressed in percent-age, describing the importance of each factor in the model (relative im-portance value of a predictor). The larger the percentage of the predic-tor, the greater the impact of the predictor on the price.

Every row specifies the value of R2 depending on the chosen predic-tors (column headings). The chosen predictor denoted to the filled cell.

49

Page 50: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Tabl

e2:

Res

ult

sof

mod

elin

gSE

3el

ectr

icit

yp

rice

sw

ith

the

help

ofG

LM

Mod

el№

R2

windprodSE

3d

consumSE

3d

captoSE

3d,max

capfromSE

3d,max

netmedianSE

3d

prodSE

3d

flowfromSE

3d,max

flowtoSE

3d,max

mu

ltic

olli

near

ity

157

,91

3,95

96,0

5-

--

--

-no

259

,03

3,89

92,3

93,

71-

--

--

no3

61,1

23,

6756

,94

--

39,3

7-

--

no4

68,4

5-

86,0

32,

05-

-11

,92

--

yes

570

3,18

52,8

4-

-34

,66

9,3

--

yes

669

,42

552

,5-

-31

,34

-11

,1-

no7

67,1

33,

147

,5-

-31

,51

--

17,8

no8

65,8

12,

9382

,04

-4,

75-

10,2

8-

-ye

s9

67,9

14,

247

,96

-2,

4527

,67

8,2

9,34

-ye

s

Tabl

e3:

Res

ult

sof

mod

elin

gSE

3el

ectr

icit

yp

rice

sw

ith

the

help

ofG

AM

Mod

el№

R2

windprodSE

3d

consumSE

3d

captoSE

3d,max

capfromSE

3d,max

netmedianSE

3d

prodSE

3d

flowfromSE

3d,max

flowtoSE

3d,max

mu

ltic

olli

near

ity

168

,63,

9596

,05

--

--

--

no2

71,6

3,89

92,3

93,

71-

--

--

no3

78,3

3,67

56,9

4-

-39

,37

--

-no

477

,5-

86,0

32,

05-

-11

,92

--

yes

579

,53,

1852

,84

--

34,6

69,

3-

-ye

s6

79,8

552

,5-

-31

,34

-11

,1-

no7

793,

147

,5-

-31

,51

--

17,8

no8

77,2

2,93

82,0

4-

4,75

-10

,28

--

yes

980

,44,

247

,96

-2,

4527

,67

8,2

9,34

-ye

s

As

can

bese

enfr

omTa

bles

2,3

win

dp

ower

pro

duct

ion,

cons

um

pti

on,

net

cap

acit

yu

tili

zati

on,

pro

duct

ion

and

flow

from

SE3

to

neig

hbor

ing

area

sha

veth

ela

rges

tin

flu

ence

onth

esp

otp

rice

form

atio

nif

we

useR

2an

da

mu

ltic

olli

near

ity

ind

icat

oras

am

easu

reof

mod

elp

erfo

rman

ce.

The

imp

act

ofp

rodu

ctio

nis

smal

ler

than

cons

um

pti

on.

Inad

dit

ion,

pro

duct

ion

inbo

thm

odel

s(t

heG

LM

and

the

GA

M)

lead

sto

mu

ltic

olli

near

ity.

Mor

eove

r,th

em

odel

wit

hth

ehi

ghes

tad

just

edco

effici

ent

ofd

eter

min

atio

nR

2is

inth

e6t

hro

w(y

ello

w

50

Page 51: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

colo

r)in

both

case

s(t

heG

LM

fore

cast

ing

and

the

GA

Mfo

reca

stin

g).

Tabl

e4:

Res

ult

sof

mod

elin

gSE

4el

ectr

icit

yp

rice

wit

hth

ehe

lpof

GL

M

Mod

el№

R2

windprodSE

4d

consumSE

4d

captoSE

4d,max

capfromSE

4d,max

netmedianSE

4d

prodSE

4d

flowfromSE

4d,max

flowtoSE

4d,max

mu

ltic

olli

near

ity

150

,61

9,21

90,7

9-

--

--

-no

264

,28

7,42

78,9

213

,65

--

--

-no

363

,19

5,06

56,5

5-

-38

,38

--

-no

466

,07

-70

,77

13,8

9-

-15

,34

--

yes

565

,75,

5447

,79

--

34,1

512

,49

--

yes

663

,36,

7149

,3-

-29

,1-

14,8

-no

765

,75

6,24

56,1

--

32,6

9-

-4,

8no

852

,95

9,41

71,3

9-

0,44

-18

,76

--

yes

967

6,27

43-

1,36

25,0

710

,813

,46

-ye

s

Tabl

e5:

Res

ult

sof

mod

elin

gSE

4el

ectr

icit

yp

rice

wit

hth

ehe

lpof

GA

M

Mod

el№

R2

windprodSE

4d

consumSE

4d

captoSE

4d,max

capfromSE

4d,max

netmedianSE

4d

prodSE

4d

flowfromSE

4d,max

flowtoSE

4d,max

mu

ltic

olli

near

ity

161

9,21

90,7

9-

--

--

-no

274

,37,

4278

,92

13,6

5-

--

--

no3

62,5

5,06

56,5

5-

-38

,38

--

-no

473

,4-

70,7

713

,89

--

15,3

4-

-ye

s5

73,9

5,54

47,7

9-

-34

,15

12,4

9-

-ye

s6

73,8

6,71

49,3

--

29,1

-14

,8-

no7

77,1

6,24

56,1

--

32,6

9-

-4,

8no

861

9,41

71,3

9-

0,44

-18

,76

--

yes

975

6,27

43-

1,36

25,0

710

,813

,46

-ye

s

As

can

bese

enfr

omTa

bles

4,5

win

dp

ower

pro

duct

ion,

cons

um

pti

on,

net

cap

acit

yu

tili

zati

on,

pro

duct

ion

and

flow

toSE

4fr

omne

ighb

orin

gar

eas

have

the

larg

est

infl

uen

ceon

the

spot

pri

cefo

rmat

ion

ifw

eu

seR

2an

da

mu

ltic

olli

near

ity

ind

icat

oras

am

easu

reof

mod

elp

erfo

rman

ce.

Situ

atio

nw

ith

pro

duct

ion

issi

mil

arto

the

pre

viou

sar

ea(S

E3)

.T

hem

odel

wit

hth

ehi

ghes

tad

just

edco

effici

ent

ofd

eter

min

atio

nR

2is

inth

e7t

hro

w(y

ello

wco

lor)

inbo

thca

ses

(the

GL

Mfo

reca

stin

gan

dth

eG

AM

fore

cast

ing)

.

51

Page 52: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Here are considered just R2 and a multicollinearity indicator as mea-sure for the model performance. But there are other ways to measureperformance of the GLM and the GAM. The p-values for the predictorsare shown in Appendix 7.3. They don’t contradict the use of R2 as ameasure for model performance.

4.1 Comparison of the models

Here the GLM and the GAM are compared. The models with the highestadjusted coefficient of determination R2 in the absence of multicollinear-ity can be seen in Table 2 and Table 3. For SE3 it is a model whichincludes the following factors: windprodSE3

d , netmedian SE3d , f lowf rom SE3

d,max ,and consumSE3

d . For prediction with these factors see the graphical fore-cast below: Figure 12 and Figure 13.

Figure 12: Spot price prognosis obtained from January 1st 2013 by the GLM

Figure 13: Spot price prognosis obtained from January 1st 2013 by the GAM

52

Page 53: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

For SE4 it is a model which includes the following factors: windprodSE4d ,

netmedian SE4d , f lowto SE4

d,max , and consumSE4d (see graphical forecast below:

Figure 14 and Figure 15).

Figure 14: Spot price prognosis obtained from January 1st 2013 by the GLM

Figure 15: Spot price prognosis obtained from January 1st 2013 by the GAM

The real spot price values from November 1st, 2011 until December31st, 2012 (light blue line) and from January 1st, 2013 until May 6th,2013 (blue line), spot price forecasts from the GLM from January 1st,2013 until May 6th, 2013 (orange line), spot price forecast accordinglythe GAM from January 1st, 2013 until May 6th, 2013 (pink line) for bothregions are shown in Figures 12, 13, 14, 15.

It should be noted that both models seem to be in good agreementwith the real data and give a reliable prediction. Note that for real data,there are outliers, which were caused by certain incidents. For example,November 7th, 2011 there was an increase in spot prices, after a roguemagpie had flown to a relay plant which caught on fire [29]. The nextmajor release occurred in early February 2012, as a result of the decom-

53

Page 54: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

missioning of a nuclear power unit. Such incidents are hard to model.The GLM showed greater resistance to outliers of the response vari-

able. In addition, if the GLM is used, then long-term forecasts are alsomore accurate.

4.1.1 The Auto Regressive Integrated Moving Average model withExplanatory

Recall that it is possible to include the multivariate linear regression toARIMA, which reveals the ARIMAX modeling (see page 27). The func-tion auto.arima() implements the linear regression in R, where residu-als are ARIMA modeled.

The ARIMAX model has the following form (see (24)):

yi = C + ν(B)xi +Ni ,

and with notation as in (25)

ν(B)xi = (ν0 + ν1B+ ν2B2 + · · · )xi = ν0xi + ν1xi−1 + ν2xi−2 + · · · ,

where

xi = x{4,n} =

windprodSE3

1 · · · windprodSE3n

netmedian SE31 · · · netmedian SE3

n

f lowf rom SE31,max · · · f lowf rom SE3

n,max

consumSE31 · · · consumSE3

n

.The ARIMAX model was used for the spot price forecasting after the

GLM and the GAM analysis. Factors of the models with the highest ad-justed coefficients of determination give the best results in the ARIMAXmodels as well as in the GLM and the GAM models. Therefore it canbe concluded that the predictors were chosen properly. The graphicalforecast of the ARIMAX models for SE3 and SE4 is shown in Figures16, 17.

The AIC (see page 29) can show the quality of the model in compar-ison to other models. The smaller the AIC coefficient, the better themodel fits to original data. Accordingly to the Table 6 the minimal AICcorresponds to the ARIMAX model, which consequently is to be pre-ferred.

Table 6: The ARIMAX model results in comparison with the GLM and the GAM results

AIC GLM GAM ARIMAXAIC SE3 2672,145 2839,1 2488,25AIC SE4 2834,691 2965,9 2618,18

54

Page 55: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 16: Spot price prognosis for SE3 is obtained by GLM, GAM and ARIMAX

Figure 17: Spot price prognosis for SE4 is obtained by GLM, GAM and ARIMAX

4.2 Analysis of predictors

The Nord Pool Spot market as any other spot market carries out the keytask of balancing supply and demand in the power market with a certainscope for forward planning.

The spot market receives bids and offers from producers and con-sumers. The system calculates the spot price which balances surplusand deficit areas. If power flows between bidding areas exceed the trad-ing capacity; area prices are calculated.

For price areas with isolated price calculation we will set as follows. Ifthe price increases in a deficit area, electricity market members will pur-

55

Page 56: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 18: The Nordic area price calculation principles

chase less and sell more (see Figure 18, right). Lower price in a surplusarea will cause less sale and more purchase (see Figure 18, left).This isthe situation for price areas with isolated price calculation. However, inthe Nordic price areas the area price calculation is repeated before thecapacity between the high price area and the low price area is utilizedto the maximum.

A volume which corresponds to the trading capacity is added as pur-chase in the surplus area and sale in the deficit area. Hence, the addi-tional sale (import flow) leads to a parallel shift of the supply curve (forthe deficit area) and additional purchase (export flow) leads to a parallelshift of the demand curve (for the surplus area).

The new equilibrium points will be obtained after the addition of ex-port/import flows. These points represent new area prices (new balanceprices). Shifts of the supply and the demand curves lead to small in-creasing of the price in the surplus area and small decreasing in thedeficit area. But despite this changes, the price is lower in the surplusarea and higher in the deficit area.

The area price in the spot market are determined by the balance ofsupply and demand and this balance is only affected by changes in thisrelationship. Hence, supply and demand impact strongly on price for-mation.

Production includes power which is taken not only from nuclear sta-tions, but also from renewable sources such as solar, wind, biomass andsmall hydro power stations. In turn, the supply of power involves notonly production, but also export/import flows of electricity.

In this thesis electricity production jointly with flows and electric-ity consumption corresponds relatively to supply and demand. Valuesof ”electricity consumption” (orange line) and ”electricity production”

56

Page 57: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 19: The supply and the demand are comparable because of adding export/import flows to thesupply data

(blue line) of SE3 don’t differ so much as ”electricity consumption” (lightblue line) and ”electricity production” (pink line) of SE4 as shown in Fig-ure 19. This means deficit of supply in the region SE4 and high price forthis region.

It is evident from Tables 2, 3, 4 and 5 that electricity consumption(about 50%), electricity production (about 10%) jointly with import/ex-port flows (about 10%) as well as net capacity utilization (about 30%)have a great affect the spot price formation.

If flows are added to supply data for the SE3 and the SE4 regions, thenthe demand (consumption) will be comparable with the supply (produc-tion and export/import flows) and prices will be recalculated (see Figure19). But any way the price in the SE3 region will be lower than in theSE4 region, because the shortage of electricity in the SE3 is less than inthe SE4.

Hence export/import of electricity (f lowto SE3d,max , f lowto SE3

d,max ) have signif-icant influence on the SE3 spot price construction. Moreover, import ofelectricity (f lowto SE4

d,max ) has a large impact on the SE4 spot price, becauseregion SE4 experiences a shortage of electricity. In addition, the net ca-pacity median (about 30%) has a large impact on SE3 and SE4 prices aswell, because this dimension represents principles of restrictions on thedistribution of electricity.

It should be noted from Tables 4, 3, 4 and 5 that the production ofelectricity by wind generators also influences the spot price formation.In region SE3 in comparison with SE4 this influence is less, because theshare of wind generators of the total electricity production is slightly

57

Page 58: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

lower.

5 User interface

5.1 Additional modules. Implementation.

User interface is the intermediate step between the user and a computerprogram. A menu-driven interface is one in which the user can selectcommand choices from menus displayed on the screen. The user inter-face is done for the purpose of making the source code of the researcheasy to use. There is a set of packages for creation of a user interface inR:

• RGtk2[30];

• cairoDevice[31];

• gWidgets[32];

• gWidgetsRGtk2[33].

The following functions:

• gwindow() (creating a container (window) for other elements);

• gframe() (creating a field for output of results or for choosing ofsource data);

• gradio() (creating radio buttons for choosing mathematical model);

• gcheckboxgroup() (creating check-box list of predictors,which canbe included in the model);

• ggraphics() (creating a field for output results graphically)

that are applied for building the user interface [34], [35], [36].Below the user interface creation procedure through the example of

SE3 is considered:

1 # s e t p o s s i b l e values of the mathematical model2 availModel<−c (GAM=”gam” ,GLM=”glm” )3 # s e t p o s s i b l e values of f a c t o r s4 ava i lVars <− c ( consumption=”consumption SE3” , wind=”

wind SE3” ,5 production=” production SE3” , capac i ty max to=”

capac i ty max d to SE3” , capac i ty max out=” capac i tymax d out SE3” , net capac i ty median=” net capac i tymedian SE3 d” , flow max out=” flow max d out SE3” ,

flow max to=” flow max d to SE3” )

58

Page 59: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

6 # s e t types of data , which w i l l be in a user form7 model <− gradio ( names ( availModel ) , h o r i z o n t a l=FALSE ,

handler=updatePlot )8 vars <− gcheckboxgroup ( avai lVars , handler=updatePlot

)9 mult<−g t e x t ( m u l t i c o l l i n e a r i t y , handler=updatePlot ,

drop=TRUE)10 # i n i t i a l a z e user form with data for modeling11 window <− gwindow ( ”Modeling” )12 BigGroup <− ggroup ( cont=window )13 group <− ggroup ( h o r i z o n t a l=FALSE , conta iner=BigGroup

)14 tmp <− gframe ( ”Model” , conta iner=group )15 add ( tmp , model )16 tmp <− gframe ( ” Factors ” , conta iner=group )17 add ( tmp , vars )18 tmp<−gframe ( ” M u l t i c o l l i n e a r i t y : ” , conta iner=group )19 add ( tmp , mult )20 add ( BigGroup , ggraphics ( ) )

Then the functions that respond to user actions are called. Similarly,the user interface for SE4 can be implemented.

At the first usage the user interface looks like in the Figure (20) below:

Figure 20: User interface. First usage

Then the model and predictors can be chosen (see Figures 21, 23).Example.In Figure 21 forecasting with the help of the GAM is shown. The

user just chooses the model and predictors. Than forecasts are con-structed automatically in real time. Factors windprodSE3

d , netmedian SE3d ,

f lowf rom SE3d,min , consumSE3

d are used as predictors. The result of the spot

59

Page 60: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 21: Example of the user interaction with interface, which was developed by the authors of thisthesis. Example 1

price forecasting is represented graphically. A multicollinearity indi-cator is located under the factors. The model is in absence of multi-collinearity in this case.

Furthermore, output of statistical estimates of the model can be seenin the work space of R (see Figure 22):

Figure 22: Statistical estimates output. Example 1

Example.In Figure 23 forecasting with the help of the GAM is shown. Factors

windprodSE3d , netmedian SE3

d , f lowf rom SE3d,min , prodSE3

d , consumSE3d are used as

predictors. The result of the spot price forecasting is represented graph-ically. The model is in the presence of multicollinearity in this case.

60

Page 61: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 23: Example of the user interaction with interface, which was developed by the authors of thisthesis. Example 2

Furthermore, output of statistical estimates of the model can be seenin the work space of R (see Figure 24).

Figure 24: Statistical estimates output. Example 2

Thus, parameters (the model, factors) are chosen by the user. Theyare input data of the interface. Hence the graphical forecast of the spotprice, statistical estimates and the indicator of multicollinearity are theoutput of the user interface.

61

Page 62: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

5.2 The user interface usage.

If the reader would like to use the interface which is considered aboveon his/her own computer, he/she needs to follow the steps below:

1. Download ”The R Project for Statistical Computing” (for example,here URL: http://cran.r-project.org/bin/windows/base/);

2. Install ”The R Project for Statistical Computing”;

3. Run ”Rgui.exe” (label on the monitor will appear after installation);

4. Install and launch additional packages: mgcv, relaimpo, HH, RGtk2,cairoDevice, gWidgets and gWidgetsRGtk2 on your R environment[37];

5. Create in the folder with ”Rgui.exe” a new folder with the name”files” and put files: ”capacities SE3 SE4.csv”, ”Flow SE3 SE4.csv”,

”Production SE3 SE4.csv”, ”Spotprices.csv”, ”Spotprices 2013.csv”,

”Wind power.csv”, ”Consumption SE.csv” to this folder. Then toput the file ”.RData” (saved work space) to the same folder or copythe source code and execute it;

6. Launch the user interface for the selected region.

62

Page 63: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

6 Conclusion

The investigation illustrates general principles of the electricity trans-mission on the Nord Pool market and how capacity influences on it.It tells about spot price prediction (depending on ”electricity capac-ity”, ”electricity flow”, ”electricity consumption”, ”electricity produc-tion”, ”wind power”, ”net capacity utilization”) with help of the GLMand the GAM for SE3 and SE4 regions.

The main factors which impact on the spot prices in the two regionsare the following variables:

• consumption and production of electricity for the both regions;

• value of electricity export/import for the region SE3 (f lowto SE3d,max ,

f lowto SE3d,max );

• value of electricity import to the region SE4 from neighboring re-gions (f lowto SE4

d,max );

• wind power.

The selection of the variables were based on the R2 values and anindicator of multicollinearity. The p-values and the predictors were aswell investigated and they don’t contradict the use of R2 as a measure ofmodel selection.

If investments are made in the electricity grid and new electricity pro-duction sources (wind power stations, solar energy generators and e.t.c.)in the areas that currently experience high demand but have a low sup-ply (SE4, and SE3 sometimes), the differences in the electricity pricesbetween this areas will be reduced.

The work provides graphical forecast of spot prices for both regionswith the help of the GLM and the GAM models. If the observed datawas for a longer period, it will be possible to use another models, for in-stance, the ARIMAX model or the seasonal model SARIMA. The ARIMA(no-seasonal model) model was also applied. Reasonable results wereobtained with this model, indicating that the main factors were chosencorrectly.

In addition, the created user interface gives the ability to construct thespot price forecast and analyze factors which impact on the spot pricewith help of the GLM and the GAM.

The GLM and GAM are generally non-linear models. The adjustedcoefficient of determination R2 is developed for linear models. Thereforethere may be more natural ways to measure the performance of these

63

Page 64: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

models than just using R2 and considering p-values of the predictors,but that is beyond the scope of this thesis.

64

Page 65: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

7 Appendix

7.1 Spot price prognosis

Here spot price prognoses are obtained by the Generalized Linear Modeland the Generalized Additive Model for comparison of them. Spot priceprognoses are obtained for the region SE3 in Appendix 7.1.1, for theregion SE4 in Appendix 7.1.2. There is R code for the GLM and theGAM models in Appendix 7.2.

7.1.1 Region SE3

Figure 25: Spot price prognosis obtained by the GLM (windprodSE3d , consumSE3

d )

Figure 26: Spot price prognosis obtained by the GAM (windprodSE3d , consumSE3

d )

65

Page 66: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 27: Spot price prognosis obtained by the GLM (windprodSE3d , capto SE3

d,max , consumSE3d )

Figure 28: Spot price prognosis obtained by the GAM (windprodSE3d , capto SE3

d,max , consumSE3d )

66

Page 67: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 29: Spot price prognosis obtained by the GLM (windprodSE3d , netmedian SE3

d , consumSE3d )

Figure 30: Spot price prognosis obtained by the GAM (windprodSE3d , netmedian SE3

d , consumSE3d )

67

Page 68: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 31: Spot price prognosis obtained by the GLM (consumSE3d , capto SE3

d,max , prodSE3d )

Figure 32: Spot price prognosis obtained by the GAM (consumSE3d , capto SE3

d,max , prodSE3d )

68

Page 69: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 33: Spot price prognosis obtained by the GLM (windprodSE3d , netmedian SE3

d , prodSE3d , consumSE3

d )

Figure 34: Spot price prognosis obtained by the GAM (windprodSE3d , netmedian SE3

d , prodSE3d , consumSE3

d )

69

Page 70: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 35: Spot price prognosis obtained by the GLM (windprodSE3d , netmedian SE3

d , f lowf rom SE3d,min ,

consumSE3d )

Figure 36: Spot price prognosis obtained by the GAM (windprodSE3d , netmedian SE3

d , f lowf rom SE3d,min ,

consumSE3d )

70

Page 71: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 37: Spot price prognosis obtained by the GLM (windprodSE3d , netmedian SE3

d , f lowto SE3d,max ,

consumSE3d )

Figure 38: Spot price prognosis obtained by the GAM (windprodSE3d , netmedian SE3

d , f lowto SE3d,max ,

consumSE3d )

71

Page 72: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 39: Spot price prognosis obtained by the GLM (windprodSE3d , capf rom SE3

d,max , prodSE3d , consumSE3

d )

Figure 40: Spot price prognosis obtained by the GAM (windprodSE3d , capf rom SE3

d,max , prodSE3d , consumSE3

d )

72

Page 73: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 41: Spot price prognosis obtained by the GLM (windprodSE3d , netmedian SE3

d ,

capf rom SE3d,max ,f lowf rom SE3

d,min , consumSE3d )

Figure 42: Spot price prognosis obtained by the GAM (windprodSE3d , netmedian SE3

d ,

capf rom SE3d,max ,f lowf rom SE3

d,min , consumSE3d )

73

Page 74: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

7.1.2 Region SE4

Figure 43: Spot price prognosis obtained by the GLM (windprodSE4d , consumSE4

d )

Figure 44: Spot price prognosis obtained by the GAM (windprodSE4d , consumSE4

d )

74

Page 75: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 45: Spot price prognosis obtained by the GLM (windprodSE4d , capto SE4

d,max , consumSE4d )

Figure 46: Spot price prognosis obtained by the GAM (windprodSE4d , capto SE4

d,max , consumSE4d )

75

Page 76: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 47: Spot price prognosis obtained by the GLM (windprodSE4d , netmedian SE4

d , consumSE4d )

Figure 48: Spot price prognosis obtained by the GAM (windprodSE4d , netmedian SE4

d , consumSE4d )

76

Page 77: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 49: Spot price prognosis obtained by the GLM (consumSE4d , capto SE4

d,max , prodSE4d )

Figure 50: Spot price prognosis obtained by the GAM (consumSE4d , capto SE4

d,max , prodSE4d )

77

Page 78: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 51: Spot price prognosis obtained by the GLM (windprodSE4d , netmedian SE4

d , prodSE4d , consumSE4

d )

Figure 52: Spot price prognosis obtained by the GAM (windprodSE4d , netmedian SE4

d , prodSE4d , consumSE4

d )

78

Page 79: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 53: Spot price prognosis obtained by the GLM (windprodSE4d , netmedian SE4

d , f lowf rom SE4d,min ,

consumSE4d )

Figure 54: Spot price prognosis obtained by the GAM (windprodSE4d , netmedian SE4

d , f lowf rom SE4d,min ,

consumSE4d )

79

Page 80: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 55: Spot price prognosis obtained by the GLM (windprodSE4d , netmedian SE4

d , f lowto SE4d,max ,

consumSE4d )

Figure 56: Spot price prognosis obtained by the GAM (windprodSE4d , netmedian SE4

d , f lowto SE4d,max ,

consumSE4d )

80

Page 81: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 57: Spot price prognosis obtained by the GLM (windprodSE4d , capf rom SE4

d,max , prodSE4d , consumSE4

d )

Figure 58: Spot price prognosis obtained by the GAM (windprodSE4d , capf rom SE4

d,max , prodSE4d , consumSE4

d )

81

Page 82: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Figure 59: Spot price prognosis obtained by the GLM (windprodSE4d , netmedian SE4

d ,

capf rom SE4d,max ,f lowf rom SE4

d,min , consumSE4d )

Figure 60: Spot price prognosis obtained by the GAM (windprodSE4d , netmedian SE4

d ,

capf rom SE4d,max ,f lowf rom SE4

d,min , consumSE4d )

82

Page 83: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

7.2 Tables of correlation coefficients r

Definition 8. Correlation dependence between two variables is functionaldependence between conditional probability of one of them, and values of theother [11].

If we have a series of n measurements of X and Y written as xi and yi(where i = 1,2, ...,n), the linear dependence between them can be mea-sured by the correlation coefficient r:

r =xy − xysxsy

, (35)

where xy =∑ni=1 xiyin , x and y are the sample means of X and Y , sx =√

1n−1

∑(x − xi)2 and sy =

√1n−1

∑(y − yi)2 are the sample standard devi-

ations of X and Y , respectively.Values of the correlation coefficient are located in the interval [−1,1].

If the |r | is closer to 1, then the correlation dependence between the vari-ables is stronger. If the |r | > 0.5, than the correlation dependence can beconsidered to be strong. The r > 0 indicates a positive linear correlationbetween the X and the Y variables (if values for X increase, values for Yalso increase). The r < 0 indicates a negative linear correlation betweenX and Y variables (if values for X increase, values for Y decrease).

All the correlation coefficients between predictors in the tables beloware significant different from zero.

Table 7: Correlation coefficients for capto SE3d,mean, capto SE3

d,max ,

capto SE3d,min

capto SE3d,mean capto SE3

d,max capto SE3d,min

capto SE3d,mean 1 0.9360778 0.9415323

capto SE3d,max 0.9360778 1 0.8145887

capto SE3d,min 0.9415323 0.8145887 1

Table 8: Correlation coefficients for capf rom SE3d,mean , capf rom SE3

d,max ,

capf rom SE3d,min

capf rom SE3d,mean cap

f rom SE3d,max cap

f rom SE3d,min

capf rom SE3d,mean 1 0.9611835 0.9555776

capf rom SE3d,max 0.9611835 1 0.8681683

capf rom SE3d,min 0.9555776 0.8681683 1

83

Page 84: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Table 9: Correlation coefficients for capSE3d,mean, capE3

d,max,

capSE3d,min

capSE3d,mean capSE3

d,max capSE3d,min

capSE3d,mean 1 0.9003918 0.9293966

capSE3d,max 0.9003918 1 0.7444077

capSE3d,min 0.9293966 0.7444077 1

Table 10: Correlation coefficients for f lowto SE3d,mean, f lowto SE3

d,max ,

f lowto SE3d,min

f lowto SE3d,mean f lowto SE3

d,max f lowto SE3d,min

f lowto SE3d,mean 1 0.902049 0.7823122

f lowto SE3d,max 0.902049 1 0.5295992

f lowto SE3d,min 0.7823122 0.5295992 1

Table 11: Correlation coefficients for f lowf rom SE3d,mean ,

f lowf rom SE3d,max , f lowf rom SE3

d,min

f lowf rom SE3d,mean f low

f rom SE3d,max f low

f rom SE3d,min

f lowf rom SE3d,mean 1 0.8582643 0.7823122

f lowf rom SE3d,max 0.8582643 1 0.5336128

f lowf rom SE3d,min 0.7823122 0.5336128 1

Table 12: Correlation coefficients for f lowSE3d,mean, f lowSE3

d,max,

f lowSE3d,min

f lowSE3d,mean f lowSE3

d,max f lowSE3d,min

f lowSE3d,mean 1 0.9548543 0.9854118

f lowSE3d,max 0.9548543 1 0.9097277

f lowSE3d,min 0.9854118 0.9097277 1

7.3 Statistical estimates and p-values

For the GLM the p-value is computed [5]. This probability shows if thevariable relevant or not. The smaller p-value the more indication wehave that the variable is important. For example 2e-16 means that ifthe consumSE3

d -coefficient is zero than the odds that its estimate is aslarge as it happened to be or larger is 1

5000000000000000 . This tell us thatit is plausible that the consumSE3

d -coefficient is non-zero. It means thatit is an important predictor. The more punctuation there is next to thevariables, the more important is the variable. For the GLM the p−value

84

Page 85: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

is obtained from the t − distribution [5]. For the GAM the p − value isobtained from the F − distribution [5].

Modelnumber

R2 Predictors t-valueP r(> |t|)(p-value)

Significancecodes

multicollinearity

1 57,91 windprodSE3d -5.151 3.98e-07 *** no

consumSE3d 23.791 < 2e − 16 ***

2 59,03 windprodSE3d -5.240 2.54e-07 *** no

capto SE3d,max -3.553 0.000424 ***

consumSE3d 23.682 < 2e − 16 ***

3 61,12 windprodSE3d -6.051 3.18e-09 *** no

netmedian SE3d -10.945 < 2e − 16 ***

consumSE3d 16.509 < 2e − 16 ***

4 68,45 capto SE3d,max -0.405 0.686 yes

prodSE3d -12.725 < 2e − 16 ***

consumSE3d 29.138 < 2e − 16 ***

5 70 windprodSE3d -4.731 3.06e-06 *** yes

netmedian SE3d -1.707 0.0885 .

prodSE3d -6.451 3.05e-10 ***

consumSE3d 13.607 < 2e − 16 ***

6 69,42 windprodSE3d -7.638 1.49e-13 *** no

netmedian SE3d -7.160 3.61e-12 ***

f lowf rom SE3d,max -5.728 1.94e-08 ***

consumSE3d 17.448 < 2e − 16 ***

7 67,13 windprodSE3d -6.063 2.97e-09 *** no

netmedian SE3d -10.236 < 2e − 16 ***

f lowto SE3d,max -1.061 0.290

consumSE3d 15.267 < 2e − 16 ***

8 65,81 capf rom SE3d,max -3.187 0.001547 ** yes

prodSE3d -11.749 < 2e − 16 ***

windprodSE3d -4.504 8.63e-06 ***

consumSE3d 29.863 < 2e − 16 ***

9 67,91 consumSE3d 14.309 < 2e − 16 *** yes

prodSE3d -6.332 6.22e-10 ***

windprodSE3d -6.200 1.35e-09 ***

netmedian SE3d 0.099 0.9212

f lowf rom SE3d,max -5.353 1.43e-07 ***

capf rom SE3d,max -2.077 0.0384 *

Table 13: Statistical estimates of GLM for the SE3 region. Significance legend: blank for apredictor indicates that it is not important, a dot indicates that it is not so important, one starindicates importance, and more stars more importance

85

Page 86: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

To choose a model it is wise to use a model with predictors with smallp−values at least below 0.05 and also have absence of multicollinearity.Accordingly to the Table 13 the best model in the sense of smallest p −values (in absence of multicollinearity) is number 6 (yellow row). Thebest models in Tables 14, 15, 16 are emphasized by yellow color as well.

Modelnumber

R2 Predictors F p-value Significancecodes

multicollinearity

1 68,6 windprodSE3d 25.49 6.53e-07 *** no

consumSE3d 121.47 < 2e − 16 ***

2 71,6 windprodSE3d 27.024 3.11e-07 *** no

capto SE3d,max 5.868 1.03e-06 ***

consumSE3d 115.570 < 2e − 16 ***

3 78,3 windprodSE3d 29.08 1.14e-07 *** no

netmedian SE3d 31.69 < 2e − 16 ***

consumSE3d 69.70 < 2e − 16 ***

4 77,5 capto SE3d,max 4.565 7.23e-05 *** yes

prodSE3d 34.099 < 2e − 16 ***

consumSE3d 167.414 < 2e − 16 ***

5 79,5 windprodSE3d 19.522 1.26e-05 *** yes

netmedian SE3d 9.839 7.60e-10 ***

prodSE3d 9.927 6.93e-06 ***

consumSE3d 48.284 < 2e − 16 ***

6 79,8 windprodSE3d 44.540 7.47e-11 *** no

netmedian SE3d 17.307 < 2e − 16 ***

f lowf rom SE3d,max 7.355 5.75e-06 ***

consumSE3d 76.452 < 2e − 16 ***

7 79 windprodSE3d 27.343 2.67e-07 *** no

netmedian SE3d 27.301 < 2e − 16 ***

f lowto SE3d,max 2.348 0.0304 *

consumSE3d 52.930 < 2e − 16 ***

8 77,2 capf rom SE3d,max 4.331 0.0108 * yes

prodSE3d 39.405 < 2e − 16 ***

windprodSE3d 24.398 1.12e-06 ***

consumSE3d 166.921 < 2e − 16 ***

9 80,4 consumSE3d 48.119 < 2e − 16 *** yes

prodSE3d 10.673 7.80e-05 ***

windprodSE3d 32.765 1.95e-08 ***

netmedian SE3d 6.914 1.30e-06 ***

f lowf rom SE3d,max 6.063 0.000146 ***

capf rom SE3d,max 0.017 0.902475

Table 14: Statistical estimates of GAM for the SE3 region. Significance legend: blank for apredictor indicates that it is not important, a dot indicates that it is not so important, one starindicates importance, and more stars more importance

86

Page 87: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Modelnumber

R2 Predictors t-valueP r(> |t|)(p-value)

Significancecodes

multicollinearity

1 50,61 windprodSE3d -7.062 6.76e-12 *** no

consumSE3d 20.150 < 2e − 16 ***

2 64,28 windprodSE3d -8.542 2.4e-16 *** no

capto SE3d,max -12.779 < 2e − 16 ***

consumSE3d 26.026 < 2e − 16 ***

3 63,19 windprodSE3d -5.630 3.29e-08 *** no

netmedian SE3d -12.078 < 2e − 16 ***

consumSE3d 17.526 < 2e − 16 ***

4 66,07 capto SE3d,max -12.971 < 2e − 16 *** yes

prodSE3d -9.952 < 2e − 16 ***

consumSE3d 24.475 < 2e − 16 ***

5 65,7 windprodSE3d -0.508 0.61200 ** yes

netmedian SE3d -12.536 < 2e − 16 ***

prodSE3d -5.654 3.05e-10 ***

consumSE3d 14.888 < 2e − 16 ***

6 63,3 windprodSE3d -5.733 1.88e-08 *** no

netmedian SE3d -8.312 1.30e-15 ***

f lowf rom SE3d,max -1.553 0.121

consumSE3d 16.299 < 2e − 16 ***

7 65,75 windprodSE3d -7.775 5.82e-14 *** no

netmedian SE3d -5.793 1.35e-08 ***

f lowto SE3d,max -5.721 2.01e-08 ***

consumSE3d 18.104 < 2e − 16 ***

8 52,95 capf rom SE3d,max 0.448 0.6542 yes

prodSE3d -4.790 2.32e-06 ***

windprodSE3d -2.019 0.0441 *

consumSE3d 15.092 < 2e − 16 ***

9 67 consumSE3d 15.146 < 2e − 16 *** yes

prodSE3d -6.652 9.03e-11 ***

windprodSE3d -1.179 0.23907

netmedian SE3d -7.000 1.02e-11 ***

f lowf rom SE3d,max -4.003 7.39e-05 ***

capf rom SE3d,max 2.992 0.00294 **

Table 15: Statistical estimates of GLM for the SE4 region. Significance legend: blank for apredictor indicates that it is not important, a dot indicates that it is not so important, one starindicates importance, and more stars more importance

87

Page 88: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Modelnumber

R2 Predictors F p-value Significancecodes

multicollinearity

1 61 windprodSE3d 46.30 3.25e-11 *** no

consumSE3d 80.94 < 2e − 16 ***

2 74,3 windprodSE3d 70.86 4.58e-16 *** no

capto SE3d,max 35.92 < 2e − 16 ***

consumSE3d 140.20 < 2e − 16 ***

3 62,5 windprodSE3d 27.59 2.36e-07 *** no

netmedian SE3d 24.93 < 2e − 16 ***

consumSE3d 50.85 < 2e − 16 ***

4 73,4 capto SE3d,max 40.44 < 2e − 16 *** yes

prodSE3d 60.30 5.3e-14 ***

consumSE3d 116.47 < 2e − 16 ***

5 73,9 windprodSE3d 4.867 0.0279 * yes

netmedian SE3d 24.802 < 2e − 16 ***

prodSE3d 5.480 0.0197 *

consumSE3d 35.947 < 2e − 16 ***

6 73,8 windprodSE3d 32.346 2.38e-08 *** no

netmedian SE3d 17.103 < 2e − 16 ***

f lowf rom SE3d,max 4.816 0.0287 *

consumSE3d 46.716 < 2e − 16 ***

7 77,1 windprodSE3d 71.53 3.49e-16 *** no

netmedian SE3d 12.28 1.53e-11 ***

f lowto SE3d,max 13.00 2.67e-14 ***

consumSE3d 73.03 < 2e − 16 ***

8 61 capto SE3d,max 0.308 0.579272 yes

prodSE3d 3.660 0.056412 .

windprodSE3d 12.668 0.000414 ***

consumSE3d 47.796 < 2e − 16 ***

9 75 consumSE3d 36.861 < 2e − 16 *** yes

prodSE3d 10.989 0.000996 ***

windprodSE3d 8.474 0.003794 **

netmedian SE3d 15.200 < 2e − 16 ***

f lowf rom SE3d,max 16.557 5.63e-05 ***

capf rom SE3d,max 13.428 0.000280 ***

Table 16: Statistical estimates of GAM for the SE4 region. Significance legend: blank for apredictor indicates that it is not important, a dot indicates that it is not so important, one starindicates importance, and more stars more importance

88

Page 89: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

7.4 Source code

R code for GLM and GAM models for SE3.

1 #Data for modeling2 d <− data . frame ( cbind ( avgdayprice= avgdayprice [ 1 : (N/

24) , 1 ] , consumption SE3 =consumption SE3 [ 1 : (N/ 24)] , wind SE3=wind SE3 [ 1 : (N/ 24) ] , production SE3=production SE3 [ 1 : (N/ 24) ] , capac i ty max d to SE3=capac i ty max d to SE3 [ 1 : (N/ 24) ] , capac i ty max d outSE3=capac i ty max d out SE3 [ 1 : (N/ 24) ] , net capac i tymedian SE3 d=s t med SE3 [ 1 : (N/ 24) ] , production a l l =

production a l l [ 1 : (N/ 24) ] , capac i ty max d to SE4=capac i ty max d to SE4 [ 1 : (N/ 24) ] , flow max d out SE3=flow max d out SE3 [ 1 : (N/ 24) ] , flow max d to SE3=flow max d to SE3 [ 1 : (N/ 24) ] ) )

3 #Example of modeling for SE34 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *5 # genera l ized a d d i t i v e model6 # funct ion for es t imat ion GAM7 es t imates gam<− funct ion ( model var , model ) {8 pr int ( model )9 #summary output

10 pr int ( ”Summary” )11 pr int ( summary ( model var ) )12 # c a l c u l a t i o n of AIC13 pr int ( ”AIC” )14 pr int ( AIC ( model var ) )15 # c a l c u l a t i o n of r e l a t i v e importance16 pr int ( ’ R e l a t i v e Importance ’ )17 pr int ( c a l c . relimp ( model var , type = c ( ”lmg” ) ,

r e l a = TRUE) )18 # building the f o r e c a s t19 pred gam<−pred i c t ( model var , type=” response ” ) #

value20 r e a l data and pre d ic t io n gam<−c ( avgdayprice [ 1 : (N/

24) ] , pred gam)21 r e a l data 2013<− s p o t p r i c e s 2013[1 :126 ,1]22 #output of the f o r e c a s t with r e a l data23 par ( mfrow=c ( 1 , 1 ) , l a s =2 ,mar=c ( 9 , 6 , 4 , 2) + 0 . 1 )24 months<− seq ( from=as . Date ( ’ 2011−11−01 ’ ) , to=as .

Date ( ’ 2013−05−06 ’ ) , by= ’ day ’ )

89

Page 90: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

25 plot ( months , r e a l data and pre d ic t i on gam[ 1 : 5 5 3 ] ,type=” l ” , co l=”deeppink” , xaxt=”n” , ylab=” Realdata and predict ion , EUR/MWh” , xlab = NA, cex .lab =1.5 , cex . a x i s =1.5 , cex . main=2 , cex . sub =1.5)

26 l i n e s ( months [ 1 : (N/ 24) ] , r e a l data and pre d ic t io ngam [ 1 : (N/ 24) ] , co l= ’ aquamarine ’ )

27 l i n e s ( months [ 4 2 8 : 5 5 3 ] , r e a l data 2013[1 :126] , co l=”blue ” )

28 legend ( ” t op r ig h t ” , i n s e t =.05 , c ( ” Real data from1.11.2011 to 31.12.2012 ” , ” Real data from1.01.2013 ” , ” Pred ic t ion ” ) , l t y=c ( 1 , 1 ) , co l= c ( ”aquamarine” , ” blue ” , ”deeppink” ) , cex =1.5)

29 a x i s . Date ( 1 , a t=seq ( from=as . Date ( ’ 2011−11−01 ’ ) , to=as . Date ( ’ 2013−05−06 ’ ) , by= ’month ’ ) , format=”%d/%m/%Y” , cex . a x i s =1.5)

30 mtext ( ”months” , s ide =1 , l a s =1 , l i n e =8 , cex =1.5)31 }32 # cons t ruc t ion of the model33 gam f i t <−gam( avgdayprice ˜ s ( wind SE3 )+s ( consumption

SE3 ) , family=gaussian ( ) , o f f s e t =NULL, method=”ML” ,data=d )

34 # c a l l of funct ion35 es t imates gam(gam f i t , ’gam SE3 1 ’ )36 #save r e s u l t in . jpg format37 dev . copy ( jpeg , f i lename=” Pred ic t ion SE3 GAM 1 . jpg ” ,

width =1280 , height =720) ;38 dev . o f f ( ) ;39 #output of r e s i d u a l s histogram40 h i s t ( r e s i d u a l s (gam f i t , type = c ( ” deviance ” ) ) , main=”

Histogram of r e s i d u a l s GAM” , xlab=” Residuals ” )41 dev . copy ( jpeg , f i lename=” Histogram of r e s i d u a l s SE3 1

gam . jpg ” , width =800 , height =600) ;42 dev . o f f ( ) ;43

44 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *45 # genera l ized l i n e a r model46 # funct ion for es t imat ion GLM47 es t imates glm<− funct ion ( model var , model ) {48 pr int ( model )49 #summary output

90

Page 91: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

50 summary ( model var )51 # c a l c u l a t i o n of r e l a t i v e importance52 c a l c . relimp ( model var , type = c ( ”lmg” ) , r e l a = TRUE)53 # building the f o r e c a s t54 pred glm<−pred ic t ( model var , type=” response ” ) #value55 r e a l data and pre d ic t i on glm<−c ( avgdayprice [ 1 : (N/ 24)

, 1 ] , pred glm )56 # c a l c u l a t i o n of $\bar {R } ˆ 2 $57 SS e r r<−sum ( ( avgdayprice [ 1 : (N/ 24) ,1] −pred glm ) ˆ 2 )58 SS t o t<−sum ( ( avgdayprice [ 1 : (N/ 24) ,1] −mean(

avgdayprice [ 1 : (N/ 24) , 1 ] ) ) ˆ 2 )59 R2 <−1 − ( SS e r r /SS t o t )60 R2 adj<−1−(1−R2 ) * ( (N/24−1)/ (N/24−2) )61 R2 adj62 #output of the f o r e c a s t with r e a l data63 par ( mfrow=c ( 1 , 1 ) , l a s =2)64 plot ( months , r e a l data and pre d ic t io n glm [ 1 : 5 5 3 ] , type

=” l ” , co l=” c o r a l ” , ylab=” Real data and pred ic t io n ” ,xaxt=”n” , xlab = NA, cex . lab =1.5 , cex . a x i s =1.5 , cex. main=2 , cex . sub =1.5 )

65 l i n e s ( months [ 1 : (N/ 24) ] , r e a l data and pre d ic t i on glm[ 1 : (N/ 24) ] , co l= ’ aquamarine ’ )

66 l i n e s ( months [ 4 2 8 : 5 5 3 ] , r e a l data 2013[1 :126] , co l=”blue ” )

67 legend ( ” t op r ig h t ” , i n s e t =.05 , c ( ” Real data from1.11.2011 to 31.12.2012 ” , ” Real data from 1.01.2013” , ” Pred ic t ion ” ) , l t y=c ( 1 , 1 ) , co l= c ( ”aquamarine” , ”blue ” , ” c o r a l ” ) , cex =1.5)

68 a x i s . Date ( 1 , a t=seq ( from=as . Date ( ’ 2011−11−01 ’ ) , to=as. Date ( ’ 2013−05−06 ’ ) , by= ’month ’ ) , format=”%d/%m/%Y”, cex . a x i s =1.5)

69 }70 # cons t ruc t ion of the model71 glm f i t <−glm ( avgdayprice ˜ wind SE3+consumption SE3 ,

family=gaussian ( ) , data=d )72 # c a l l of funct ion73 es t imates glm ( glm f i t , ’gam SE3 1 ’ )74 #save r e s u l t in . jpg format75 dev . copy ( jpeg , f i lename=” Pred ic t ion SE3 GLM 1 . jpg ” ,

width =1280 , height =720) ;

91

Page 92: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

76 dev . o f f ( ) ;77 #output of r e s i d u a l s histogram78 h i s t ( r e s i d u a l s ( glm f i t , type = c ( ” deviance ” ) ) , main=”

Histogram of r e s i d u a l s GLM” , xlab=” Residuals ” )79 dev . copy ( jpeg , f i lename=” Histogram of r e s i d u a l s SE3 1

glm . jpg ” , width =800 , height =600) ;80 dev . o f f ( ) ;81

82 #output GAM and GLM f o r e c a s t , r e a l data in the samef i g u r e

83 par ( mfrow=c ( 2 , 1 ) )84 months<− seq ( from=as . Date ( ’ 2011−11−01 ’ ) , to=as . Date ( ’

2013−05−06 ’ ) , by= ’ day ’ )85 plot ( months , r e a l data and pre d ic t io n gam[ 1 : 5 5 3 ] , type

=” l ” , co l=”deeppink” , xaxt=”n” , ylab=” Real data andpre d ic t i on ” , main=” Pr ice pr ed ic t i on SE3 (Generalized a dd i t i v e model ) ” )

86 l i n e s ( months [ 1 : (N/ 24) ] , r e a l data and pre d ic t i on gam[ 1 : (N/ 24) ] , co l= ’ aquamarine ’ )

87 l i n e s ( months [ 4 2 8 : 5 5 3 ] , r e a l data 2013[1 :126] , co l=”blue ” )

88 legend ( ” t op r ig h t ” , i n s e t =.05 , c ( ” Real data from1.11.2011 to 31.12.2012 ” , ” Real data from 1.01.2013” , ” Pred ic t ion ” ) , l t y=c ( 1 , 1 ) , co l= c ( ”aquamarine” , ”blue ” , ”deeppink” ) )

89 a x i s . Date ( 1 , a t=seq ( from=as . Date ( ’ 2011−11−01 ’ ) , to=as. Date ( ’ 2013−05−06 ’ ) , by= ’month ’ ) , format=”%d/%m/%Y”)

90 plot ( months , r e a l data and pre d ic t io n glm [ 1 : 5 5 3 ] , type=” l ” , co l=” c o r a l ” , ylab=” Real data and pred ic t io n ” ,xaxt=”n” , main=” Pr ice pre d ic t i on SE3 ( Generalizedl i n e a r model ) ” )

91 l i n e s ( months [ 1 : (N/ 24) ] , r e a l data and pre d ic t i on glm[ 1 : (N/ 24) ] , co l= ’ aquamarine ’ )

92 l i n e s ( months [ 4 2 8 : 5 5 3 ] , r e a l data 2013[1 :126] , co l=”blue ” )

93 legend ( ” t op r ig h t ” , i n s e t =.05 , c ( ” Real data from1.11.2011 to 31.12.2012 ” , ” Real data from 1.01.2013” , ” Pred ic t ion ” ) , l t y=c ( 1 , 1 ) , co l= c ( ”aquamarine” , ”blue ” , ” c o r a l ” ) )

92

Page 93: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

94 a x i s . Date ( 1 , a t=seq ( from=as . Date ( ’ 2011−11−01 ’ ) , to=as. Date ( ’ 2013−05−06 ’ ) , by= ’month ’ ) , format=”%d/%m/%Y”)

95 #save r e s u l t in . jpg format96 dev . copy ( jpeg , f i lename=” Pred ic t ion SE3 GAM GLM 1 . jpg

” , width =1720 , height =1280) ;97 dev . o f f ( ) ;98 par ( mfrow=c ( 1 , 1 ) )

93

Page 94: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

References

[1] Weron R. (2007): ”Modeling and Forecasting Electricity Loads andPrices” (Wiley, 192 p., ISBN: 978-0-470-05753-7)

[2] Shtridbek U. (2005): ”Knowledge learned from the liberalization ofelectricity markets” (The International Energy Agency, 277 p.)

[3] Ostberg H., Bergstrom M. (2011): ”The Swedish electricity and nat-ural gas markets 2010” (GM Gruppen, 72 p.)

[4] Bixia, URL: http://bixia.se/Bixia/ (date: 15.04.2013)

[5] Wood S. (2006): ”Generalized Additive Models: an introduction withR” (Chapman and Hall, 391 p., ISBN: 1584884746)

[6] Annette J. Dobson (2002): An introduction to Generalized LinearModels, second edition (Chapman and Hall/CRC, 212 ISBN: 1-58488-165-8)

[7] Generalized Linear Models (German Rodriguez, PrincetonUniversity), URL: http://data.princeton.edu/wws509/notes/(date:2.01.2013)

[8] McCullagh, P. and Nelder, J.A. (1989): ”Generalized Linear Models,2nd ed.” (Chapman and Hall, 532 p., ISBN: 978-0412317606)

[9] Trevor Hastie, Robert Tibshirani(1986): ”Generalized Additive Mod-els” (Statistical Science, 14 p., ISBN: 297-318)

[10] M.R. Osborne, Fishers method of scoring, Int. Stat. Rev. 86 (1992),271286.

[11] Kremer N., Putko B.: (2002): ”Econometrics: A Textbook for uni-versities” (Moscow,311 p., ISBN: 5-238-00333-1)

[12] ”FAQ: What are pseudo R-squareds”,URL:http://statistics.ats.ucla.edu/stat/mult pkg/f aq/general/P suedo RSquareds.htm(date:28.12.2013)

[13] G.E.P. Box, G.M. Jenkins, and G.C. Reinsel (may 1999): ”Time Se-ries Analysis Forecasting and Control” (Wiley, 784 p., ISBN: 978-0470272848)

[14] P. Durka, S. Pastorekova ”ARIMA vs. ARIMAX, which approachis better to analyze and forecast macroeconomic time series?” (Pro-ceedings of 30th International Conference Mathematical Methods inEconomics, p. 5)

94

Page 95: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

[15] Chapter 13 ”AIC and BIC” (pp. 45-47) URL:https://www.scss.tcd.ie/Rozenn.Dahyot/ST7005/13AICBIC.pdf(date: 15.09.2013)

[16] E. Rublikova, L. Marek (2001): ”Linear transfer function model foroutflow rates” (Ekonomicke rozhlady, vol. 30, no. 4, pp. 457-466)

[17] A. Pankratz (1991):”Forecasting with Dynamic Regression Models”(Wiley-Interscience, 400 p., ISBN: 978-0471615286)

[18] Nord Pool Spot, URL: http://www.nordpoolspot.com/ (date:15.05.2013)

[19] Nord Pool Spot: ”The Nordic electricity exchange and theNordic model for a liberalized electricity market”, URL:http://www.nordpoolspot.com/Global/Download%20Center/Rules-and-regulations/The-Nordic-Electricity-Exchange-and-the-Nordic-model-for-a-liberalized-electricity-market.pdf (date: 18.11.2013)

[20] IST University Computing Systems, URL:http://web.njit.edu/all topics/Prog Lang Docs/html/library/base/html/glm.html (date: 15.05.2013)

[21] Seminar for Statistics, URL: http://stat.ethz.ch/R-manual/R-patched/library/base/html/summary.html (date: 15.05.2013)

[22] Groemping U. (2010): ”Package relaimpo” (URL: http://cran.r-project.org/web/packages/relaimpo/relaimpo.pdf, 36 p.)

[23] The Comprehensive R Archive Network, URL: http://cran.r-project.org/web/packages/relaimpo/ (date: 15.05.2013)

[24] Seminar for Statistics, URL: http://stat.ethz.ch/R-manual/R-patched/library/stats/html/predict.html (date: 15.05.2013)

[25] Wood S. (2013): ”Packagemgcv” (URL: http://cran.r-project.org/web/packages/mgcv/mgcv.pdf, 215 p.)

[26] The Comprehensive R Archive Network, URL: http://cran.r-project.org/web/packages/mgcv/index.html (date: 15.05.2013)

[27] Calculate the Variance Inflation Factor, URL:http://rss.acs.unt.edu/Rdoc/library/HH/html/vif.html (date:15.05.2013)

[28] The Comprehensive R Archive Network, URL: http://cran.r-project.org/web/packages/HH/index.html (date: 15.05.2013)

95

Page 96: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

[29] The Local Sweden’s news in English, URL:http://www.thelocal.se/37212/20111107/#.UYpERSraq30 (date:9.06.2013)

[30] The Comprehensive R Archive Network, URL: http://cran.r-project.org/web/packages/RGtk2/index.html (date: 5.06.2013)

[31] The Comprehensive R Archive Network, URL: http://cran.r-project.org/web/packages/cairoDevice/index.html (date:5.06.2013)

[32] The Comprehensive R Archive Network, URL: http://cran.r-project.org/web/packages/gWidgets/ (date: 5.06.2013)

[33] The Comprehensive R Archive Network, URL: http://cran.r-project.org/web/packages/gWidgetsRGtk2/index.html (date:5.06.2013)

[34] Verzani J. (2012): ”Examples for gWidgets” (URL: http://cran.r-project.org/web/packages/gWidgets/vignettes/gWidgets.pdf, 40p.)

[35] Lawrence M., Temple Lang D. (2010): ”RGtk2: A Graphical UserInterface Toolkit for R” (Journal of Statistical Software, December2010, Volume 37, Issue 8, 52 p.)

[36] Lawrence M., Temple Lang D. (2013): ”Package Gtk2” (URL:http://cran.r-project.org/web/packages/RGtk2/RGtk2.pdf, 37 p.)

[37] How to Install an R Package, URL:http://math.usask.ca/ longhai/-software/installrpkg.html (date: 5.06.2013)

96

Page 97: Electricity spot price forecasting in two Swedish …876430/FULLTEXT01.pdfDenition 1. A market clearing price (MCP) is the price of a good (for example, electricity) when supply and

Faculty of Technology

SE-391 82 Kalmar | SE-351 95 Växjö

Phone +46 (0)772-28 80 00

[email protected]

Lnu.se/faculty-of-technology?l=en