final year project report
DESCRIPTION
Application of Neural Network in PredictionTRANSCRIPT
APPLICATION OF NEURAL NETWORK IN FORECASTING
List of Abbreviations
ANN Artificial Neural Network
GWL Ground Water Level
RWH Rain Water Harvesting
h1 Hidden layer 1
h2 Hidden layer 2
NARX Nonlinear Autoregressive Exogenous Model
R Correlation Coefficient
MSE Mean Square Error
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
INTRODUCTION
Department of Computer Science & Engg., FIET Bareilly
CHAPTER 1
APPLICATION OF NEURAL NETWORK IN FORECASTING
A prediction (Latin præ-, "before," and dicere, "to say") or forecast is a statement about the way
things will happen in the future, often but not always based on experience or knowledge. While
there is much overlap between prediction and forecast, a prediction may be a statement that some
outcome is expected, while a forecast is more specific, and may cover a range of possible
outcomes. Although guaranteed information about the future is in many cases impossible,
prediction is necessary to allow plans to be made about possible developments; Howard H.
Stevenson writes that prediction in business "... is at least two things: Important and hard."
Football (Soccer) has becoming increasingly popular over the years. During the last decade,
soccer’s biggest event, the FIFA World Cup, has attracted millions of fans worldwide; in fact, the
viewership of the FIFA World Cup match arguably surpasses the Super Bowl event as soccer is the
more widely played sport in the world. To showcase its international reach, the World Cup was
hosted in Japan and South Korea in 2002. South Africa hosted its first World Cup in 2010, and
Brazil will host the next World Cup in 2014. Even in the Middle East, soccer appeals to the
audience in that part of the world, with Qatar hosting the World Cup in 2022. The exposure of
soccer to different continents highlights the growing popularity for the sport that originated in
Europe.
With great viewership comes great opportunity for soccer viewers to bet on scores too. This is
especially true in Asia, where soccer betting is common among soccer fans. For instance, in
Singapore, the Singapore Pools, which is Singapore’s legalized gambling institution, allows
Singapore citizens to bet on almost anything, with soccer betting being one of its largest revenue
generators. These bets range from predicting the outcome of the score, the exact result of the score,
the winning margin and whether a certain player will score first. They are based on analyses
including whether a certain player is playing, the coach’s record against the opposition and the
strategies that the coach has been using. These factors are all based on human analyses with a tinge
of biasness in them, which is inevitable for any sport that involves human judgment.
In this experiment it will be shown how neural networks and Neuroph Studio are used when it
comes to problems of classification. Several architectures will be tried out, and it will be
determined which ones represent a good solution to the problem, and which ones do not.
Department of Computer Science & Engg., FIET Bareilly
1.1 An Overview
APPLICATION OF NEURAL NETWORK IN FORECASTING
1.2 Football : The Game
Football refers to a number of sports that involve, to varying degrees, kicking a ball with the foot
to score a goal. The most popular of these sports worldwide is association football, more
commonly known as just "football" or "soccer". Unqualified, the word football applies to
whichever form of football is the most popular in the regional context in which the word appears,
including association football, as well as football, Australian rules football, Canadian
football, Gaelic football, rugby league, rugby union, and other related games. These variations of
football are known as football codes.
Various forms of football can be identified in history, often as popular peasant games.
Contemporary codes of football can be traced back to the codification of these games at English
public schools in the eighteenth and nineteenth centuries. The influence and power of the British
Empire allowed these rules of football to spread to areas of British influence outside of the directly
controlled Empire, though by the end of the nineteenth century, distinct regional codes were
already developing: Gaelic Football, for example, deliberately incorporated the rules of local
traditional football games in order to maintain their heritage. In 1888, The Football League was
founded in England, becoming the first of many professional football competitions. During the
twentieth century, several of the various kinds of football grew to become among the most popular
team sports in the world.
1.3 Rules and facts of game
There are 17 laws in total, each one briefly summarized below.
Field of Play. The game can be played on either natural or artificial surfaces, the surface must be
green and rectangular in shape. The two long sides of the rectangle are called touch lines and the
two shorter sides are called goal lines. The field is divided in half by the halfway line.
Ball. Must be spherical, made of leather (or similar) 68-70 cm in circumference and of a certain
pressure.
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
Number of Players. Two teams of no more than 11 players (one of which is the goalkeeper). A
game cannot start if either team has less than 7 players. Players are divided into three
combination : Defender , midfielder , Forward player .
Equipment. Players must wear a jersey, shorts, stockings, shin guards and footwear.
Referee. The referee ensures the Laws of the Game are respected and upheld.
Assistant Referees. There may be at most 2 assistant referees.
Duration of the Match. The game is played in 2 halves consisting of 45 minutes each. The half
time interval must not exceed more than 15 minutes. At the discretion of the referee more time is
allowed to compensate for any stoppage during play e.g. Due to substitutions or care and attention
of injured players.
Start and Restart of Play. A kick-off starts play at the start of the match or after a goal. A kick-
off involves one player kicking the ball, from stationary, forward from the centre spot. All players
must be in their own half prior to kick-off. A coin is tossed pre-game, the team which loses the
toss are awarded the kick-off to start the game whilst the team that win the toss are allowed to
choose which direction they want to play. After half time the teams switch direction and the other
team will kick-off. After a goal is scored, the team which conceded the goal will kick-off to restart
play.
Ball in and Out of Play. The ball is out of play once a goal has been scored or when the referee
has stopped the game. The ball is in play at all other times.
Method of Scoring. The ball crosses the goal line inside the goal mouth.
Offside. It is an offence for a player to be in contact with the ball when they are closer to the
opponents' goal than both the ball and the second-last opponent. The offside rule exists to ensure
there are always opponents (generally the goal keeper and a defender) between a player receiving
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTINGthe ball and the goal. Without the offside rule, play can become boring with repeated long balls
being kicked to a player stood next to the goalkeeper for an easy goal.
Fouls/Misconduct. These are many and varied, broadly speaking it is an offence to use excessive
force whilst playing the game either deliberately or undeliberately or to handle the ball (unless you
are a goal keeper). The referee may show the yellow card to caution players for less serious
offences and the red card for more serious offences resulting in the player being sent off. Two
yellow cards are equivalent to one red card.
Free Kicks. Are given by the referee for fouls and misconduct. A free kick can either be direct or
indirect. A goal can be scored directly from a direct free kick. A goal can only be scored from an
indirect free kick if it touches at least one other player first. The free kick must be taken from a
stationary position with that position varying depending on whether the free kick was given inside
or outside the goal area and whether it's direct or indirect. The opposing team must be a minimum
of 9.15 m from the ball when the free kick is taken.
Penalty Kicks. Are given against a team when they commit an offence which would normally be
awarded a direct free kick inside their goal area. The ball is kicked from stationary from the
penalty spot. The opposing team must be outside of the penalty area and at least 9.15 m from the
ball.
Throw-in. Used to restart play after the whole of the ball has crossed the touch line.
Goal kick. Used to restart play after a goal has been scored.
Corner Kick. Is given when the whole of the ball crosses the goal line and was last touched by a
member of the defending team (and no goal was scored). A corner kick is taken from inside the
corner arc closest to the point where the ball crosses the goal line. The defending team must be at
least 9.15 m from the ball when the corner kick is taken.
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
1.4 Role of prediction in Football
Predicting the outcome of sporting events is a business which has grown in popularity in recent
years. Handicappers predict the outcome of games using a variety of mathematical formulas,
simulation models or qualitative analysis. Early, well known sports bettors, such as Jimmy the
Greek, were believed to have access to information that gave them an edge. Information ranged
from personal issues, such as gambling or drinking to undisclosed injuries; anything that may
affect the performance of a player on the field.
Recent times have changed the way sports are predicted. Predictions now typically consist of two
distinct approaches: Situational plays and statistical based models. Situational plays are much more
difficult to measure because they usually involve the motivation of a team. Dan Gordon, noted
handicapper, wrote “Without an emotional edge in a game in addition to value in a line, I won’t
put my money on it”. These types of plays consist of: Betting on the home underdog, betting
against Monday Night winners if they are a favorite next week, betting the underdog in “look
ahead” games etc. As situational plays become more widely known they become less useful
because they will impact the way the line is set.
The widespread use of technology has brought with it more modern sports betting systems. These
systems are typically algorithms and simulation models based on regression analysis. Jeff Sagarin,
a sports statistician, has brought attention to sports by having the results of his models published in
USA Today. He is currently paid as a consultant by the Dallas Mavericks for his advice on lineups
and the use of his Winval system, which evaluates free agents. Brian Burke, a former Navy fighter
pilot turned sports statistician, has published his results of using regression analysis to predict the
outcome of NFL games. Ken Pomeroy is widely accepted as a leading authority on college
basketball statistics. His website includes his College Basketball Ratings, a tempo based statistics
system. Some statisticians have become very famous for having successful prediction systems.
Dare wrote “the effective odds for sports betting and horse racing are a direct result of human
decisions and can therefore potentially exhibit consistent error”. Unlike other games offered in a
casino, prediction in sporting events can be both logical and consistent
In short we can say that we can use prediction techniques to find out best combination for any
team by predicting individual performance of players and the total effect for their performances in
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTINGthe team .Using this process we can select best combination for any playing condition and
opponent easily and effectively.
1.5 Neural Network
1.5.1 Biological neural networks
A neuron ( or nerve cell ) is a special biological cell that processes information (see Figure
1.8). It is composed of a cell body, or soma, and two types of out-reaching tree-like
branches: the axon and the dendrites. The cell body has a nucleus that contains information
about hereditary traits and plasma that holds the molecular equipment for producing material
needed by the neuron. A neuron receives signals (impulses) from other neurons through its
dendrites (receivers) and transmits signals generated by its cell body along the axon
(transmitter), which eventually branches into strands and sub strands. At the terminals of
these strands are the synapses. A synapse is an elementary structure and functional unit
between two neurons (an axon strand of one neuron and a dendrite of another), When the
impulse reaches the synapse’s terminal, certain chemicals called neurotransmitters are
released. The neurotransmitters diffuse across the synaptic gap, to enhance or inhibit,
depending on the type of the synapse, the receptor neuron's own tendency to emit electrical
impulses. The synapse's effectiveness can be adjusted by the signals passing through it so that
the synapses can learn from the activities in which they participate. This
dependence on history acts as a memory, which is possibly responsible for human
memory [3].
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
Figure1.1 Biological Neuron [3]
The cerebral cortex in humans is a large flat sheet of neurons about 2 to 3 millimeters
thick with a surface area of about 2,200 cm2, about twice the area of a standard
computer keyboard. The cerebral cortex contains about 10" neurons, which is
approximately the number of stars in the Milky Way." Neurons are massively connected,
much more complex and dense than telephone networks. Each neuron is connected to
103 to l04 other neurons.
In total, the human brain contains approximately 1014 to l015 interconnections.
Neurons communicate through a very short train of pulses, typically milliseconds in
duration. The message is modulated on the pulse-transmission frequency. This frequency
can vary from a few to several hundred hertz, which is a million times slower than the
fastest switching speed in electronic circuits. However, complex perceptual decisions such
as face recognition are typically made by humans within a few hundred milliseconds.
These decisions are made by a network of neurons whose operational speed is only a few
milliseconds. This implies that the computations cannot take more than about 100 serial
stages. In other words, the brain runs parallel programs that are about
100 steps long for such perceptual tasks. This is known as the hundred step rule.12 The
same timing considerations show that the amount of information sent from one neuron
to another must be very small (a few bits). This implies that critical information is not
transmitted directly, but captured and distributed in the interconnections-hence the
name, connectionist model, used to describe A’s .
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
1.5.2 Artificial Neural Network
Artificial neural network is a biologically inspired mathematical tool used widely
for prediction and classification. In basic Artificial neural network there are three
layers of artificial neurons given below:
1) Input layer: The function of this layer is to take input.
2) Hidden layer: This layer process that input as per the given activation function.
3) Output layer: After processing by hidden layer output layer provides the
output.
Figure 1.2 Artificial neural network
An Artificial neuron is a processing unit which process the input. The input is taken by
a input vector x. Each edge connecting two neurons have some weight Wij shows
the connection strength between two neurons. The weights are given in weight matrix W.
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
1.5.3 Models of a neuron
A neuron is an information-processing unit that is fundamental to the operation of a
neural network. We may identify three basic elements of the neuron model:
Figure 1.3 Nonlinear model of a neuron
1. A set of synapses, each of which is characterized by a weight or strength of its own.
Specifically, a signal by the synaptic weight at the input of synapse j connected to neuron k is
multiplied. It is important to make a note of the manner in which the subscripts of the synaptic
weight are written. The first subscript refers to the neuron in question and the second subscript
refers to the input end of the synapse to which the weight refers. The weight is positive if the
associated synapse is excitatory; it is negative if the synapse is inhibitory.
2. An adder for summing the input signals, weighted by the respective synapses of the neuron.
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING3. An activation function for limiting the amplitude of the output of a neuron. The activation
function is also referred to in the literature as a squashing function in that
it squashes (limits) the permissible amplitude range of the output signal to some finite value.
Typically, the normalized amplitude range of the output of a neuron is written as the closed unit
interval [0, 1] or alternatively [-1, 1].
4. The model of a neuron also includes an externally applied bias (threshold) wk0 = bk that has the
effect of lowering or increasing the net input of the activation function.
In mathematical terms, we may describe a neuron k by writing the following pair of equations:
or in a matrix form
Types of activation functions [3]
a) Threshold activation function (McCulloch–Pitts model)
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
Figure 1.4 Threshold activation function.
In this model, the output of a neuron takes on the value of 1 if the total internal activity
level of that neuron is nonnegative and 0 otherwise. This statement describes the all-
or-none property of the McCulloch–Pitts model.
The McCulloch–Pitts model of a neuron is sometimes used with the output either -1 or +1:
b) Piecewise-linear activation function
Figure 1.5 Piecewise-linear activation function
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
The amplification factor inside the linear region is assumed to be unity. The following
two situations may be viewed as special forms of the piecewise linear function:
1. A linear combiner arises if the linear region of operation in maintained without
running into saturation.
2. The piecewise-linear function reduces to a threshold function if the amplification factor
of the linear region in made infinitely large.
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
c) Sigmoid (logistic) activation function
Figure 1.6 Sigmoid (logistic) activation functions.
The sigmoid function is the most common form of activation function used in the
construction of artificial neural networks. Whereas a threshold function assumes the value of
0 or 1, a sigmoid function assumes a continuous range of values form 0 and 1. Note also that
the sigmoid function is differentiable, which is an important feature of neural network theory.
The derivative of the sigmoid function has a nice property, which makes is easy to calculate:
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
d) Hyperbolic tangent function
Figure 1.7 Hyperbolic tangent activation function
The hyperbolic tangent function can be easily expressed in terms of the logistic function: (2 ×
logistic function – 1). Its derivative is also easy to calculate:
e) Softmax activation function
One approach toward approximating probabilities is to choose the output neuron nonlinearity
to be exponential rather than sigmoidal and for each pattern to normalize the outputs to sum
to 1. Let c be the number of output neurons. Each output is generated by the following
activation function:
It is clear that the softmax activation function is a smoothed version of a winner-
take-all nonlinearity in which the maximum output is transformed to 1, and all others
reduced to 0.
The use of softmax is appropriate when the network is to be used for estimating probabilities.
For example, each output yi can represent a probability that the corresponding neural network
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTINGinput belongs to class Ci.
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
1.5.4 Multilayer feedforward network
Figure 1.8 A 10-4-2 fully connected feedforward network.
The source nodes in the input layer of the network supply respective elements of the
activation pattern (input vector), which constitute the input signals applied to the neurons
(computation nodes) in the second layer (i.e. the first hidden layer). The output signals of the
second layer are used as inputs to the third layer, and so on for the rest of the network.
Typically, the neurons in each layer of the network have as their inputs the output signals of
the preceding layer only. The set of output signals of the neurons in the output layer of the
network constitutes the overall response of the network to the activation pattern supplied by
the source nodes in the input layer. For brevity, the network shown above is referred to as a
10-4-2 network in that it has 10 source nodes, 4 hidden neurons, and 2 output neurons. As
another example, a feed forward network with p source nodes, h1 neurons in the first
hidden layer, h2 neurons in the second layer, and q neurons in the output layer is referred to
as a p-hl-h2-q network [3].
A neural network is said to be fully connected if every node in each layer of the network is
connected to every other node in the adjacent forward layer. If, however, some of the
communication links (synaptic connections) are missing from the network, we say that the
network is partially connected.
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
1.6 Learning
The ability to learn is a fundamental trait of intelligence. Although a precise definition
of learning is difficult to formulate, a learning process in the ANN context can be viewed as
the problem of updating network architecture and connection weights so that a network
can efficiently perform a specific task. The network usually must learn the connection
weights from available training patterns. Performance is improved over time by iteratively
updating the weights in the network. ANNs' ability to automatically learn from examples
makes them attractive and exciting. Instead of following a set of rules specified by human
experts, ANNs appear to learn underlying rules (like input-output relationships) from the
given collection of representative examples. This is one of the major advantages of
neural networks over traditional expert systems. To understand or design a learning
process, you must first have a model of the environment in which a neural network
operates, that is, you must know what information is available to the network. We refer
to this model as a learning paradigm. Second, you must understand how network weights
are updated, that is, which learning rules govern the updating process. A learning algorithm
refers to a procedure in which learning rules are used for adjusting the weights .
There are three main learning paradigms: supervised, unsupervised, and hybrid. In
supervised learning, or learning with a “teacher,” the network is provided with a correct
answer (output) for every input pattern. Weights are determined to allow the network to
produce answers as close as possible to the known correct answers. Reinforcement learning is
a variant of supervised learning in which the network is provided with only a critique on the
correctness of network outputs, not the correct answers themselves. In contrast, unsupervised
learning, or learning without a teacher, does not require a correct answer associated with each
input pattern in the training data set. It explores the underlying structure in the data, or
correlations between patterns in the data, and organizes patterns into categories from these
correlations. Hybrid learning combines supervised and unsupervised learning. Parts of the
weights are usually determined through supervised learning, while the others are obtained
through unsupervised learning.
Learning theory must address three fundamental and practical issues associated with learning
from samples: capacity, sample complexity, and computational complexity.
Capacity concerns how many patterns can be stored, and what functions and decision
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTINGboundaries a net-work can form. , Sample complexity determines the number of training
patterns needed to
train the network to guarantee a valid generalization. Too few patterns may cause “over-
fitting” (where in the network performs well on the training data set, but poorly on
independent test patterns drawn from the same distribution as the training patterns).
Computational complexity refers to the time required for a learning algorithm to estimate
a solution from training patterns. Many existing learning algorithms have high
computational complexity. Designing efficient algorithms for neural network learning is
a very active research topic.
1.6.1 Backpropogation learning
Consider a multilayer feed forward network, such as the three-layer network of Figure.
Figure 1.9 Three layer feedforward network
The net input to unit i in layer k + 1 is
(2.1)
The output of unit i will be
(2.2)
For an M layer network the system equations in matrix form are given by
(2.3)
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
(2.4)
The task of the network is to learn associations between a specified set of input-output pairs
The performance index for the network is
(2.5)
Where is the output of the network when the qth input, is presented, and
is the error for the qth input. For the standard backpropagation algorithm
we use an approximate steepest descent rule. The performance index is approximated by
(2.6)
where the total sum of squares is replaced by the squared errors for a single input/output
pair. The approximate steepest (gradient) descent algorithm is then
(2.7)
(2.8)
where is the leaming rate. Define
(2.9)
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTINGas the sensitivity of the performance index to changes in the net input of unit i in layer k.
Now it can be shown, using (2.l), (2.6), and (2.9), that
(2.10)
(2.11)
It can also be shown that the sensitivities satisfy the following recurrence relation
(2.12)
Where
(2.13)
And
(2.14)
This recurrence relation is initialized at the final layer
(2.15)
The overall learning algorithm now proceeds as follows; first, propagate the input forward
using (2.3)-(2.4); next, propagate the sensitivities back using (2.15) and (2.12); and finally,
update the weights and offsets using (2.7), (2.8), (2.l0), and (2.11).
1.6.2 Marquardt-Levenberg modification
While backpropagation is a steepest descent algorithm, the Marquardt-Levenberg algorithm
is an approximation to Newton’s method. Suppose that we have a function V( ) which
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTINGwe want to minimize with respect to the parameter vector I, then Newton's method would
be
(2.16)
Where is the Hessian matrix and is the gradient. If we assume that
is a sum of squares function
(2.17)
then it can be shown that
(2.18)
(2.19)
where J(x) is the Jacobian matrix
(2.20)And
(2.21)
For the Gauss-Newton method it is assumed that and the update (3.16) becomes
(2.22)
The Marquardt-Levenberg modification to the Gauss-Newton method is
(2.23)
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTINGThe parameter is multiplied by some factor ( ) whenever a step would result in an
increased . When a step reduces , is divided by . Notice that when is
large the algorithm becomes steepest descent (with step ), while for small the
algorithm becomes Gauss-Newton. The Marquardt-Levenberg algorithm can be considered a
trust- region modification to Gauss-Newton.
The key step in this algorithm is the computation of the Jacobian matrix. For the
neural network mapping problem the terms in the Jacobian matrix can be computed by a
simple modification to the backpropagation algorithm. The performance index for the
mapping problem is given by (3.5). It is easy to see that this is equivalent in form to
(3.17), where
and N = Q x SM. Standard backpropagation calculates terms like
(2.24)
For the elements of the Jacobian matrix that are needed for the Marquardt algorithm we
need to calculate terms like
(2.25)
These terms can be calculated using the standard backpropagation algorithm with one
modification at the final layer
(2.26)
Note that each column of the matrix in (3.26) is a sensitivity vector which must be
backpropagated through the network to produce one row of the Jacobian.
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
1.7 Project Objective
Prediction is very useful thing for games .Here we are predicting the result of a football match.
Artificial neural network provide the better prediction result comparing other techniques.
Objective of this research work is given as:
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
PROJECT LITERATURE
Department of Computer Science & Engg., FIET Bareilly
CHAPTER 2
APPLICATION OF NEURAL NETWORK IN FORECASTING
2.1 Literature Survey
A number of studies have been done on different forecasting. Various data mining
techniques have been used in these studies for prediction, and the calculated results are too
much close to the real values. Heesung Yoon et al. [8] done their study at coastal aquifer and
made a comparison between artificial neural network and support vector machine for
prediction. The results of the model performance show that root mean squared error
(RMSE) values of ANN models are lower than those of SVM in model training and
testing stages. However, the overall model performance criteria of the SVM are similar to or
even better than those of the ANN in model prediction stage.
Radha Krisna Balla [5] uses feed forward backpropogation artificial neural network for
result prediction at English Premier League in 2006-07. The model provided the good
fit and the predicted trend followed the observed data closely 65% correctness .This
proves the artificial neural network can be used for football match result prediction.
Monica Adya And Fred Collopy perform 48 studies , only eleven met all of criteria for
effectiveness of validation and implementation. Of the remaining 38, 17 presented effective
validations but suffered with respect to implementation. Eleven of these reported positive
results despite implementation problems. Altogether then, of the 48 studies, 22 contributed to
our knowledge regarding the applicability of NNs to forecasting and prediction. Nineteen
(86%) of these produced results that were favour-able, three produced results that were not.,
NNs, when they are effectively implemented and validated, show potential for forecasting and
prediction. Second, a significant portion of the NN research in forecasting and prediction lacks
validity.
Ahmad Abrishamchi et al. [7] investigate the capability of two ANN models to predict the
urban groundwater level using different sets of available input data, and then we compare the
results of these two models. A multi-input-single-output network has been trained using
Levenberg-Marquardt algorithm. The aforementioned models are evaluated using three
statistical performance criteria namely mean square error (MSE), root mean squared error
(RMSE), and efficiency (R2). This paper identifies the proficiency of ANN modeling
technique to capture the complex dynamics involved in urban groundwater level fluctuations.
The results show the importance of input data selection and its effect on prediction accuracy.
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
Sreenivasulu D and Deka P.C. [8] investigate the potential and applicability of Radial Basis
Function (RBF) for forecasting groundwater level with limited data. The results showed that
ANN with RBF network can be an effective tool for better groundwater level fluctuations
forecasting with limited data.
P. Sujatha1 and Dr. G.N. Pradeep Kumar [10] examined Performance of four types of
functionally different artificial neural network (ANN) models, namely Feed forward neural
network, Elman type recurrent neural network, Input delay neural network and Radial basis
function network and fourteen types of algorithms, namely Batch gradient descent (traingd),
Batch gradient descent with momentum (traingdm), Adaptive learning rate (traingda),
Adaptive learning rate with momentum (traingdx), Resilient backpropagation (trainrp),
Fletcher-Reeves update (traincgf), Polak-Ribiere update (traincgp), Powell-Beale restarts
(traincgb), Scaled conjugate gradient (trainscg), BFGS algorithm (trainbfg), One step
secant algorithm (trainoss), Levenberg-Marquardt (trainlm), Automated regularization
(trainbr) and Random order incremental training (trainr) were examined in order to
identify an efficient ANN architecture and algorithm that can simulate the water table
fluctuations using a relatively short length of groundwater level records. Results showed
that Feed forward neural network trained with training algorithm Levenberg-Marquardt
is suitable for accurate prediction of groundwater levels.
Ioannis N. Daliakopoulos et al. [11] investigate and compare seven different types of network
architectures and training algorithms in terms of model prediction efficiency and accuracy.
The different experiment results show that accurate predictions can be achieved with a
standard feedforward neural network trained with the Levenberg-Marquardt algorithm
providing the best results for up to 18 months forecast.
From all these studies we can conclude that feedforward neural network with Levenberg-
Marquardt training algorithm shown to be a promising tool for ground water level prediction
even with limited amount of data.
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
PROPOSED METHODOLOGY
& SIMULATION
Department of Computer Science & Engg., FIET Bareilly
CHAPTER 3
APPLICATION OF NEURAL NETWORK IN FORECASTING
3.1 Proposed Methodology
Any result of football match mainly based on performance of participating teams. This
performance should be equal or best in different roles and areas of the game. For best
performance of any team their players should be fit and inform .For this purpose teams are
managing the performance and fitness data of the players and they evaluate this data to
select right person for right place in the team. Then they forecast the performance of the
team using historic data of players in different matches.
It was a old approach to use only evaluation of the individual performance of any player
here we does not relate the effect of individual performance in the team performance. So the
prediction of the match was very tricky because we were not used the overall effect of
performance by a player of the team, a department of the team (e.g. defense, forward,
midfielder, goalkeeper). It is very easy to maintain the data of performance but to relate the
data in the context of performance is difficult in different tools. We can use curve fitting and
other different statics tool but these tools are failed to maintain the best error free result.
Many models are based on mathematical calculations with dynamic programming, mainly in
these calculation they use some mathematical concept such as probability with dynamic way
and they predict result dynamically. For example WASP (Winning and Score Prediction) is
a technique which is used in cricket at New Zealand for prediction of score and winning
percentage of any team during the match.
In this model we include the performance rating of team in four main parts of team. In these
parts we include defense, goalkeeping, midfielder and forward. we collect the historic data
of performance of players and final result of match in season 2013 to 2014 in Barclays
premier league in match by match manner we randomly choose 106 matches .Then all these
data sets are implemented in training of neural network with performance of players in
different sectors as input and possible result of match as output. Then we can provide some
test set for known matches and if predicted result is same as real then model is ready to use.
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
3.2 Designing ANN models
Designing ANN models follows a number of systemic procedures. In general, there are
five basics steps: (1) Collecting data, (2) Preprocessing data, (3) Building the network, (4)
Train, and (5) test performance of model as shown in Figure.
Collecting data
Preprocessing data
Building the network
Train
Test Performance
Figure 3.1 Designing ANN model
Data Collection
Collecting and preparing sample data is the first step in designing ANN
models.as input. First we need data set. For this problem we choose result of Premier League
season 2011-12 . Because of great number of matches we randomly sampled 106 results.
Each result has 8 input and 3 output attributes. Input attributes are:
1. Home team goalkeeper rating
2. Home team defence rating
3. Home team midfield rating
4. Home team attack rating
5. Visitor team goalkeeper rating
6. Visitor team defiance rating
7. Visitor team midfield rating
8. Visitor team attack rating
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
Output attributes are:
1. Home team wins
2. Draw
3. Visitor team wins
Data pre-processing
After data collection, three data preprocessing procedures are conducted to train the
ANNs more efficiently. These procedures are:
a. Solve the problem of missing data,
b. Normalize data
c. Randomize data.
The missing data are replaced by the average of neighboring values during the same year.
Normalization procedure before presenting the input data to the network is generally a
good
practice, since mixing variables with large magnitudes and small magnitudes will
confuse the learning algorithm on the importance of each variable and may force it
to finally reject the variable with the smaller magnitude.
All input attributes are have integer values that can be very distant from each other. For
example, goalkeeper rating can have maximum value of 100, and attack rating can have
maximum of 300. In that case attack rating will influence more on problem than goalkeeper.
To prevent that we will normalize data set using Max-Min normalization formula.
B = (A - min(A)) / (max(A) - min(A)) * ( D - C ) + C
Where B is the standardized value, and D and C determines the range in which we want our
value to be. In this case, D = 1 and C = 0.
Normalized values are saved in PremierLeagueResults.txt file because they will be used for
training and testing neural network
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
Building the network
At this stage, the designer specifies the number of hidden layers, neuron in each
layer,
transfer function, weight/bias learning function, and performance function. In this
work multilayer feedforward network is used with Levenberg-Marquardt learning
algorithm.
Training the network
In order to neural network learn the problem we need traaining data set. The training data
set consists of input signals assigned with corresponding target (desired output). The
neural network is then trained using one of the supervised learning algorithms, which
uses the data to adjust the network's weights and thresholds so as to minimize the error in
its predictions on the training set. If the network is properly trained, it has then learned to
model the (unknown) function that relates the input variables to the output variables, and
can subsequently be used to make predictions where the output is not known.
In general, if you use a neural network, you will not know the exact nature of the
relationship between inputs and outputs – if you knew the relationship, you would model
it directly. The other key feature of neural networks is that they learn input/output
relationship through training. There are two types of training used in neural networks,
with different types of networks using different types of training. These are supervised
and unsupervised training, of which supervised is the most common. In supervised
learning, the network user assembles a set of training data. The training data contains
examples of inputs together with the corresponding outputs, and the network learns to
infer the relationship between the two. In other words, supervised learning is used for
classification. For an unsupervised learning rule, the training set consists of input training
patterns only. Unsupervised learning, on other hand, is used for clustering.
Training set can be created in two ways. You can either create training set by entering
elements as input and desired output values of neurons in input and output label, or you
can create training set by choosing an option load file. The first method of data entry is
time consuming, and there is also a risk to make a mistake when entering data. Since we
already have training set we will choose second way.
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
Testing the network
The next step is to test the performance of the developed model. At this stage unseen data are
exposed to the model. 147 samples are given to the network for the testing and evaluate the
performance. In order to evaluate the performance of the developed ANN model quantitatively
and verify whether there is any underlying trend in performance of ANN models, statistical
analysis involving the correlation coefficient (R), the Mean square error (MSE) were
conducted.
3.3 Creating the ANN model
In this model we used the Neuroph Studio which is Java Neural Network development
environment on top of NetBeans Platform based on Neuroph framework. It has been licensed
under the Common Development and Distribution License (CDDL) . we are using Neuroph
Studio 201207171143 (version 7.5) and java version 1.7 ,Java HotSpot(TM) Server VM 11.0 .
Complete procedure of the project is given below –
a. After normalizing all data we can start with Neuroph Studio. First we will
create new Neuroph project then a new project is created and it will appear in
the 'Projects' window, in the top left corner of Neuroph studio .
b. In order to neural network learn the problem we need training data set. The
training data set consists of input signals assigned with corresponding target
(desired output). The neural network is then trained using one of the
supervised learning algorithms, which uses the data to adjust the network's
weights and thresholds so as to minimize the error in its predictions on the
training set. If the network is properly trained, it has then learned to model the
(unknown) function that relates the input variables to the output variables, and
can subsequently be used to make predictions where the output is not known.
c. In general, if you use a neural network, you will not know the exact nature of
the relationship between inputs and outputs – if you knew the relationship,
you would model it directly. Training set can be created in two ways. You
can either create training set by entering elements as input and desired output
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
values of neurons in input and output label, or you can create training set by
choosing an option load file. The first method of data entry is time consuming,
d. and there is also a risk to make a mistake when entering data. Since we
already have training set we will choose second way
e. After the network is trained, we click 'Test', in order to see the total error, and
all the individual errors with respect to our training sets.
f. To predict the future value of any match we have to create a specific test set in
specified manner (same number and format of input and output variables as in
training data set )
Some pictures are given below that are showing the process
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
Figure 3.2 Creation of neuroph project
Figure 3.3 Creation of training set
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
Figure 3.4 training set with data
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
Figure 3.5 Learning parameters before training
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
Figure 3.6 network error graph
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
Figure 3.7 weights for different neurons for a training set
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
Figure 3.8 weights histogram for different neurons
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
3.4 Equivalent Java code for model
package predictingtheresultoffootballmatch;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.Arrays;
import org.neuroph.core.NeuralNetwork;
import org.neuroph.core.learning.DataSet;
import org.neuroph.core.learning.DataSetRow;
import org.neuroph.nnet.MultiLayerPerceptron;
import org.neuroph.nnet.learning.MomentumBackpropagation;
import org.neuroph.util.TrainingSetImport;
import org.neuroph.util.TransferFunctionType;
public class PredictingTheResultofFootballMatch {
/* @param args the command line arguments */
public static void main(String[] args) {
String trainingSetFileName = "PremierLeagueResults.txt";
int inputsCount = 8;
int outputsCount = 3;
System.out.println("Running Sample");
System.out.println("Using training set " + trainingSetFileName);
// create training set
DataSet trainingSet = null;
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING try {
trainingSet = TrainingSetImport.importFromFile(trainingSetFileName, inputsCount,
outputsCount, ",");
} catch (FileNotFoundException ex) {
System.out.println("File not found!");
}
catch (IOException | NumberFormatException ex) {
System.out.println("Error reading file or bad number format!");
} // create multi layer perceptron
System.out.println("Creating neural network");
MultiLayerPerceptron neuralNet = new
MultiLayerPerceptron(TransferFunctionType.SIGMOID, 8, 30, 3);
// set learning parametars
MomentumBackpropagation learningRule = (MomentumBackpropagation)
neuralNet.getLearningRule();
learningRule.setLearningRate(0.3);
learningRule.setMomentum(0.7);
// learn the training set
System.out.println("Training neural network...");
neuralNet.learn(trainingSet);
System.out.println("Done!");
// test perceptron
System.out.println("Testing trained neural network");
testPredictingTheResultofFootballMatch(neuralNet, trainingSet);
}
public static void testPredictingTheResultofFootballMatch(NeuralNetwork nnet, DataSet
dset) {
for (DataSetRow trainingElement : dset.getRows()) {
nnet.setInput(trainingElement.getInput());
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING nnet.calculate();
double[] networkOutput = nnet.getOutput();
System.out.print("Input: " + Arrays.toString(trainingElement.getInput()));
System.out.println(" Output: " + Arrays.toString(networkOutput));
}
}
}
RESULT AND INTERPOLATION
Department of Computer Science & Engg., FIET Bareilly
CHAPTER 4
APPLICATION OF NEURAL NETWORK IN FORECASTING
This chapter presents the best achieved results for feedforward backpropogation network
model. The network has 8 neurons in input layer, 2 to 25 neurons in hidden layer and 3 s
neuron for the predicted match result output. There are 106 samples are given to the network,
which divided by tool in 3 parts, training, validation and testing. After the completion of
training the Performance graph is given as:
Figure 4.1 network error graph for training set
The results are very impressive for the mean square error, and same as for the regression
coefficient as shown in the graphs
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
Figure 4.2 testing result for a testset
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
CONCLUTION AND FUTURE WORK
Department of Computer Science & Engg., FIET Bareilly
CHAPTER 5
APPLICATION OF NEURAL NETWORK IN FORECASTING
5.1 Conclusion
In this model we simply create a architecture that can predict the result of a football match
(best result for Barclays premier League) . As we know that we can forecast and predict the
future values using neural network techniques and historical data . So we used the same
concept and created a simple model for forecasting of match result , to achieve this we collect
historical data ( performance rating of players for specific match with result of match ) from
genius website of premier league ( Barclays Premier League ) . After data collection we
proceed the preprocessing of data in this data cleaning, data normalization etc takes place .
Normalized data set was used to train the network algorithm , for this we create several
network architectures and and perform training to each architecture then after training and
testing of each network architecture we check the performance of each architecture .
Performance mainly based on minimum error (MSE).
After these steps we can say that which architecture gave us the best result, so we select that
best architecture and perform prediction for any upcoming match in the league. We can use
same architecture for other leagues and matches but the accuracy of the prediction may be
different from specified league.
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
Neural network is very healthy area for forecasting or prediction. Neural network contains several
different algorithms and techniques to forecast the values in various fields. Different techniques are
based on historical data and different algorithms of training can easily forecast the values. We know
that each technique of network needs data in its manner ( i.e. some format) for perfect prediction
and minimum error.
In this model we use 8 input variables which are mainly based on performance of players of both
teams but we know that the performance of player not only depends upon their rating, it also
depends on the place (match venue) , fitness of player ,weather , strategy of opposition etc . As we
know that we use historical data for these attributes so it is difficult to collect the data for above
given attributes from historic data but if we start collection of data of these attributes in present
then we can easily and effectively create a perfect forecasting or prediction model in future .
It is possible to create a model that can give best result for any match and any team, location,
situation with historical data of different attributes. We also know that for perfect result also based
on training algorithm and network architecture so we can say that modification and creation of new
algorithm and architecture can also change the result and can give best result . So we can perform
research work on it in future for various fields where forecasting and prediction can be applied.
Department of Computer Science & Engg., FIET Bareilly
5.2 Future work
APPLICATION OF NEURAL NETWORK IN FORECASTING
Department of Computer Science & Engg., FIET Bareilly
REFERENCESE
APPLICATION OF NEURAL NETWORK IN FORECASTING
[1] Kahn, J. (2003) \Neural Network Prediction of NFL Football Games, “World Wide Web
electronic publication, homepages.cae.wisc.edu/~ece539/project/f03/kahn.pdf.
[2] Official website of Barclays Premier league ,for dats collection ,Available:
http://www.premierleague.com/en-gb/matchday/results.html?paramComp_100=true&view
=.dateSeason, accessed on 5 April, 2014.
[3] M. Hajek, “Neural Networks” 2005, Available:
www.sciag.ukzn.ac.za/comp/notes/NeuralNetworks2005.pdf accessed on 4 May 2014
[4] Martin T. Hagan and Mohammad B. Menhaj “Training Feedforward Networks with the
Marquardt Algorithm”, IEEE Transactions on Neural Networks, Vol. 5, No.6, November
1994, pp 989-993.
[5] Alan McCabe, "An Artificially Intelligent Sports Tipper," in Proceedings : 15th Australian
Joint Conference on Artificial Intelligence (2002)
[6] A. P. Rotshtein, M. Posner and A. B. Rakityanskaya, “Football Predictions Based on a
Fuzzy Model with Genetic and Neural Tuning," Cybernetics and Systems Analysis Journal
(2005)
[7] P. D. Sreekanth, N. Geethanjali, P. D. Sreedevi, Shakeel Ahmed, N. Ravi Kumar and P. D.
Kamala Jayanthi “Forecasting groundwater level using artificial neural networks” , Current
science, vol. 96, no. 7, 10 april 2009.
[8] David, John A.; Pasteur, R. Drew; Ahmad,M. Saif; and Janning, Michael C. (2011)
"NFL Prediction using Committees of Arti_cial Neural Networks," Journal ofQuantitative
Analysis in Sports: Vol. 7: Iss.2, Article 9.
[9] ThePredictionTracker, (2007), “FootballPredictions", World Wide Web electronic
publication,www.thepredictiontracker.com
[10] Howard Demuth, Mark Beale, “Neural Network Toolbox, For use with Matlab” , User’s
Guide, Version 4, The MathWorks
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
[11] V. Nourani, A. Hosseini Baghanam, F. Daneshvar Vousoughi, M.T. Alami
“Classification of groundwater level data using som to develop ann-based forecasting model” ,
International Journal of Soft Computing and Engineering (IJSCE), ISSN: 2231-2307,
Volume-2, Issue-1, March 2012.
[12] Heesung Yoon, Seong-Chun Jun, Yunjung Hyun , Gwang-Ok Bae, Kang-Kun Lee, “A
comparative study of artificial neural networks and support vector machines for predicting
groundwater levels in a coastal aquifer”, Journal of Hydrology 396 (2011), pp 128–138.
[13] Faridah Othman and Mahdi Naseri “Reservoir inflow forecasting using artificial neural
network”, International Journal of the Physical Sciences Vol. 6(3), , 4 February, 2011, pp.
434-440.
[14] Purna C. Nayak,Y.R. Satyaji Rao and K. P. Sudheer “Groundwater Level Forecasting in
a Shallow Aquifer Using Artificial Neural Network Approach”, Water
Resources Management (2006) 20: pp 77–90.
[15] http://www.mathworks.in/matlabcentral/fileexchange/19997-neural-network-for-pattern-
recognition-tutorial/content/NN_tutorial.m, accessed on 11 May 2013.
[16] Chi Leung Patrick Hui. (11 April, 2011). “Artificial Neural Networks – Application”,
ISBN 978-953-307-188-6, InTec.
Department of Computer Science & Engg., FIET Bareilly
APPLICATION OF NEURAL NETWORK IN FORECASTING
Department of Computer Science & Engg., FIET Bareilly