Neural NetworksJohn Riebe and Adam Profitt

What is a neuron?

Weights: Weights are scalars that multiply each input elementSummer: The summer sums the input elements, PR, together with the biasBias: A bias is a number that is added to the total from the summer

Translation Function:A translation function is one of many specific functions used inneural networking.

PR Elements of the input vectorW Weights∑ Summerb Biasn Sum of all P elements and bƒ Translation Functiona Output

Layers of the Neural Network

There are only three different types of layers in a network:•The Input Layer

•Moves the input vectors into each neuron of the first hidden layer

•The Hidden Layers•Performs the bulk of the computations in most networks•Hidden layers are not always required

•The Output Layer•Each neuron in the output layer outputs it’s own result

Translation Functions

Types of Neural Networks

Perceptrons:1. Used to classify data.2. Applies the hard-limit transfer function.3. Usually does not have any hidden layers.

Linear Filters1. Used to solve linearly separable problems.2. Applies the linear transfer function.

Backpropagation1. Generally has only one hidden layer.2. Can solve any reasonable problem.3. Hidden layers use sigmoid translations, outputs use the linear transfer function

Training NeuronsTraining a network sets the biases and weights in each neuron

Training algorithms1. Gives a network an input2. Receives the output3. Calculates error between output and target4. Adjusts weights and biases5. Goes back to step 1

Each time the algorithm goes through the steps is called an epoch.Most networks go through many epochs.

To train a network you need:•A network•An input•A target vector

There are many different types of training algorithms. To name a few:•Levenberg-Marquardt•BFGS quasi-Newton•Bayesian regularization•One step secant •Random order incremental

MatlabApplication

Create a feed-forward network

Syntaxnet = newffnet = newff(PR,[S1 S2...Si],{TF1 TF2...TFi})

Descriptionnet = newff creates a new network with a dialog box.

newff(PR,[S1 S2...Si],{TF1 TF2...TFi}) takes,PR - R x 2 matrix of min and max values for R input elements. Si - Size of ith layer, for Nl layers.TFi - Transfer function of ith layer, default = 'tansig'.

The newff Function

The train Function

Trains a neural network

Syntaxnet = train(net,P,T)

Descriptiontrain trains a network.train(net,P,T) takes,net - Neural network object. P - Network inputs.T - Network targets, default = zeros.

The sim Function

Syntaxsim(net,P)

Descriptionsim simulates neural networks.sim(net,P) takes,net - Network. P - Network inputs.

The sim function simulates a neural network.This function feeds the network the input, P, and displays the results.

Transfer Functions Revisited

Transfer functions:•Hard-Limit

a = hardlim(n)•Linear

a = purelin(n)

•Log-Sigmoida = logsig(n)

•Tan-Sigmoida = tansig(n)

Outputs either a 1 or a 0

Outputs the scaled and summed input

Squeezes the input to between 0 and 1

Squeezes the input to between -1 and 1

The Baum-Haussler Rule

The Baum-Haussler Rule is one of the most useful rules for neural networks.

Nhidden ≤ (Ntrain • Etolerance) / (Npts + Noutputs)

This rule helps you determine the maximum number of neurons you will needfor your network to function properly.

This is NOT a law: it will not work in all situations.Sometimes you just have to use another method.

Bibliography

Demuth, Howard and Mark Beale. “Neural Network Toolbox User’s Guide.“ 1992-2003URL: http://www.mathworks.com/access/helpdesk/help/toolbox/nnet/nnet.shtml