artificial neural networks...2013/09/23 · layer and 1 output layer. x 1 x 2 z 0 0 0 0 1 1 1 0 1 1...
TRANSCRIPT
Scholastic Video Book Series
Artificial Neural Networks
Part 2
(Multilayer Perceptrons)
(with English Narrations)
http://scholastictutors.webs.com
(http://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdf)
1
©Scholastic Tutors (Sep, 2013) ISVT 911-0-20-130923-1
ARTIFICIAL NEURAL NETWORKS
International Baccalaureate (IB)
2
Artificial Neural Networks - #2 Classification using Multi Layer Perceptron Model
http://scholastictutors.webs.com
(ANN-002)
http://youtu.be/K5HWN5oF4lQ Click here to see the video
3
Limitations of Perceptrons
http://scholastictutors.webs.com
• If there is a solution to be found then the single layer perceptron learning algorithm will find it.
• Please see the video http://youtu.be/S3iQgcoQVbc.
• It can separate classes that lie either side of a straight line easily.
• But in reality division between classes are much more complex.
• Take for example the classical exclusive-or (XOR) problem.
• XOR logic function has two inputs and one output.
(1,1) (0,1)
(0,0) (1,0) x1
x2 x1 x2 z
0 0 0
0 1 1
1 0 1
1 1 0
4
• We consider this as a problem in which we want the perceptron to learn to solve: – output 1 if x1 is on and x2 is off, or is x2 is on and x1 is off,
otherwise output a 0.
• This appears a simple problem, but there is no linear solution, and this problem is linearly inseparable.
• Hence single-layer perceptrons can not solve this problem.
XOR Problem
http://youtu.be/K5HWN5oF4lQ Click here to see the video
5
Multilayer Perceptrons
• Multilayer perceptrons have been applied to solve some difficult problems.
• This consist of input layer, one or more hidden layer and an output layer.
• The training of the network is done by the highly popular algorithm known as the error back-propagation algorithm.
• This algorithm is based on the error-correcting learning rule.
• Basically there are two passes through the different layers of the network: forward pass and the backward pass. http://scholastictutors.webs.com
6
• Now consider the following network with two inputs, 1 hidden layer and 1 output layer.
x1 x2 z
0 0 0
0 1 1
1 0 1
1 1 0
b1
Output
w11
w21
w12
w22
w31
w32
Output
layer
Hidden
layer
Input
layer
Neuron 1
Neuron 2
Neuron 3
x1
x2
+1
b2 +1
b3
+1
XOR Problem – with two layers
http://scholastictutors.webs.com
7
• Verify that the network shown in the previous slide solves an XOR problem for the following two sets of parameters. Assume that the activation function, , in the neurons are threshold functions, where,
. – (1) w11= w12 = w21 = w22 = w32 = +1, w31 =-2, b1=-1.5 and b2=b3=-0.5
– (2) w11= w12 = w21 = w22 = w32 = -1, w31 =1, b1=1.5, b2=0.5 and b3=-0.5
1 if 0( )
0 if 0
vv
v
( )v
XOR Problem – with two layers
http://youtu.be/K5HWN5oF4lQ Click here to see the video
8
1 5 0 5
1 1
1 1
. .
Hw n
1
0
1
x n
1
2
1 5 0 5 1 11 5 1 1 0 5 0
1 1 0 00 5 1 1 0 5 1
1 1 1 1
. .. .
w ( )x( ). .
T
H
H T
H
yy n n n
y
3
31 1
32 2
1 1
0 5 2 1 0 0 5 1
1
w ( )x( ) . .
T
o T H
H
b
y n n n w y
w y
For [0,1] input:
(1) w11= w12 = w21 = w22 = w32 = +1, w31 =-2, b1=-1.5 and b2=b3=-0.5
XOR Problem – Verification
1 if 0( )
0 if 0
vv
v
http://scholastictutors.webs.com
1 5 0 5
1 1
1 1
. .
Hw n
1
0
0
x n
1
2
1 5 0 5 1 11 5 1 1 1 5 0
1 1 0 00 5 1 1 0 5 0
1 1 0 0
. .. .
w ( )x( ). .
T
H
H T
H
yy n n n
y
3
31 1
32 2
1 1
0 5 2 1 0 0 5 0
0
w ( )x( ) . .
T
o T H
H
b
y n n n w y
w y
For [0,0] input:
9
XOR Problem – Verification
1 if 0( )
0 if 0
vv
v
http://scholastictutors.webs.com
1 5 0 5
1 1
1 1
. .
Hw n
1
1
0
x n
1
2
1 5 0 5 1 11 5 1 1 0 5 0
1 1 1 10 5 1 1 0 5 1
1 1 0 0
. .. .
w ( )x( ). .
T
H
H T
H
yy n n n
y
3
31 1
32 2
1 1
0 5 2 1 0 0 5 1
1
w ( )x( ) . .
T
o T H
H
b
y n n n w y
w y
For [1,0] input:
10
XOR Problem – Verification
1 if 0( )
0 if 0
vv
v
http://scholastictutors.webs.com
1 5 0 5
1 1
1 1
. .
Hw n
1
1
1
x n
1
2
1 5 0 5 1 11 5 1 1 0 5 1
1 1 1 10 5 1 1 1 5 1
1 1 1 1
. .. .
w ( )x( ). .
T
H
H T
H
yy n n n
y
3
31 1
32 2
1 1
0 5 2 1 1 1 5 0
1
w ( )x( ) . .
T
o T H
H
b
y n n n w y
w y
For [1,1] input:
11
XOR Problem – Verification
1 if 0( )
0 if 0
vv
v
Therefore using the given set of weights and bias we have obtained the desired output for the all the four inputs.
1 5 0 5
1 1
1 1
. .
Hw n
1
0
1
x n
1
2
1 5 0 5 1 11 5 1 1 0 5 1
1 1 0 00 5 1 1 0 5 0
1 1 1 1
. .. .
w ( )x( ). .
T
H
H T
H
yy n n n
y
3
31 1
32 2
1 1
0 5 1 1 1 0 5 1
0
w ( )x( ) . .
T
o T H
H
b
y n n n w y
w y
For [0,1] input:
(2) w11= w12 = w21 = w22 = w32 = -1, w31 =1, b1=1.5, b2=0.5 and b3=-0.5
12
XOR Problem – Verification
1 if 0( )
0 if 0
vv
v
http://scholastictutors.webs.com
1 5 0 5
1 1
1 1
. .
Hw n
1
0
0
x n
1
2
1 5 0 5 1 11 5 1 1 1 5 1
1 1 0 00 5 1 1 0 5 1
1 1 0 0
. .. .
w ( )x( ). .
T
H
H T
H
yy n n n
y
3
31 1
32 2
1 1
0 5 1 1 1 0 5 0
1
w ( )x( ) . .
T
o T H
H
b
y n n n w y
w y
For [0,0] input:
13
XOR Problem – Verification
1 if 0( )
0 if 0
vv
v
http://scholastictutors.webs.com
1 5 0 5
1 1
1 1
. .
Hw n
1
1
0
x n
1
2
1 5 0 5 1 11 5 1 1 0 5 1
1 1 1 10 5 1 1 0 5 0
1 1 0 0
. .. .
w ( )x( ). .
T
H
H T
H
yy n n n
y
3
31 1
32 2
1 1
0 5 1 1 1 0 5 1
0
w ( )x( ) . .
T
o T H
H
b
y n n n w y
w y
For [1,0] input:
14
XOR Problem – Verification
1 if 0( )
0 if 0
vv
v
http://youtu.be/K5HWN5oF4lQ Click here to see the video
1 5 0 5
1 1
1 1
. .
Hw n
1
1
1
x n
1
2
1 5 0 5 1 11 5 1 1 0 5 0
1 1 1 10 5 1 1 1 5 0
1 1 1 1
. .. .
w ( )x( ). .
T
H
H T
H
yy n n n
y
3
31 1
32 2
1 1
0 5 1 1 1 0 5 1
0
w ( )x( ) . .
T
o T H
H
b
y n n n w y
w y
For [1,1] input:
15
XOR Problem – Verification
1 if 0( )
0 if 0
vv
v
Therefore using the given set of weights and bias we have obtained the desired output for the all the four inputs.
0 [-0.5]=[0]
International Baccalaureate (IB)
16
Artificial Neural Networks - #2 Classification using Multi Layer Perceptron Model
http://scholastictutors.webs.com
(ANN-002)
END of the Book
If you like to see similar solutions to any Mathematics problems please contact us at: [email protected] with your request.
http://youtu.be/K5HWN5oF4lQ Click here to see the video
Videos at: http://www.youtube.com/user/homevideotutor
17
(http://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdf)
Scholastic Video Book Series
Artificial Neural Networks
Part 2
(with English Narrations)
(END)
©Scholastic Tutors (Sep, 2013) ISVT 911-0-20-130923-1