pattern recognition: statistical and neural
DESCRIPTION
Nanjing University of Science & Technology. Pattern Recognition: Statistical and Neural. Lonnie C. Ludeman Lecture 19 Oct 26, 2005. Lecture 19 Topics. 1. Structures of Optimal Statistical Classifiers 2. Neural Network History 3. Biological Neural Networks - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/1.jpg)
1
Pattern Recognition:Statistical and Neural
Lonnie C. Ludeman
Lecture 19
Oct 26, 2005
Nanjing University of Science & Technology
![Page 2: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/2.jpg)
2
Lecture 19 Topics
1. Structures of Optimal Statistical Classifiers
2. Neural Network History
3. Biological Neural Networks
4. Modified McColloch Pitts Model – Example
5. Artificial Neural Element - Definition
![Page 3: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/3.jpg)
3
Structures from Statistical Decision Theory and Perceptron Algorithm
(a). Two Class Likelihood Ratio test
(b) Structures to Motivate Neural Networks
(c) Two Class General Linear
(d). N Class General minimum P(error)
(e) N Class Gaussian
![Page 4: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/4.jpg)
4
if l(x) > T decide x from C1 < T decide x from C2 = T decide x randomly
Decision Rule
between C1 and C2
![Page 5: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/5.jpg)
5
if f(x) > T<C2
C1Decision Rule
![Page 6: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/6.jpg)
6
if f(x) > T<C2
C1Decision Rule
![Page 7: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/7.jpg)
7
if p(x | Ck) > p(x | Cj ) for all j ≠ k decide Ck
Decision Rule
![Page 8: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/8.jpg)
8(e) M Class Gaussian
![Page 9: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/9.jpg)
9
These Structures will be seen to be similar to the structures used in designs with Neural Networks
![Page 10: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/10.jpg)
10
Important Events in Neural Networks History
1943 McColloch Pitts Model
1958 Perceptron Algorithm- Rosenblatt
1960-62 ADALINE – Widrow and Hoff
1969 Minsky and Papert- Limitations
1980’s Grossberg , Hopfield, and Rumelhart – Backpropagation algorithms, Adaptive Resonance Theory
Period of Revival
1990’s Maturation of Field
![Page 11: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/11.jpg)
11
IMPORTANT BOOKS
1990 Artificial Neural Systems- Jacek M. Zurada
1992 Neural Networks and Fuzzy Systems- Bart Kosco
1994 Neural Networks: A Comprehensive Foundation- Simon Haykin
![Page 12: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/12.jpg)
12
Neural Networks
Biological (Real)
Mathematical (Artificial)
How do you tell them apart ???
Squeeze them !!!
Excite them !!!
![Page 13: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/13.jpg)
13
Biological Neuron
![Page 14: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/14.jpg)
14
Action Potential
![Page 15: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/15.jpg)
15
![Page 16: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/16.jpg)
16
Biological Neural Network
![Page 17: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/17.jpg)
17
End Bulb Connection
![Page 18: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/18.jpg)
18
Integration at axon-dendrite junction
![Page 19: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/19.jpg)
19
Modified McColloch Pitts Neuronal Model
(Threshold Logic Unit (TLU) )
![Page 20: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/20.jpg)
20
What can we do with a Modified McColloch-Pitts Model ???
Question
Answer
Surprisingly we can model all logical expressions !!!
![Page 21: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/21.jpg)
21
T = n - 1/2 T = 1/2
Implementation of Logical AND and OR
![Page 22: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/22.jpg)
22
Since we can model logical AND, OR and
NOT we can model all logical expressions
Implementation of Logical NOT
-1/2
![Page 23: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/23.jpg)
23
Example – Implementation of a given logic expression
Given:
Implement f(x) using Modified McColloch-Pitts Neurons
![Page 24: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/24.jpg)
24
Solution:
-1/2
-1/2
-1/2
-1/2
2.5
3.5
T= 1/2
-1
![Page 25: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/25.jpg)
25
Artificial Neural Element (ANE)
Node
netInput Vector
![Page 26: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/26.jpg)
26
net = w1x1 + w2x2 + …+ wnxn + wn+1
= wTx
y = f(net) = f( wTx)
Artificial Neural Element Mathematical Model
Nonlinear Activation Function
Linear Operation on x
NonlinearOperation on x
![Page 27: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/27.jpg)
27
Artificial Neural Element Nodal Representation
Vector Notation
![Page 28: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/28.jpg)
28
Some Activation Functions
![Page 29: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/29.jpg)
29
![Page 30: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/30.jpg)
30
Hyperbolic Tangent Activation Function
f(net) Equivalent Form
Definition
![Page 31: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/31.jpg)
31
Hyperbolic Activation Function
![Page 32: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/32.jpg)
32
![Page 33: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/33.jpg)
33
Non Monitonic Activation Functions can be very Useful
Examples:
Potential Function Approach
Radial Basis Functions
![Page 34: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/34.jpg)
34
Summary Lecture 19
1. Structures of Optimal Statistical Classifiers
2. Neural Network History
3. Biological Neural Networks
4. Modified McColloch Pitts Model – Example
5. Artificial Neural Element - Definition
![Page 35: Pattern Recognition: Statistical and Neural](https://reader035.vdocument.in/reader035/viewer/2022062222/5681589c550346895dc5fadd/html5/thumbnails/35.jpg)
35
End of Lecture 19