neural logic
TRANSCRIPT
SEMINAR ON:
ANALOG VLSI NEURAL NETWORK
Presented by: Monica Swain -1359017 Manisha charathi -1359016
CONTENTS Introduction Basic neuron design Artificial neuron model Algorithm associated with ANN Implementing ANN WITH VLSI Neuron circuit description Operation of vlsi artificial NN Advantage and disadvantage Application Conclusion Reference
INTRODUCTION An Artificial Neural Network (ANN) is an information
processing paradigm that is inspired by the way biological nervous systems, such as the brain, process information.
The key element of this paradigm is the novel structure of the information processing system. It is composed of a large number of highly interconnected processing elements (neurones) working in unison to solve specific problems. ANNs, like people, learn by example.
An ANN is configured for a specific application, such as pattern recognition or data classification, through a learning process. Learning in biological systems involves adjustments to the synaptic connections that exist between the neurones. This is true of ANNs as well.
BASIC NEURON DESIGN
ARTIFICIAL NEURAL NETWORKA neural network is a powerful data modeling tool that is able to capture and represent complex input/output relationships.
weights
COMPARASION BETWEEN Biological
Soma
Dendrite
Axon
Synapse
Artificial
Neuron
Input
Output
Weight
TYPES OF ARTIFICIAL NETWORK NETWORK
1. Single layered Neural Network:
TYPES OF ARTIFICIAL NEURAL NETWORK2. Multi layer perceptron (MLP) It consist of three layers: -> Input layer. -> Hidden layer. -> output layer.
TYPE OF ARTIFICIAL NEURAL NETWORK HIDDEN LAYER
ALGORITMS1.FEED FORWARD PROPAGATION
Information flow is unidirectional. Data is presented to the input. Passed on to the hidden layer. Information is distributed. Processing is parallel. No cycle or loop present.
Feedforward
Inpu
ts
Out
puts
ALGORITHMS2.BACK PROPAGATION ALGORITHM Back propagation consists of the repeated application
of the following two passes:-> Forward pass (Network activation)-> Backward pass (Error propagation) Back propagation adjusts the weights of the NN in order to
minimize the network total mean squared error
Network activation
Back propagation
ALGORITHM Consider a network of three
layers. Let us use i to represent nodes
in input layer, j to represent nodes in hidden layer and k represent nodes in output layer.
wij refers to weight of connection between a node in input layer and node in hidden layer.
The following equation is used to derive the output value Yj of node j
Weight Wij
j
k
Input i
jXe
1
1Yj
ALGORITHM● The error of output neuron k after the
activation of the network on the n-th training example (x(n), d(n)) is:
ek(n) = dk(n) – yk(n) ● The network error is the sum of the squared
errors of the output neurons:
● The total mean squared error is the average of the network errors of the training examples.
N
1nN
1AV (n)EE
(n)eE(n) 2k
ALGORITHM● The Backprop weight update
rule is based on the gradient descent method: − It takes a step in the
direction yielding the maximum decrease of the network error E.
− This direction is the opposite of the gradient of E.
ijijij www
ALGORITHM
ijij w
-w
E
VLSI NEURAL SYSTEM CHARACTERISTICS. The main characteristics of the analog neural
network chip are:
reduced dimensions and power consumption
high parallelism of data processing;
high speed of computations compatible with the precision required;
high error tolerance;
VLSI NEURAL CIRCUIT AS A WHOLEThe circuit consist of: Synapse Summing element Activation function
SYNAPSE DESIGN Neural network learn by modifying
weight(synapse)
REQUIREMENT: To get a storagable synaptic weight. Need for an alterable, non -volatile, on chip
analog memory. Use of a sample and hold circuit. Need for a capacitive storage.
PROGRAMMABLE SYNAPSE DESIGNPROBLEM: Relative chip area is large. Excess power consumption Limit the no of synapse . Reduce the computational area. Suffers from short storage time.
SOLUTION: To use a programmable threshold voltage
transistor.
PROGRAMMABLE THRESHOLD VOLTAGE TRANSISTOR
CHARACTERISTICS: Based on standard MOS transistor. Small size and power consumption. Slow memory decay. Compatibility with standard fabrication
process. Typically have one or more gate layers used
to store charge. Charge shift the threshold voltage. Generally of two types.
TYPES OF PROGRAMMABLE THRESHOLD VOLTAGE TRANSISTOR
1.FLOATING GATE MOS TRANSISTOR
TYPES OF PROGRAMMABLE THRESHOLD VOLTAGE TRANSISTOR
CHARACTERISTICS:
Addition of a new gate “floating gate” between original gate and channel.
Original gate is called control gate.
Floating gate is an isolated conducting island.
Electrons put in the floating gate stay for a long time.
FLOATING GATE MOS TRANSISTOR
When electrons are present on the floating gate:
Attract holes from p substrate.
Enhanced holes change the conductivity of the channel.
No gate volt but electrons present
Case-1 P-doped floating gate transistor
FLOATING GATE MOS TRANSISTOR
When the floating gate is devoid of electrons:
Holes are scattered in the channel.
Need of a high control gate voltage
Case-2 P-doped floating gate transistor
PROGRAMMING OF FLOATING GATE MOS TRANSISTOR
PROGRAM (Write operation)
Putting electrons into the floating gate
It means logic 0
Accomplished by:
i) FN tunnelling ii) Channel hot electron
injection
PROGRAMMING OF FLOATING GATE TRANSISTOR
ERASE OPERATION Removing the charge from the floating gate
IT means logic ‘1’
Accomplished by: i) Quantum tunnelling
WORKING PRINCIPLE High voltage at the
control gate.
Electrons at the floating gate.
Holes accumulated at the channel region.
Barrier is reduced.
Tunnelling is enhanced.
Electrons in the floating gate as well as control gate.
DISADVANTAGE OF FGMOS Defect in the tunnel
oxide.
Electrical stressing due to programming.
Difficult to scale down the oxide.
SOLUTION Data retention is not
sensitive to oxide defect.
Programming speed increase.
FN tunnelling consumes less power.
Thin tunnelling oxide.
To use MNOS( Metal nitride oxide sillicon)
COMPARING TWO TRANSISTOR
PROGRAMMABLE SYNAPSE :DESIGN Concept is to provide a
small change in weight M1 and M2 operate in
triode region. Equal Vgs Equal Vth I proportional to(Vgs2-
Vgs1). Vgs1< Vgs2(+ve
weight) Vgs1> Vgs2 (-ve weight) Adaptive w achieved
using control unit
PROGRAMMABLE SYNAPSE:DESIGN CONTROL UNITConsist of: 8 bit input (B0-B7) 1:8 analog MUX A mux with control code
(S0,S1,S2) Enable terminal ‘E’ to
enable the mux. Output current ‘I’
applied to the CMOS switch.
NEURON CIRCUIT : DESIGN SUMMING ELEMENT Designed in weak inversion
region M4-M5 operate in sub
threshold region. Common mode input
should be wide. Input voltage of p-channel: Vss< Vcommon< (Vdd-Vsd(sat)-Vsg,p) Second stage consist of: CS amp (M9) CS amp(M10) Vss+ Vds< Vout< (Vdd-Vsd(sat).p)
Vdd
Vss
NEURON CIRCUIT :DESIGN TANH ACTIVATION
FUNCTION Input is
differential(M13-M14)
The current generates ‘I’ proportional to difference between two Vds of the differential pair that represent tanh function.
COMPLETE NEURON DESIGN
ADVANTAGE DISADVANTAGE Massively
parallelism neural system.
Full potential of sillicon can be exploited.
Fault tolerance Low power Real world interface Computational
power being derived from massive parallelism
Mismatch Non-ideal behaviour
of the circuit
APPLICATION Clustering
Prediction system
Artificial system building
Optical character recognization
Image processor
OPTICAL CHARACTER RECOGNIZATION
IMAGE PROCESSING
CONCLUSION In this paper, a programmable analog VLSI neural
network has been introduced. The synaptic weight circuit was realized using four-MOS transistors and SR technique.
Moreover, the summing element and the activation function have been designed in sub threshold region. Such system was realized in a standard’0.8pm CMOS technology and operated with f 1V power supply.
This system has been used for recognizing handwritten words, and zip codes.
REFERENCE 1. E, A. Viltoz , “Analog VLSI implementation of neural networks,” in IEEE Int.
Symp. Circuits Syst. Proc., 1990, PP. 2524-2527. 2. Alan Murray and Lionel Tarassenko , “Analogue Neural VLSI- A Pulse Stream
Approach,” Chapman & Hall Publishing Company, 1994. 3. J. B. Lont and W. Guggenbuhl , “Analog CMOS implementation of multilayer
perceptron with nonlinear synapses,” IEEE Trans. on neural networks, Vol. 3, No. 3, PP. 457-462, May 1992.
4. T. Morishita et ul , “A BiCMOS analog neural network with dyniunically updated weights,” in IEEE Int. Solid-Statc Circuits Conf. Proc., 1990, PP. 142- 143.
5. M A. Abo -Elsoud, “Analog Circuits for electronic neural network,” The Proc. of 35* Midwest Symp. on Circuits and
6. Andreas G. Andreou . Kwabena A. Boahen , Philippe 0. Pouliquen , Aleksandra Pavasovic Robert E. Jenkins and Kim Strohbehn , “Current-Mode Sub threshold MOS Circuits for Analog VLSI Neural Systems,” IEEE Trans. on neural networks,
7. Mead C., “Analog VLSI and Neural Systems,” Addison-Wesley Publishing Company, 1989.
8. Johan H. Huijsing , Ron Hogervorst , and KIass -Jan de Langen , “Low-power Low-voltage VLSI Operational Amplifier Cells,” IEEE Trans. on Circuits and Syst.-I: Fundamental theory and applications, Vol. 42, No. I 1, PP. 841 -852, November 1995.
THANK YOU