12 artificial neural networks

8/3/2019 12 Artificial Neural Networks

1/6

ARTIFICIAL NEURAL NETWORKS

S.R.K.R. ENGINEERING COLLEGE

COMPUTER SCIENCE AND ENGINEERING

B.S.N.V SANTHOSH KAMALIII/IV B.TECH,E-MAIL:[email protected]

P.V.PHANI KUMAR,III/IV B.TECH,E-MAIL:[email protected]


2/6

Abstract:

This paper mainly concentrates on details ofArtificial Neural Networks. At first, anintroduction to neural networks is given .Theproperties of neural networks vs. conventional

computers is explained. The uses of ArtificialNeural Networks (ANN) in real time operationsand fault tolerant systems are given. The varioustypes of architectures of neural networks such asFeed forward and Feedback neural networks areexplained and demonstrated. A detailed view ofa simple neuron and perceptrons is alsoprovided.

Various network layers are specified briefly andproperties of neural networks are also explained.The connection between the artificial and thereal thing is also explained. The transferfunction for neural networks is explainedbriefly.

Finally, Applications of neural networks likeANNs in pattern classification associativememories, optimization, and vector quantizationare described. The mathematical modelsinvolved are presented and demonstrated.

Neural Network:

An artificial neural network is a physical cellularsystem which can acquire, store and utilize

experimental knowledge. The knowledge is inthe form of stable stagesOr mappings embedded in networks that can berecalled in response to the presentation of uses.The inspiration for such systems comes fromneuron science.

An Artificial Neural Network(ANN) is an information processing paradigmthat is inspired by the way biological nervoussystems, such as the brain, process information.The key element of this paradigm is the novelstructure of the information processing system.It is composed of a large number of highlyinterconnected processing elements (neurons)working in unison to solve specific problems.An ANN is configured for a specific application,such as pattern recognition or data classification,through a learning process. Learning inbiological systems involves adjustments to the

synaptic connections that exist between theneurons. This is true of ANNs as well.

Use of neural networks:

Neural networks, with their remarkable ability toderive meaning from complicated or imprecisedata, can be used to extract patterns and detecttrends that are too complex to be noticed byeither humans or other computer techniques. Atrained neural network can be thought of as an"expert" in the category of information it hasbeen given to analyze. This expert can then beused to provide projections given new situationsof interest and answer "what if" questions.Other advantages include:

1. Adaptive learning: An ability to learn howto do tasks based on the data given fortraining or initial experience.

2. Self-Organization: An ANN can create itsown organization or representation of theinformation it receives during learning time.

3. Real Time Operation: ANN computationsmay be carried out in parallel, and specialhardware devices are being designed andmanufactured which take advantage of thiscapability.

4. Fault Tolerance via Redundant InformationCoding: Partial destruction of a networkleads to the corresponding degradation of

performance.

Neural networks versusconventional computers :

Neural networks take a different approach toproblem solving than that of conventionalcomputers. Conventional computers use analgorithmic approach i.e. the computer follows aset of instructions in order to solve a problem.Unless the specific steps that the computer needsto follow are known the computer cannot solvethe problem. That restricts the problem solving

capability of conventional computers toproblems that we already understand and knowhow to solve. But computers would be so muchmore useful if they could do things that we don'texactly know how to do.Neural networks process information in a similarway the human brain does. The network iscomposed of a large number of highlyinterconnected processing elements (neurons)


3/6

working in parallel to solve a specific problem.Neural networks learn by example. They cannotbe programmed to perform a specific task. Theexamples must be selected carefully otherwiseuseful time is wasted or even worse the networkmight be functioning incorrectly. The

disadvantage is that because the network findsout how to solve the problem by itself, itsoperation can be unpredictable.On the other hand, conventional computers use acognitive approach to problem solving; the waythe problem is to solved must be known andstated in small unambiguous instructions. Theseinstructions are then converted to a high levellanguage program and then into machine codethat the computer can understand. Thesemachines are totally predictable; if anythinggoes wrong is due to a software or hardwarefault.Neural networks and conventional algorithmiccomputers are not in competition butcomplement each other. There are tasks aremore suited to an algorithmic approach likearithmetic operations and tasks that are moresuited to neural networks. Even more, a largenumber of tasks, require systems that use acombination of the two approaches (normally aconventional computer is used to supervise theneural network) in order to perform at maximumefficiency.

Human Brain Learning:In the human brain a typical neuron collectssignals from others through a host of finestructures called dendrites. The neuron sends outspikes of electrical activity through a long, thinstand known as an axon, which splits intothousands of branches. At the end of eachbranch, a structure called a synapse converts theactivity from the axon into electrical effects thatinhibit or excite activity from the axon intoelectrical effects that inhibit or excite activity inthe connected neurons. When a neuron receivesexcitatory input that is sufficiently largecompared with its inhibitory input, it sends aspike of electrical activity down its axon.Learning occurs by changing the effectivenessof the synapses so that the influence of oneneuron on another changes.

Human Neurons to ArtificialNeurons:

We conduct these neural networks by firsttrying to deduce the essential features ofneurons and their interconnections. We thentypically program a computer to simulatethese features. However because our

knowledge of neurons is incomplete and ourcomputing power is limited, our models arenecessarily gross idealizations of realnetworks of neurons.

The neuron model

A simple neuron:

An artificial neuron is a device with many inputsand one output. The neuron has two modes ofoperation; the training mode and the usingmode. In the training mode, the neuron can betrained to fire (or not), for particular inputpatterns. In the using mode, when a taught inputpattern is detected at the input, its associatedoutput becomes the current output. If the input

pattern does not belong in the taught list of inputpatterns, the firing rule is used to determinewhether to fire or not.

Components of a neuron


4/6

A simple neuron

Architecture of neural networks:

1. Feed-forward networks:

Feed-forward ANNs (figure 1) allow signals totravel one way only; from input to output. Thereis no feedback (loops) i.e. the output of anylayer does not affect that same layer. Feed-forward ANNs tend to be straight forward

networks that associate inputs with outputs.They are extensively used in pattern recognition.This type of organization is also referred to asbottom-up or top-down.

2 .Feedback networks

Feedback networks (figure 1) can have signalstravelling in both directions by introducing loopsin the network. Feedback networks are verypowerful and can get extremely complicated.Feedback networks are dynamic; their 'state' ischanging continuously until they reach anequilibrium point. They remain at the

equilibrium point until the input changes and anew equilibrium needs to be found. Feedbackarchitectures are also referred to as interactive orrecurrent, although the latter term is often usedto denote feedback connections in single-layerorganizations.

Figure 4.1 An example of a simple feedforward network

Network layers:

The commonest type of artificial neural networkconsists of three groups, or layers, of units: alayer of "input" units is connected to a layer of"hidden" units, which is connected to a layer of"output" units. (See Figure 4.1)

The activity of the input units

represents the raw information that isfed into the network.

The activity of each hidden unit isdetermined by the activities of the inputunits and the weights on the connectionsbetween the input and the hidden units.

The behavior of the output unitsdepends on the activity of the hiddenunits and the weights between thehidden and output units.

This simple type of network is interesting

because the hidden units are free to constructtheir own representations of the input. Theweights between the input and hidden unitsdetermine when each hidden unit is active, andso by modifying these weights, a hidden unit canchoose what it represents.We also distinguish single-layer and multi-layerarchitectures. The single-layer organization, inwhich all units are connected to one another,constitutes the most general case and is of morepotential computational power thanhierarchically structured multi-layerorganizations. In multi-layer networks, units areoften numbered by layer, instead of following aglobal numbering.

Perceptrons:

The perceptron (figure 4) turns out to be anMCP model (neuron with weighted inputs) withsome additional, fixed, pre--processing. Unitslabeled A1, A2, Aj, Ap are called associationunits and their task is to extract specific,localized featured from the input images.Perceptrons mimic the basic idea behind themammalian visual system. They were mainly

used in pattern recognition even though theircapabilities extended a lot more.


5/6

Figure 4

The Learning Process:

The memorization of patterns and thesubsequent response of the network can becategorized into two general paradigms:Associative mapping in which the networklearns to produce a particular pattern on the setof input units whenever another particular

pattern is applied on the set of input units. Theassociative mapping can generally be brokendown into two mechanisms.

1. Auto-association: an input pattern isassociated with itself and the states of input andoutput units coincide. This is used to providepattern completition, i.e. to produce a patternwhenever a portion of it or a distorted pattern ispresented. In the second case, the networkactually stores pairs of patterns building anassociation between two sets of patterns.

2. Hetero-association: is related to two recallmechanisms:

2.1 Nearest-neighbor recall, where the outputpattern produced corresponds to the inputpattern stored, which is closest to the patternpresented.

2.2Iinterpolative recalls, where the outputpattern is a similarity dependent interpolation ofthe patterns stored corresponding to the patternpresented. Yet another paradigm, which is avariant associative mapping, is classification, i.e.when there is a fixed set of categories into which

the input patterns are to be classified.

Regularity detection in which unitslearn to respond to particular propertiesof the input patterns. Whereas inassociative mapping the network storesthe relationships among patterns, inregularity detection the response of eachunit has a particular 'meaning'. This type

of learning mechanism is essential forfeature discovery and knowledgerepresentation. Following the waylearning is performed, we candistinguish two major categories ofneural networks:

Fixed networks in which the weightscannot be changed, ie dW/dt=0. In suchnetworks, the weights are fixed a prioriaccording to the problem to solve.

Adaptive networks which are able tochange their weights, i.e. dW/dt not= 0.

Transfer Function: The behavior of anANN (Artificial Neural Network)depends on both the weights and theinput-output function (transfer function)that is specified for the units. This

function typically falls into one of threecategories: 1.linear 2.threshold,3.sigmoid

For the linear units, the output activityis proportional to the total weightedoutput.

Forthreshold units, the output are setat one of two levels, depending onwhether the total input is greater than orless than some threshold value.

Forsigmoid units, the output variescontinuously but not linearly as the input

changes. Sigmoid units bear a greaterresemblance to real neurons than do linear orthreshold units, but all three must be consideredrough approximations.

Theoretical properties of ANN:

Capacity:

Artificial neural network models have a propertycalled 'capacity', which roughly corresponds totheir ability to model any given function. It isrelated to the amount of information that can bestored in the network and to the notion of

complexity.

Generalization and statistics

In applications where the goal is to create asystem that generalizes well in unseen examples,the problem of overtraining has emerged. Thisarises in over complex or over specified systemswhen the capacity of the network significantlyexceeds the needed free parameters. There are


6/6

two schools of thought for avoiding thisproblem:The first is to use cross-validation and similartechniques to check for the presence ofovertraining and optimally select hyperparameters such as to minimize the

generalization error. The second is to use someform ofregularization. This is a concept thatemerges naturally in a probabilistic (Bayesian)framework, where the regularization can beperformed by putting a larger prior probabilityover simpler models; but also in statisticallearning theory, where the goal is to minimizeover two quantities: the 'empirical risk' and the'structural risk', which roughly correspond to theerror over the training set and the predicted errorin unseen data due to over fitting.

Applications of artificial neural networks:

1. Pattern classification

Recognisation of Olympic symbols

Recognisation of printed characters

Making an opening bid in contract bridgegame

2. Associative memories

Image pattern recall

Content addressable memory

Information retrieval

3. Optimization

Graph bipartition problem

Linear programming problem

Traveling salesman problem

Conclusion:

The computing world has a lot to gain fromneural networks. Their ability to learn byexample makes them very flexible and powerful.Furthermore there is no need to devise analgorithm in order to perform a specific task; i.e.there is no need to understand the internalmechanisms of that task. They are also very wellsuited for real time systems because of their fastresponse and computational times which are dueto their parallel architecture.

Neural networks also contribute to other areas ofresearch such as neurology and psychology.They are regularly used to model parts of livingorganisms and to investigate the internalmechanisms of the brain.Perhaps the most exciting aspect of neuralnetworks is the possibility that some day'conscious' networks might be produced. Thereare a number of scientists arguing thatconsciousness is a 'mechanical' property and that'conscious' neural networks are a realisticpossibility.

12 artificial neural networks

Documents