learning process

Learning ProcessCS/CMPE 537 Neural Networks

CS/CMPE 537 - Neural Networks (Sp 2004/2005) - Asim Karim @ LUMS

LearningLearning?Learning is a process by which the free parameters of a neural network are adapted through a continuing process of stimulation by the environment in which the network is embeddedThe type of learning is determined by the manner in which the parameter changes take placeTypes of learningError-correction, memory-based, Hebbian, competitive, BoltzmannSupervised, reinforced, unsupervised


Learning ProcessAdapting the synaptic weightwkj(n + 1) = wkj(n) + wkj(n)


Learning AlgorithmsLearning algorithm: a prescribed set of well-defined rules for the solution of a learning problemIn the context of synaptic weight updating, the learning algorithm prescribes rules for wLearning rulesError-correctionMemory basedBoltzmannHebbianCompetitiveLearning paradigmsSupervisedReinforcedSelf-organizing (unsupervised)


Error-Correction Learning (1)ek(n) = dk(n) yk(n)

The goal of error-correction learning is to minimize a cost function based on the error functionLeast-mean-square error as cost functionJ = E[0.5kek2(n)]E = expectation operatorMinimizing J with respect to the network parameters is the method of gradient descent


Error-Correction Learning (2)How do we find the expectation of the process?We avoid its computation, and use an instantaneous value of the sum of squared errors as the error function (as an approximation)(n) = 0.5kek2(n)Error correction learning rule (or delta rule)wkj(n) = ek(n)xj(n) = learning rateA plot of error function and weights is called an error surface. The minimization process tries to find the minimum point on the surface through an iterative procedure.


Memory-based Learning (1)All (or most) of the past experiences are stored explicitly in memory of correctly classified input-output examples: {(xi, di)}i = 1, NGiven a test vector xtest , the algorithm retrieves the classification of the xi closest to xtest in the training examples (and memory)IngredientsDefinition of what is closest or local neighborhoodLearning rule applied to the training examples in the local neigborhood


Memory-based Learning (2)Nearest neigbor ruleK-nearest neighbor ruleRadial-basis function rule (network)


Hebbian Learning (1)Hebb, a neuropsychologist, proposed a model of neural activation in 1949. Its idealization is used as a learning rule in neural network learning.Hebbs postulate (1949)If the axon of cell A is near enough to excite cell B and repeatedly or perseistently takes part in firing it, some growth process or metabolic change occurs in one or both cells such that As efficiency as one of the cells firing B is increased.


Hebbian Learning (2)Hebbian learning (model of Hebbian synapse)If two neurons on either side of a synapse are activated simultaneously, then the strength of that synapse is selectively increasedIf two neurons on either side of synapse are activated asynchronously, then that synapse is selectively weakened or eliminatedProperties of Hebbian synapseTime-dependent mechanismLocal mechanismInteractive mechanismCorrelational mechanism


Mathematical Models of Hebbian Learning (1)General form of Hebbian rulewkj(n) = F[yk(n), xj(n)]F is a function of pre-synaptic and post-synaptic activities.A specific Hebbian rule (activity product rule)wkj(n) = yk(n)xj(n) = learning rateIs there a problem with the above rule?No bounds on increase (or decrease) of wkj


Mathematical Models of Hebbian Learning (2)Generalized activity product rulewkj(n) = yk(n)xj(n) yk(n)wkj(n)Orwkj(n) = yk(n)[cxk(n) - wkj(n)]where c = / and = positive constant


Mathematical Models of Hebbian Learning (3)


Mathematical Models of Hebbian Learning (4)Activity covariance rulewkj(n) = cov[yk(n), xj(n)]= E[(yk(n) y)(xj(n) x)]where = proportionality constant and x and y are respective meansAfter simplificationwkj(n) = {E[yk(n)xj(n)] xy}


Competitive Learning (1)The output neurons of a neural network (or a group of output neurons) compete among themselves for being the one to be active (fired)At any given time, only one neuron in the group is activeThis behavior naturally leads to identifying features in input data (feature detection)Neurobiological basisCompetitive behavior was observed and studied in the 1970sEarly self-organizing and topographic map neural networks were also proposed in the 1970s (e.g. cognitron by Fukushima)


Competitive Learning (2)Elements of competitive learningA set of neuronsA limit on the strength of each neuronA mechanism that permits the neurons to compete for the right to respond to a given input, such that only one neuron is active at a time


Competitive Learning (3)


Competitive Learning (4)Standard competitive learning rulewji = (xi wji) if neuron j wins the competition0 otherwiseEach neuron is allotted a fixed amount of synaptic weight which is distributed among its input nodesi wji = 1 for all j


Competitive Learning (5)


Boltzmann Learning Stochastic learning algorithm based on information-theoretic and thermodynamic principlesThe state of the network is captured by an energy function, EE = -1/2 k j wkjsisk where sj = state of neuron j [0, 1] (i.e. binary state)Learning processAt each step, choose a neuron at random (say kj) and flip its state sk (to - sk ) by the following probabilityw(sk -> -sk) = (1 + exp(-Ek/T)]-1 The state evolves until thermal equilibrium is achieved


Credit-Assignment ProblemHow to assign credit and blame for a neural networks output to its internal (free) parameters ?This is basically the credit-assignment problemThe learning system (rule) must distribute credit or blame in such a way that the network evolves to the correct outcomesTemporal credit-assignment problemDetermining which actions, among a sequence of actions, are responsible for certain outcomes of the networkStructural credit-assignment problemDetermining which internal components behavior should be modified and by how much


Supervised Learning (1)


Supervised Learning (2)Conceptually, supervised learning involves a teacher who has knowledge of the environment and guides the training of the networkIn practice, knowledge of the environment is in the form of input-output examplesWhen viewed as a intelligent agent, this knowledge is current knowledge obtained from sensorsHow is supervised learning applied?Error-correction learningExamples of supervised learning algorithmsLMS algorithmBack-propagation algorithm


Reinforcement Learning (1)Reinforcement learing is supervised learning in which limited information of the desired outputs is knownComplete knowledge of the environment is not available; only basic benefit or reward informationIn other words, a critic rather than a teacher guides the learning processReinforcement learning has roots in experimental studies of animal learningTraining a dog by positive (good dog, something to eat) and negative (bad dog, nothing to eat) reinforcement


Reinforcement Learning (2)Reinforcement learning is the online learning of an input-output mapping through a process of trail and error designed to maximize a scalar performance index called reinforcement signalTypes of reinforcement learningNon-associative: selecting one action instead of associating actions with stimuli. The only input received from the environment is reinforcement information. Examples include genetic algorithms and simulated annealing.Associative: associating action and stimuli. In other words, developing a action-stimuli mapping from reinforcement information received from the environment. This type is more closely related to neural network learning.


Supervised Vs Reinforcement Learning


Unsupervised Learning (1)There is no teacher or critic in unsupervised learningNo specific example of the function/model to be learnedA task-independent measure is used to guide the internal representation of knowledgeThe free parameters of the network are optimized with respect to this measure


Unsupervised Learning (2)Also known as self-organizing when used in the context of neural networksThe neural network develops an internal representation of the inputs without any specific informationOnce it is trained it can identify features in the input, based on the task-independent (or general) criterion


Supervised Vs Unsupervised Learning


Learning TasksPattern associationPattern recognitionFunction approximationControlFilteringBeamforming


Adaptation and Learning (1)Learning, as we know it in biological systems, is a spatiotemporal processSpace and time dimensions are equally significantIs supervised error-correcting learning spatiotemporal?Yes and no (trick question )

Stationary environmentLearning one time procedure in which environment knowledge is built-in (memory) and later recalled for useNon-stationary environmentAdaptation continually update the free parameters to reflect the changing environment


Adaptation and Learning (2)


Adaptation and Learning (3)e(n) = x(n) - x(n)where e = error; x = actual input; x = model output

Adaptation needed when e not equal to zeroThis means that the knowledge encoded in the neural network has become outdated requiring modification to reflect the new environmentHow to perform adaptation?As an adaptive control systemAs an adaptive filter (adaptive error-correcting supervised learning)


Statistical Nature of LearningLearning can be viewed as a stochastic processStochastic process? when there is some element of randomness (e.g. neural network encoding is not unique for the same environment that is temporal)Also, in general, neural network represent just one form of representation. Other representation forms are also possible.Regression modeld = g(x) + where g(x) = actual model; = statistical estimate of error



learning process

Documents

neural network learning

neural networks sp

lumshebbian learning

asim karim

learning problemin

learning processcscmpe

lumserrorcorrection

lumsmemorybased learning