lin jie, jin xiao-gang, yang jian-gang (institute of artificial intelligence, zhejiang university,...

LIN Jie, JIN Xiao-gang, YANG Jian-gang(Institute of Artificial Intelligence, Zhejiang

University, Hangzhou 310027, China) Date of publication: Mar, 2004

Presented by: Bhuban M Seth, Joydip DattaUnder the guidance of: Prof. Dr. Pushpak Bhattacharyya

MotivationUltimate goal of Artificial Neural Net is to

imitate a human brain.But human brain is too complex to

understand.Question: What is a consciousness and How it

is generated in brain? Is there any hierarchical organization in the

brain?How can we incorporate these newfound

insights of human brain into an ANN?

Understanding the brain(Different Approaches)Taylor (1994): Relational MindRakovic (1997): hierarchically organized

and interconnected paradigm for information processing inside the brain.

Vitiello (2003): Quantum ModelRennie et. al. (2002): Evoked potential

Where all these things leads to?Cognitive processes are carried out at different

levels in the brain. Higher levels may be reduced to lower

levels.Thus, higher levels of complex brain

functions require a number of neural modules to cooperate together.

Example: We see a rose, smell the fragrance and remember some memory… this way a conscious state of mind emerges in a thinking process.

Level 1: Physical Mnemonic LayerPhysical Mnemonic Layers (PML) capture input

from external senses and produce a feature vector (patterns) from them. Many modular PMLs run in parallel.

There may be two kinds of external inputs:Arousal Inputs: Reach only up to recognition Layer

– Do not take part in Associative RecognitionAware Inputs: Reaches Abstract Thinking Layer and

may take part in Associative RecognitionThe feature vectors are input to recognition layer

Level 2: Recognition LayerIt is a searching tree composed of layered

storage neurons.It receives a pattern from the PML.

Level 3: The Global WorkspaceIt belongs to the Abstract Thinking Layer.It describes the state of Consciousness.It can project the abstract information it has

and mobilize different parts of the brain.This global availability of information define

the conscious state of mind.

Recognition LayerDivided into levelsEach level consists of number of knowledge

clustersInput is the pattern formed by Physical

Mnemonic Layer (PML)This pattern is compared with stored

patterns at all levels

Recognition LayerIf the pattern is similar to some existing

patterns it will be recognized.

Else, the pattern will be saved (New neurons will be created)

Similarity is measured by resonant coefficient

Definition:It is the feature vector of a pattern.

Inherent frequency of a neuron group k that

memorizes a knowledge pattern can be described by

the weights from one neuron in the group to other

members, as K=[wl, w2 ..... wi .... ].

Similar PatternsSimilarity of two patterns A, B are determined by

their Resonant Coefficient R(A,B). The resonant coefficient is a kind of delta

similarity relation satisfying the following properties:Reflexive: R(A, A)=1Symmetric: R(A,B) = R(B,A)And

1 - | R(A,C) – R(B,C) | >= R(A,B) --(Upper bound) R(A,B) >= max(0, R(A,C)+R(B,C)-1 ) --(Lower bound)

ExampleSuppose a series of four-dimension patterns

Pi (i=0,1,2 .... ,9) formed by PML models enter RL. Say, Pi is the binary format of i as

P3=[0,0,1,1], P5=[0,1,0,1], P1=[0,0,0,1]. We can define resonant coefficient R(Pi, Pj)

asR(Pi,Pj) = 1 – (XOR(Pi and Pj)/ 4)

Then R(P0,P0)=1,R(P0,P1)=0.75,R(P0,P2)=0.75 , R(P0,P3)=0.5 and so on.

Resonant SpaceIt is a representation of pattern showing

similarity between them.Definition: It is a space of patterns to which

any other pattern can be compared to evaluate resonant coefficient.

A pattern P is represented in resonant space by a single point, whose projection on an axis represents the resonant coefficient between the pattern corresponding to the axis and the pattern P.

Resonant Space(contd…)

The resonant space formed by patterns P0 and P5

Cntd…Consider a resonant space Rn with n patterns

Pi and the resonant coefficient R(Pi, Pj) between any two patterns Pi and Pj .

From the resonant space formed by n patterns , a pattern Pm may be represented on Rn as:

where is the unit vector along Pi axis.

Threshold in Recognition LayerDefinition:

The thresholds exist in RL corresponding to different levels (numbered from zero to TOP): to>tL>tL+l>tTOP, patterns are clustered at those levels. For example, at level L, two patterns ~ belong to the same cluster if and only if tL>f(u,v)>tL+I. There also exists a highest threshold tmax and two patterns are recognized to be the same if f(u,v)>tmax.

Abstract Thinking LayerIt can associatively compare (and recognize)

different types of inputs. It can broadcast it’s contents to the nervous

system as a whole allowing different modules to interact.

E.g. The ATL cat take input from the auditory and the vision subsystem and while associatively recognizing the inputs it can mobilize the olfactory subsystem.

Abstract Thinking Layer

Abstract Thinking LayerThe ATL is an Bi-directional Backpropagation

network (BBP).A1 and A2 are both input to of the BBP.The computation is interleaved: only one-way

learning is going on at a particular interval.The structure (no of neurons in different layers of

the BBP) of the ATL may vary depending on the inputs.

A subset of the neurons are excited at a time while rest of them are inhibited. This in general represents the consciousness.

Consciousness in ATLDynamic workspace

states are self sustained and follow one another in a continuous stream, without external help

Consciousness generation requires a stable activation loop.

The system enters a stable state V* (attractor) when there no more change in the state possible:V* = V(t+∆t) = V(t), ∆t > 0

Time span threshold in GWEstablishment of stable state requires a

minimal duration.There is a temporal span of successive

workspace states.If patterns from several subsystems appear in

the ATL longer than some Time span threshold then a conscious state emerges.

Otherwise they can not establish a self sustained activation loop – They are called sub-consciousness.

ConclusionDifferent levels exists in consciousness

generation process.Partial recognition layer threshold helps to

form clusters within RL unconsciously.Strong pattern that persists for more than a

time span threshold can accomplish associative recognition resulting in consciousness.

Background StudyWikipedia articles on: Brain, Human Brain, Cerebral

Cortex, Hippocampus etc (different parts of brain), Neuron, Action Potential, Depolarizing, Hyperpolarizing, Inhibited Neurons, Excited Neurons, Axon Hillock, Back-propagation, Neural Back-propagation, Resting potential, Layered perceptron, MLP, Electrical Inductance, Electrical Resonance etc.

Hierarchical Learning in Neural Network: http://www.cs.iastate.edu/~baojie/acad/current/hnn/hnn.htm

A Bi-Directional Multilayer PerceptronM. JEDRA, A. EL OUARDIGHI, A. ESSAID and M. LIMOURILaboratoire Conception & Systèmes, Faculté des Sciences, Avenue Ibn Batouta, B.P. 1014, Rabat10 000, Morocco, e-mail: jedra@fsr.ac.ma