Software Simulation of a Self-organizing Learning

Array System

Janusz Starzyk & Zhen Zhu

School of EECS

Ohio University

Theme

SOLAR = Self-organizing Learning Array

Introduction to SOLAR

Software simulation

Performance of SOLAR

Introduction to SOLAR

SOLAR: Artificial neural networks (ANN)

Self-organizing structure

Re-configurable hardware

Introduction to SOLAR Basic frame of SOLAR:

A fixed lattice of processing units (neurons) Self-organization:

Interconnections among the units refined during learning

Software Simulation - SOLAR

Simulation tasks: Pre-processing of input data to SOLAR Behavior of a single neuron Network structure Classification Assembly of various networks

Software Simulation - SOLAR

Inputs & outputs of SOLAR:

sn

2n

1n

s1

21

11

X...X.X

....

X ...X X

s individual inputs

n dimensions of features SOLAR

s21 c ...c c

classification outputs

Software Simulation - SOLAR

Real world input data features X: Incomplete set – data missing Symbolic – unacceptable to neural

computation Unbalance weighted – needs to be equalized

Pre-processing: Calculate default substitutes for missing data Set continuous values to all symbols Rescaling

Software Simulation - SOLAR Missing data problem:

Find defaults for missing items in each individual inputs to minimize Mahalanobis distance.

Separate known items Xk, and missing items Xm X=[Xk, Xm].

Compute covariance matrix and its inversed matrix .

Partition matrix . Compute default Xm

1cc CD

cC

mmmk

kmkkc D,D

D,DD

kmkmm-1

m XDDX

Software Simulation - SOLAR Inputs & outputs of a single SOLAR

neuron:

p ro ce ss in g u n it

c la ss ifi ca tio n in fo rm a tio n sa ve d in s id e

s1

21

11 ...II I

s2

22

12 I ...I I

s21 O ...O Oc lo ck o u tp u ts

c lo ck in p u ts ,

m a y b e f ro m o th e rs ’ o u tp u ts

o r a ll va lid

d a ta in p u ts ,

m a y b e f ro m o th e rs ’ o u tp u ts O

o r o r ig in a l in p u ts X

Software Simulation - SOLAR

Behavior of a single SOLAR neuron: Output behaves a selected functions of

input. Unary input operations: O=Y(I1) or O=Y(I2). Binary input operations: O=Y(I1, I2). All the operations are redesigned arithmetic

operations.-Linear/ non-linear-Input/output range is set as 0-255.

Software Simulation - SOLAR

Unary input operations: Identical function : Y=IDENT(x)= Half function: Y=HALF(x)= Logarithm function:

Y=NLOG2(x)= Exponential function:Y= NEXP2(x)=

Binary input operations: Addition function: Y=NADD(x1,x2)= Subtraction function: Y=NSUB(x1,x2)=

32/2 x5)))),1(max(log,1(max(log 222 x

2/xx

)21(5.0 xx )0,21max( xx

Software Simulation - SOLAR Example: Y=NLOG2(x)= 5)))),1(max(log,1(max(log 222 x

Software Simulation - SOLAR

HOW does a neuron learn from training data and process on testing data? Each neuron chooses an operation and a threshold. The whole input space will be cut into 2 parts (subspaces).

Ex:

4805 . 39 )) 2 ( 2 ), 1 ( ( Input NEXP Input HALF NSUB

Software Simulation - SOLAR

Neuron learning Neurons learn from each other and

generates more complicated cuttings.

Software Simulation - SOLAR

Neuron learning In order to effectively separate different

classes, a neuron may choose from different configure options.

processing unit

I1

I2

I3

I4

Input clock 1 2 3

1 function and 1 threshold are selected

Software Simulation - SOLAR

Classification On each individual testing input data point,

some of or all the neurons are active in classification.

Neurons are activated with input clocks. Each neuron saves classification probabilities

based on subspace division.Ex: subspace 1 subspace 2

class 1 60% 10%class 2 10% 80%class 3 30% 10%

Software Simulation - SOLAR

Classification On each testing input data point, some

neurons have sufficient knowledge from learning and become eligible.

They vote on the classification of this point.

… classificationvoting

Software Simulation - SOLAR

Classification Several independent SOLAR networks form

an ensemble to vote on the same problem.

SOLAR 1

SOLAR 2

SOLAR 25

votingclassification

Performance Evaluation - SOLAR

An Australian credit card data set [1] is used to evaluate SOLAR performance.14 input features, 690 individuals, 2 classes

This data set is a typical classification problem and has been used to test other classic classification algorithms [2].

Performance Evaluation - SOLAR

Divide the data set into 10 groups randomly.

Run the simulation 10 times. Each time use 1 group for testing

the the remaining for training. Average the resultant classification

rate. Experimented on single SOLAR and

SOLAR ensemble.

Performance Evaluation - SOLAR

Method Miss Rate Method Miss Rate

CAL5 0.131 Naivebay 0.151DIPOL92 0.141 CASTLE 0.148

Logdisc 0.141 ALLOC80 0.201

SMART 0.158 CART 0.145C4.5 0.155 NewID 0.181

IndCART 0.152 CN2 0.204

Bprop 0.154 LVQ 0.197Discrim 0.141 Kohenen -

RBF 0.145 Quadisc 0.207Baytree 0.171 Default 0.440

ITule 0.137    

AC2 0.181 SOLAR 0.183

k-NN 0.181 ensemble 0.135

Performance Evaluation - SOLAR

Conclusion: Although SOLAR was not designed

with any particular purposes, it works well with several classification problems.

SOLAR behaviors are observed in this simulation.

References[1] Y. Liu, X. Yao and T. Higuchi,

“Evolutionary Ensembles with Negative Correlation Learning”, IEEE Trans. on Evolutionary Computation, Vol. 4, No. 4, Nov 2000.

[2] D. Michie, D. J. Spiegelhalter, and C. C. Taylor, “Machine Learning, Neural and Statistical Classification” London, U. K. Ellis Horwood Ltd. 1994