dr.-ing. erwin sitompul president university lecture 1 introduction to neural networks and fuzzy...

Dr.-Ing. Erwin SitompulPresident University

Lecture 1

Introduction to Neural Networksand Fuzzy Logic

President University Erwin Sitompul NNFL 1/1

http://zitompul.wordpress.com


Textbook:“Neural Networks. A Comprehensive Foundation”, 2nd Edition, Simon Haykin, Prentice Hall, 1999.

“Fuzzy Systems Theory and Its Application”, Toshiro Terano et. al., Academic Press, 1992.

TextbooksIntroduction to Neural Networks and Fuzzy Logic


Grade PolicyIntroduction to Neural Networks and Fuzzy Logic

Final Grade = 20% Homework + 20% Quizzes + 30% Midterm Exam + 30% Final Exam + Extra Points

Homeworks will be given in fairly regular basis. The average of homework grades contributes 20% of final grade.

Written homeworks are to be submitted on A4 papers, otherwise they will not be graded.

Homeworks must be submitted on time. If you submit late,< 10 min. No penalty10 – 60 min. –20 points> 60 min. –40 points

There will be 3 quizzes. Only the best 2 will be counted. The average of quiz grades contributes 20% of final grade.

Midterm and final exam schedule will be announced in time. Make up of quizzes and exams will be held one week after the

schedule of the respective quizzes and exams.


Grade PolicyIntroduction to Neural Networks and Fuzzy Logic

The score of a make up quiz or exam, upon discretion, can be multiplied by 0.9 (the maximum score for a make up is then 90).

Extra points will be given if you solve a problem in front of the class. You will earn 1, 2, or 3 points.

Lecture slides can be copied during class session. It is also available on internet. Please check the course homepage regularly.

http://zitompul.wordpress.com

• Heading of Written Homework Papers (Required)


Introduction to Neural Networks

Validation:• Generally, means confirming that a

product or service meets the needs of its users.

• Testing whether the mathematical model is good enough or not to describe the empirical phenomenon.

Theoretical modeling

Experimentalmodeling validation

measurement

Empirical Phenomenon

Mathematical Model

Data

IntroductionNeural Networks


Experimental Modeling Experimental modeling consists of three steps:

1. The choice of model class2. The choice of model structures (number of parameters, model

order, time delay)3. The calculation of the parameters and time delay.

The model may be chosen to be linear, nonlinear, or multi locally-linear.

A-priori (prior, previous) knowledge of the system to be modeled is required in most cases.

Artificial Neural Networks (or simply Neural Networks) offers a general solution for experimental modeling.



Experimental Modeling Using Neural Networks A neural network is a massively-parallel distributed processor

made up of simple processing unit, which has natural propensity for storing experiential knowledge and making it available for use.

It resembles the brain in two respects:1. Knowledge is acquired by the network from its environment

through a learning process.2. Interneuron connection strengths, known as synaptic weights,

are used to store the acquired knowledge.



Biological and Artificial Neuron

soma

synapse

dendrite

axon

Structure of Biological neuron

ky( )f

1kw

2kw

kmwkb

1x

mx

2x

1

net

Structure of Artificial neuron

( )ky f net1

m

ki i ki

net w x b

Activation function



Activation Function

y

x

( )y f x x

1

y

x

22( ) 1

1 xy f x

e

1y

x

1( )1 xy f xe

Any continuous (differentiable) function can be used as an activation function in a neural network.

The nonlinear behavior of the neural networks is inherited from the used nonlinear activation functions.

y

x

2( ) axy f x e

Linearfunction

Tangentsigmoidfunction

Logarithmicsigmoidfunction

Radial basisfunction



Network ArchitecturesSingle layer feedforward network(Single layer perceptron)

Inputlayer

Outputlayer

Multilayer feedforward network(Multilayer perceptron)

Inputlayer

Outputlayer

Hidden

layer



Network Architectures

1z

1z

1z1z

Diagonal recurrent networks

Inputlayer

Outputlayer

Hiddenlayer

Inputlayer

Outputlayer

Hiddenlayer

Delay element in arecurrent network

Fully recurrent networks



Network Architectures

1z

1z

1z

1z

1z

1z

Elman’s recurrent networks Jordan’s recurrent networks



Preparation AssignmentIntroductionNeural Networks

Ensure yourself to install Matlab 7 in your computer, along with Matlab Simulink, Control System Toolbox, and Fuzzy Logic Toolbox.

Quizzes, Midterm Exam, and Final Exam will be computer-based.

dr.-ing. erwin sitompul president university lecture 1 introduction to neural networks and fuzzy...

Documents