ics 586: neural networks dr. lahouari ghouti information & computer science department
TRANSCRIPT
ICS 586: Neural NetworksICS 586: Neural Networks
Dr. Lahouari GhoutiDr. Lahouari Ghouti
Information & Computer Science DepartmentInformation & Computer Science Department
ICS 586: Neural NetworksICS 586: Neural Networks
First Semester 2008/2009 (081)First Semester 2008/2009 (081) Instructor:Instructor:Dr. Lahouari GhoutiDr. Lahouari Ghouti
Information and Computer Science DepartmentInformation and Computer Science Department
Email: [email protected]: [email protected]
Office: Building 22 – Room 128Office: Building 22 – Room 128
Tel: 1922Tel: 1922
Grading PolicyGrading Policy
Task Weight
Four Quizzes 10%
Homeworks 10%
Research Paper (Lecture) Presentation 10%
One Major Exam 15%
Final Exam 25%
Research Project [Proposal 5% - Final Report + Prototype 25% - Class Presentation 10%]
40%
Tentative ScheduleTentative Schedule• Introductory Meeting [Introduction to Neural Networks]• Single Layer Perceptron• Multilayer Perceptron• ADALINE• The LMS Algorithm• Backpropagation Learning• Overfitting, Cross-Validation, and Early Stopping• Simple Recurrent Networks• Pattern Classification (Guest Speaker?)• Radial Basis Functions• Support Vector Machines• Competitive Learning and Kohonen Nets• Hebbian Learning• Principal Components Analysis (PCA)• Adaptive Principal Components Extraction (A Student)• Non-Negative Matrix factorization (A Student)• Hopfield Networks and Boltzmann Machines• Bayesian Networks (A Student)• Hidden Markov Models (A Student)•Extreme Learning Machines (A Student)
Programming EnvironmentProgramming Environment
We will be using MATLAB in this course.If you know, that’s fineIf you do not, you will need to learn it.
Getting Started With ANNsGetting Started With ANNs
Foundations of Neural Computation:
Understand the operation of single neurons or small neural circuits.
Detailed biophysical models of nerve cells (receptors, ion channels, membrane voltage), and collections of cells.
Varieties of “Neural Networks” ResearchVarieties of “Neural Networks” Research
1- Neuronal Modeling
2- Computational Neuroscience
3- Connectionist / Parallel Distributed Processing (PDP) Models
4- Artificial Neural Networks (ANNs)
Connectionist (PDP) ModelingConnectionist (PDP) Modeling
Model human cognition in a brain-like way:
• Massively parallel constraint satisfaction.
• Distributed activity patterns instead of symbols.
• Models are fairly abstract.
ANN LandscapeANN Landscape
Artificial Neural NetworksArtificial Neural Networks
Models:Simple, abstract, .neuron-like. computing elements; local computation.
Applications: Pattern recognition, adaptive control, time series prediction.
This is where the money gets made!
Reference: Pomerleau 1993: ALVINN
Artificial Neural Networks: The BeginningsArtificial Neural Networks: The Beginnings
W. S. McCulloch and W. Pitts (1943) Logical calculus of the ideas immanent in nervous activity. Philosophy of Science 10(1), 18-24.
Warren McCulloch Walter Pitts
Revolutionary Idea: think of neural tissue as circuitry performing mathematical computations!
The McCulloch-Pitts NeuronThe McCulloch-Pitts Neuron
Linear weighted sum of inputs:
Learning rule:
Nonlinear, possibly stochastic transfer function:
Transfer function g(x)
i
iixwnetact
netactgy
iw
What to do now?What to do now?
• Check WebCT for course syllabus + soft copy of textbook
• Start learning MATLAB if you do not know it!
• Select a topic you want to present in the class from the tentative schedule (first-come first-serve basis!)
• Select your time slot to discuss with me your project. The earlier you start, the better off you will be.