Widrow-Hoff Learning
Outline
• 1 Introduction• 2 ADALINE Network• 3 Mean Square Error• 4 LMS Algorithm• 5 Analysis of Converge• 6 Adaptive Filtering
Introduction
• In 1960, Bernard Widrow and his doctoral student Marcian Hoff introduced the ADALINE (ADAptive LInear NEuron)network and LMS(Least Mean Square) algorithm.
Perceptron Network
• Figure: a=hardlim(Wp+b)
ADALINE Network
• Figure: a=purelin(Wp+b)=Wp+b
Single ADALINE
decision boundary
Mean Square Error
Mean Square Error(conti.)
Mean Square Error(conti.)
Error analysis
𝐹 (𝐱 )=𝑐+𝐝𝑇 𝐱+𝟏𝟐𝐱𝑻 𝐀𝐱
Error analysis(conti.)
d = -2h and A = 2R
= 0
definite
Example 1
Example 1(conti.)
Example 1(conti.)
Approximate Steepest Descent
Approximate Gradient
Approximate Gradient(conti.)
Approximate Gradient(conti.)
LMS Algorithm
LMS Algorithm (conti.)
Example 2
Example 2(conti.)
, W(0)=
Example 2(conti.)
Example 2(conti.)
Example 2(conti.)
Analysis of Convergence
Analysis of Convergence(conti.)
Analysis of Convergence(conti.)
Example 3
Perceptron rule V.S. LMS algorithm
Perceptron rule V.S. LMS algorithm(conti.)
Perceptron rule V.S. LMS algorithm(conti.)
Perceptron rule V.S. LMS algorithm(conti.)
Adaptive Filtering
Tapped Delay Line
Adaptive Filter
Adaptive Noise Cancellation
