what is unsupervised learning? learning without a teacher. no feedback to indicate the desired...
TRANSCRIPT
![Page 1: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/1.jpg)
What is Unsupervised Learning?
• Learning without a teacher.
• No feedback to indicate the desired
outputs.
• The network must by itself discover the
relationship of interest from the input
data.
![Page 2: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/2.jpg)
The Nearest Neighbor Classifier
11 22
33 44
x(1) x(2)
x(3)x(4)
![Page 3: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/3.jpg)
The Nearest Neighbor Classifier
11 22
33 44
x(1) x(2)
x(3)x(4)
?Class
![Page 4: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/4.jpg)
The Hamming Networks
• Stored a set of classes represented by a set of binary prototypes.
• Given an incomplete binary input, find the class to which it belongs.
• Use Hamming distance as the distance measurement.
• Distance vs. Similarity.
![Page 5: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/5.jpg)
The Hamming Net
Similarity Measurement
MAXNET Winner-Take-All
x1 x2 xn
![Page 6: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/6.jpg)
The Hamming Distance
y = 1 1 1 1 1 1 1
x = 1 1 1 1 1 1 1
Hamming Distance = ?Hamming Distance = ?
![Page 7: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/7.jpg)
The Hamming Distance
y = 1 1 1 1 1 1 1
x = 1 1 1 1 1 1 1
Hamming Distance = ?Hamming Distance = ?
![Page 8: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/8.jpg)
y = 1 1 1 1 1 1 1
x = 1 1 1 1 1 1 1
The Hamming Distance
Hamming Distance = 3Hamming Distance = 3
![Page 9: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/9.jpg)
y = 1 1 1 1 1 1 1
The Hamming Distance
1 1 1 1 1 1 1
Sum=1
12( , ) (7 1) 3HD x y
x = 1 1 1 1 1 1 1
![Page 10: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/10.jpg)
The Hamming Distance
1 2( , , , ) {1, 1}Tm iy y y y y
1 2( , , , ) {1, 1}Tm ix x x x x
( , ) ?HD x y
( , ) ?Similarity x y
![Page 11: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/11.jpg)
The Hamming Distance
1 2( , , , ) {1, 1}Tm iy y y y y
12( , ) ( )THD m x y x y
12
1 12 2
( , ) ( )
T
T
Similarity m m
m
x y x y
x y
1 2( , , , ) {1, 1}Tm ix x x x x
![Page 12: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/12.jpg)
The Hamming Distance
1 2( , , , ) {1, 1}Tm iy y y y y
12( , ) ( )THD m x y x y
12
1 12 2
( , ) ( )
T
T
Similarity m m
m
x y x y
x y
1 2( , , , ) {1, 1}Tm ix x x x x
![Page 13: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/13.jpg)
The Hamming Net
Similarity Measurement
MAXNET Winner-Take-All
11 22 n1n1 nn
x1 x2 xm1 xm
11 22 n1n1 nn
y1 y2 yn1 yn
![Page 14: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/14.jpg)
The Hamming Net
Similarity Measurement
MAXNET Winner-Take-All
11 22 n1n1 nn
x1 x2 xm1 xm
11 22 n1n1 nn
y1 y2 yn1 yn
WS=?WS=?
WM=?WM=?
![Page 15: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/15.jpg)
The Stored Patterns
Similarity Measurement
MAXNET Winner-Take-All
11 22 n1n1 nn
x1 x2 xm1 xm
11 22 n1n1 nn
y1 y2 yn1 yn
WS=?WS=?
WM=?WM=?
1 2
1 2( , , ) { 1,
Stored patterns
with 1 }.
n
k k k k T km i
n , , ,
s s s s
s s s
s
1 2
1 2( , , ) { 1,
Stored patterns
with 1 }.
n
k k k k T km i
n , , ,
s s s s
s s s
s
kTk mSimilarity sxsx 21
21),( kTk mSimilarity sxsx 2
121),(
1 12 2
1
( , )m
k ki i
i
Similarity m x s
x s 1 12 2
1
( , )m
k ki i
i
Similarity m x s
x s
![Page 16: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/16.jpg)
The Stored Patterns
Similarity Measurement
1 2
1 2( , , ) { 1,
Stored patterns
with 1 }.
n
k k k k T km i
n , , ,
s s s s
s s s
s
1 2
1 2( , , ) { 1,
Stored patterns
with 1 }.
n
k k k k T km i
n , , ,
s s s s
s s s
s
),( kSimilarity sx ),( kSimilarity sxk
x1 x2 xm. . .
112ks 1
22ks
12
kms
m/2
kTk mSimilarity sxsx 21
21),(
kTk mSimilarity sxsx 21
21),(
1 12 2
1
( , )m
k ki i
i
Similarity m x s
x s 1 12 2
1
( , )m
k ki i
i
Similarity m x s
x s
![Page 17: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/17.jpg)
• Weight update: – Method 1: Method 2
In each method, is moved closer to il
– Normalize the weight vector to unit length after it is updated
– Sample input vectors are also normalized
– Distance
)( jlj wiw lj iw
jjj www
jjj www /
wj
il
il – wj
η (il - wj)
wj + η(il - wj)
jw
il
wjwj + ηil
ηil
il + wj
lll iii /
i ijiljljl wiwiwi 2,,2)(
![Page 18: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/18.jpg)
• is moving to the center of a cluster of sample vectors after repeated weight updates – Node j wins for three training
samples: i1 , i2 and i3
– Initial weight vector wj(0)– After successively trained
by i1 , i2 and i3 ,the weight vector
changes to wj(1),
wj(2), and wj(3),
jw
i2
i1
i3
wj(0)
wj(1)
wj(2)
wj(3)
![Page 19: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/19.jpg)
Example
will always win no matter
the sample is from which class
is stuck and will not participate
in learning
unstuck:
let output nodes have some conscience
temporarily shot off nodes which have had very high
winning rate (hard to determine what rate should be
considered as “very high”)
2w
1ww1
w2
![Page 20: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/20.jpg)
Example
Results depend on the sequenceof sample presentation
w1
w2
Solution:Initialize wj to randomly selected input vector il that are far away from each other
w1
w2
![Page 21: What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship](https://reader033.vdocument.in/reader033/viewer/2022051116/5697bfec1a28abf838cb88bd/html5/thumbnails/21.jpg)