csse463: image recognition day 11 due: due: written assignment 1 tomorrow, 4:00 pm written...
DESCRIPTION
Baseline: Hand-tuned decision boundaries You did this based on observations for fruit classification You did this based on observations for fruit classification You’ll do the same thing in Lab 4 for shapes You’ll do the same thing in Lab 4 for shapes But what if the features were much more complex? But what if the features were much more complex? We now discuss classifiers that learn class boundaries based on exemplars (e.g., labeled training examples) We now discuss classifiers that learn class boundaries based on exemplars (e.g., labeled training examples)TRANSCRIPT
CSSE463: Image Recognition CSSE463: Image Recognition Day 11Day 11 Due: Due:
Written assignment 1 tomorrow, 4:00 pmWritten assignment 1 tomorrow, 4:00 pm Start thinking about term project ideas.Start thinking about term project ideas.
Lab 4 (shape) tomorrow: feel free to start in Lab 4 (shape) tomorrow: feel free to start in advanceadvance
Questions?Questions?
Next 1.5 weeks: Pattern recognitionNext 1.5 weeks: Pattern recognition Concepts, error types (today)Concepts, error types (today) Basic theory and how to use classifiers in Matlab:Basic theory and how to use classifiers in Matlab:
Neural networks Neural networks Support vector machines (SVM).Support vector machines (SVM).
Pattern recognitionPattern recognition Making a decision from Making a decision from
datadata A classification problem: A classification problem:
assign a single class label assign a single class label to a datum pointto a datum point
Can include a special class, Can include a special class, rejectreject, ,
if a sample (a single datum if a sample (a single datum point) appears not to belong to point) appears not to belong to any known classany known class
If it is on the boundary between If it is on the boundary between classesclasses
Else Else forced classificationforced classification Boundaries between Boundaries between
classes-how?classes-how? There’s tons of theory, can There’s tons of theory, can
be applied to many areas. be applied to many areas. We focus on small subset We focus on small subset of those used for visionof those used for vision
Q1
Baseline: Hand-tuned decision Baseline: Hand-tuned decision boundariesboundaries
You did this based on observations for fruit You did this based on observations for fruit classificationclassification
You’ll do the same thing in Lab 4 for You’ll do the same thing in Lab 4 for shapesshapes
But what if the features were much more But what if the features were much more complex?complex?We now discuss classifiers that learn class We now discuss classifiers that learn class
boundaries based on boundaries based on exemplarsexemplars (e.g., labeled (e.g., labeled training examples)training examples)
Ex: Nearest neighbor classifierEx: Nearest neighbor classifier Assumes we have a feature Assumes we have a feature
vector for each imagevector for each image Calculate distance from Calculate distance from new new
test sampletest sample to to each labeled each labeled training sampletraining sample. .
Assign label as closest training Assign label as closest training samplesample
Generalize by assigning same Generalize by assigning same label as the majority of the k label as the majority of the k nearest neighbors. No majority?nearest neighbors. No majority?
d
i
ipipppdDIn
ypypxpxpppDIn
1
22121
221
22121
))()((,
))()(())()((,2
Nearest class meanNearest class mean Find class means and Find class means and
calculate distance to calculate distance to each meaneach mean Pro?Pro? Con?Con?
Partial solution: Partial solution: clusteringclustering Learning vector Learning vector
quantization (LVQ): quantization (LVQ): finds optimal clustersfinds optimal clusters
LVQ
Test point
Q2
Common model of learning Common model of learning machinesmachines
StatisticalLearning
Labeled TrainingImages
Extract Features(color, texture)
TestImage
Summary
LabelClassifierExtract Features(color, texture)
Focus on testingFocus on testingLet m = the number of possible class labelsLet m = the number of possible class labelsConsider m==2. Consider m==2. Example: Calculate distance to cluster Example: Calculate distance to cluster
means for 2 classes.means for 2 classes.
Dist/prob1
Dist/prob2
TestImage
Extract Features(color, texture)
Decide Class:1, 2
Multiclass problems Multiclass problems
Consider m>2. Consider m>2. Example: Calculate distance to cluster Example: Calculate distance to cluster
means for 10 classes.means for 10 classes.
Dist/prob1
Dist/prob2
Dist/probN
TestImage
Extract Features(color, texture)
Decide Class:1, 2,…N
How good is your classifier?How good is your classifier? Example from medicine: Example from medicine:
Disease detectionDisease detection Consider costs of false Consider costs of false
neg. vs. false pos.neg. vs. false pos. Lots Lots of different error of different error
measuresmeasures Accuracy = 10500/10800 = Accuracy = 10500/10800 =
97%. Is 97% accuracy OK?97%. Is 97% accuracy OK? Recall (or true positive Recall (or true positive
rate) = 500/600=83%rate) = 500/600=83% Precision = 500/700=71%Precision = 500/700=71% False pos rate = 200/10200 False pos rate = 200/10200
= 2%= 2%
DetectedDetected
TrueTrue
YesYes NoNo
YesYes 500500(true (true pos.)pos.)
100100(false (false neg.)neg.)
NoNo 200200(false (false pos.)pos.)
1000010000(true (true neg.)neg.)
10200Total actualnegative
600Total actualpositive
700Total det. as pos.
10100Total det. as neg.
How good is your classifier?How good is your classifier? Write out definitions of Write out definitions of
each measure noweach measure now
ExamplesExamples Accuracy = 10500/10800 = Accuracy = 10500/10800 =
97%. 97%. Recall (or true positive Recall (or true positive
rate) = 500/600=83%rate) = 500/600=83% Precision = 500/700=71%Precision = 500/700=71% False pos rate = 200/10200 False pos rate = 200/10200
= 2%= 2%
Detected:Detected:
Has:Has:
YesYes NoNo
YesYes 500500(true (true pos.)pos.)
100100(false (false neg.)neg.)
NoNo 200200(false (false pos.)pos.)
1000010000(true (true neg.)neg.)
Q3
What if I have a tunable What if I have a tunable threshold, t?threshold, t?
If t == 0: TPR = ___, FPR = ___If t == 0: TPR = ___, FPR = ___9/129/12 2/82/8
If t == 1: TPR = ___, FPR = ___If t == 1: TPR = ___, FPR = ___
Repeat for many values of tRepeat for many values of t
-3 -2 -1 0 1 2 3
NN N P N P N N P N PP N PP PP PPP
Simple example: single real-valued output.Simple example: single real-valued output.Thresholding: label = output > t ? P : NThresholding: label = output > t ? P : N
False Pos Rate
True
Pos
Rat
e
True class:
ROC curve ROC curve Receiver-operating Receiver-operating
characteristiccharacteristic Useful when you Useful when you
can change a can change a threshold to get threshold to get different true and different true and false positive ratesfalse positive rates
Consider extremesConsider extremes Much more Much more
information information recorded here!recorded here!
Q3
Confusion matrices for m>2 Confusion matrices for m>2 (outdoor image example)(outdoor image example)
Beach recall: 169/(169+0+2+3+12+14)=84.5%Beach recall: 169/(169+0+2+3+12+14)=84.5% Note confusion between mountain and urban Note confusion between mountain and urban
classes due to featuresclasses due to features Similar colors and spatial layoutSimilar colors and spatial layout
Bch Sun FF Fld Mtn UrbBch 169 0 2 3 12 14Sun 2 183 5 0 5 5FF 3 6 176 6 4 5Fld 15 0 1 173 11 0Mtn 11 0 2 21 142 24Urb 16 4 8 5 27 140
Detected
True
Q4
Why do we need separate training Why do we need separate training and test sets?and test sets?
Exam analogyExam analogy
But working on practice questions is helpful…get But working on practice questions is helpful…get the analogy? the analogy? We hope our ability to do well on We hope our ability to do well on practice questions helps us on the actual exampractice questions helps us on the actual exam
Application to nearest-neighbor classifiersApplication to nearest-neighbor classifiers
Often reserve a 3Often reserve a 3rdrd set for validation as well (to set for validation as well (to tune parameters of training set)tune parameters of training set)
Q5-8