Download - Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University
![Page 1: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/1.jpg)
Introduction toPattern Recognition
Charles TappertSeidenberg School of CSIS, Pace
University
![Page 2: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/2.jpg)
Pattern Classification
Most of the material in these slides was taken from the figures in Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley & Sons, 2001
![Page 3: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/3.jpg)
What is pattern recognition?
Definition from Duda, et al. – the act of taking in raw data and taking an action based on the “category” of the pattern
We gain an understanding and appreciation for pattern recognition in the real world – visual scenes, noises, etc. Human senses: sight, hearing, taste, smell, touch
Recognition not an exact match like a password
![Page 4: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/4.jpg)
An Introductory Example
“Sorting incoming Fish on a conveyor according to species using optical sensing”
Sea bassSpecies
Salmon
![Page 5: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/5.jpg)
Set up a camera and take some sample images to extract features Length Lightness Width Number and shape of fins Position of the mouth, etc…
Problem Analysis
![Page 6: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/6.jpg)
Preprocessing Segment (isolate) fishes from one another
and from the background Feature Extraction
Reduce the data by measuring certain features
Classification Divide the feature space into decision
regions
Pattern Classification System
![Page 7: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/7.jpg)
![Page 8: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/8.jpg)
Initially use the length of the fish as a possible feature for discrimination
Classification
![Page 9: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/9.jpg)
![Page 10: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/10.jpg)
The length is a poor feature alone!
Select the lightness as a possible feature
Feature Selection
![Page 11: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/11.jpg)
![Page 12: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/12.jpg)
Move decision boundary toward smaller values of lightness in order to minimize the cost (reduce the number of sea bass that are classified salmon!)
Task of decision theory
Threshold decision boundary and cost relationship
![Page 13: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/13.jpg)
Adopt the lightness and add the width of the fish to the feature vector
Fish xT = [x1, x2]
Lightness Width
Feature Vector
![Page 14: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/14.jpg)
Straight line decision boundary
![Page 15: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/15.jpg)
We might add other features that are not highly correlated with the ones we already have. Be sure not to reduce the performance by adding “noisy features”
Ideally, you might think the best decision boundary is the one that provides optimal performance on the training data (see the following figure)
Features
![Page 16: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/16.jpg)
Is this a good decision boundary?
![Page 17: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/17.jpg)
Our satisfaction is premature because the central aim of designing a classifier is to correctly classify new (test) input
Issue of generalization!
Decision Boundary Choice
![Page 18: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/18.jpg)
Better decision boundary
![Page 19: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/19.jpg)
Baysian Decision Theory
Pure statistical approach – parametric
Assumes the underlying probability structures are known perfectly
Makes theoretically optimal decisions
![Page 20: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/20.jpg)
Non-parametric algorithm
![Page 21: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/21.jpg)
Pattern Recognition Stages
Sensing Use of a transducer (camera or
microphone) PR system depends on the bandwidth, the
resolution sensitivity distortion of the transducer
Preprocessing Segmentation and grouping - patterns
should be well separated and not overlap
![Page 22: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/22.jpg)
Feature extraction Discriminative features Invariant features with respect to
translation, rotation, and scale Classification
Use the feature vector provided by a feature extractor to assign the object to a category
Post Processing Exploit context-dependent information to
improve performance
Pattern Recognition Stages (cont)
![Page 23: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/23.jpg)
![Page 24: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/24.jpg)
The Design Cycle
Data collection Feature Choice Model Choice Training Evaluation Computational Complexity
![Page 25: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/25.jpg)
![Page 26: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/26.jpg)
How do we know when we have collected an adequately large and representative set of examples for training and testing the system?
Data Collection
![Page 27: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/27.jpg)
Depends on the characteristics of the problem domain
Simple to extract, invariant to irrelevant transformations, insensitive to noise
Choice of Features
![Page 28: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/28.jpg)
Unsatisfied with the performance of our fish classifier and want to jump to another class of model
Model Choice
![Page 29: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/29.jpg)
Use data to determine the classifier
(Many different procedures for training classifiers and choosing models)
Training
![Page 30: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/30.jpg)
Measure the error rate (or performance)
Possibly switch from one set of features to another one
Evaluation
![Page 31: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/31.jpg)
What is the trade-off between computational ease and performance?
How does an algorithm scale as a function of the number of features, patterns, or categories?
Computational Complexity
![Page 32: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/32.jpg)
Learning and Adaptation
Supervised learning A teacher provides a category label
for each pattern in the training set Unsupervised learning
The system forms clusters or “natural groupings” of the unlabeled input patterns
![Page 33: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/33.jpg)
Introductory example conclusion
Reader may be overwhelmed by the number, complexity, and magnitude of the sub-problems of Pattern Recognition
Many of these sub-problems can indeed be solved
Many fascinating unsolved problems still remain
![Page 34: Introduction to Pattern Recognition Charles Tappert Seidenberg School of CSIS, Pace University](https://reader033.vdocument.in/reader033/viewer/2022061612/56649e315503460f94b2273e/html5/thumbnails/34.jpg)
DPS Pattern Recognition Dissertations
Completed Visual systems – Rick Bassett, Sheb Bishop, Tom
Lombardi, John Casarella Speech recognition – Jonathan Law Handwriting – Mary Manfredi Natural Language Processing – Bashir Ahmed Keystroke Biometric – Mary Curtin, Mary Villani,
Mark Ritzmann, Robert Zack, John Stewart, Ned Bakelman
Fundamental research areas – Kwang Lee, Carl Abrams, Ted Markowitz, Dmitry Nikelshpur
In progress Jonathan Leet, Amir Schur