from last time: pr methods
DESCRIPTION
From last time: PR Methods. Feature extraction + Pattern classification Training, testing, overfitting, overtraining Minimum distance methods Discriminant Functions Linear Nonlinear (e.g, quadratic, neural networks) -> Statistical Discriminant Functions. Statistical Pattern Recognition. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/1.jpg)
From last time:PR Methods
• Feature extraction + Pattern classification
• Training, testing, overfitting, overtraining
• Minimum distance methods• Discriminant Functions• Linear• Nonlinear (e.g, quadratic, neural
networks)• -> Statistical Discriminant
Functions
![Page 2: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/2.jpg)
Statistical Pattern Recognition
• Many sources of variability in speech signal
• Much more than known deterministic factors
• Powerful mathematical foundation• More general way of handling
discrimination
![Page 3: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/3.jpg)
Statistical Discrimination Methods
• Minimum error classifier and Bayes rule
• Gaussian classifiers• Discrete density estimation• Mixture Gaussians• Neural networks
![Page 4: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/4.jpg)
![Page 5: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/5.jpg)
![Page 6: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/6.jpg)
![Page 7: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/7.jpg)
we decide x is in class 2
we decide x is in class 1
![Page 8: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/8.jpg)
![Page 9: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/9.jpg)
![Page 10: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/10.jpg)
![Page 11: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/11.jpg)
![Page 12: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/12.jpg)
How to approximate a Bayes classifier• Parametric form with single pass
estimation
• Discretize, count co-occurrences
• Iterative training (mixture Gaussians, ANNs)
• Kernel estimation
![Page 13: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/13.jpg)
![Page 14: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/14.jpg)
![Page 15: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/15.jpg)
![Page 16: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/16.jpg)
Minimum distance classifiers• If Euclidean distance used,
optimum if:• Gaussian
• Equal priors
• Uncorrelated features
• Equal variance per feature
• If different variances per feature, correlated features, MD could be better
![Page 17: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/17.jpg)
![Page 18: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/18.jpg)
•Then the discriminant function can be
Di(x) = wiTx + wi0
•Where
Wi = Σi-1μi
•Andwi0 = - ½ (μi
T Σi-1μi) + log p(ωi)
•This is a linear classifier
![Page 19: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/19.jpg)
General Gaussian case
•Unconstrained covariance matrices per class
•Then the discriminant function is
Di(x) = xTWix + wiTx + wi0
•This is a quadratic classifier
•Gaussians are completely specified by 1st and 2nd order statistics
•Is this enough for general populations of data?
![Page 20: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/20.jpg)
![Page 21: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/21.jpg)
log p(x |ωi) + log p (ωi )
A statistical discriminant function
![Page 22: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/22.jpg)
![Page 23: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/23.jpg)
![Page 24: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/24.jpg)
![Page 25: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/25.jpg)
![Page 26: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/26.jpg)
P(a|b) = P(a,b)/P(b)
P(a,b) = P(a|b)P(b) = P(b|a)P(a)
Remember:
![Page 27: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/27.jpg)
![Page 28: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/28.jpg)
![Page 29: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/29.jpg)
![Page 30: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/30.jpg)
![Page 31: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/31.jpg)
![Page 32: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/32.jpg)
![Page 33: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/33.jpg)
![Page 34: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/34.jpg)
![Page 35: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/35.jpg)
![Page 36: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/36.jpg)
![Page 37: From last time: PR Methods](https://reader035.vdocument.in/reader035/viewer/2022062723/56813dfd550346895da7d610/html5/thumbnails/37.jpg)
Upcoming quiz etc.• Monday, 1st the guest talk on
“deep” neural networks
• Then the quiz. Topics: ASR basics, pattern recognition overview. Typical questions are multiple choice plus short explanation. Aimed at a 30 minute length.
• There will be one more HW, one more quiz, then all oriented towards project.