title goes here correlation pattern recognitionusers.ece.cmu.edu/~kumar/dowdseminar.pdf · title...
TRANSCRIPT
1
Vijayakumar Bhagavatula
Vijayakumar Bhagavatula
Title Goes HereCorrelation Pattern Recognition
December 10, 2003
2
Vijayakumar Bhagavatula
Outline
! Correlation pattern recognition! Pattern recognition examples! Book! Demos
3
Vijayakumar Bhagavatula
18-794 Pattern Recognition Theory
! Speech recognition! Optical character recognition (OCR)! Fingerprint recognition! Face recognition! Automatic target recognition! Biomedical image analysis
Objective: To provide the background and techniques needed for pattern classification
For advanced UG and starting graduate students
Example Applications:
4
Vijayakumar Bhagavatula
Pattern Recognition Methods
Feature ExtractionInput Classifier Class
! Statistical methods (e.g., Bayes decision theory)! Machine learning methods! Artificial neural networks! Correlation filters
Most approaches are based in image domain whereas significant advantages exist in spatial frequency domain.
6
Vijayakumar Bhagavatula
Example Feature-based Matching
Minutiae
Minutiae Extraction
Input Image
Minutiae
Orientation Field
Region of Interest
Thinned Ridges
Extracted RidgesRidge Ending
Ridge Bifurcation
Orientation Estimation
Fingerprint Locator
Ridge Extraction
Thinning f
Minutiae Extraction
! Features based on intuition & experience
! Significant preprocessing needed
! Sensitive to occlusions
7
Vijayakumar Bhagavatula
Correlation Pattern Recognition
! Normalized correlation between r(x) and s(x) between -1 and +1; reaches +1 if and only if r(x) = s(x).
! Problem: Reference patterns rarely have same appearance! Solution: Find the pattern that is consistent (i.e., yields large
correlation) among the observed variations.
( ) ( )
( ) ( )2 21 1
r x s x dx
r x dx s x dx− ≤ ≤∫
∫ ∫
! r(x) test pattern! s(x) reference pattern
8
Vijayakumar Bhagavatula
Pattern Variability
! Facial appearance may change due to illumination! Fingerprint image may change due to plastic deformation
9
Vijayakumar Bhagavatula
Pattern Locations
! Desired Pattern can be anywhere in the input scene.! Multiple patterns can appear in the scene.! Pattern recognition methods must be shift-invariant.
11
Vijayakumar Bhagavatula
Cross-Correlation Function
! Determine the cross-correlation between the reference and test images for all possible shifts
!When the target scene matches the reference image exactly, output is the autocorrelation of the reference image.
! If the input r(x) contains a shifted version s(x-x0) of the reference signal, the correlator will exhibit a peak at x=x0.
! If the input does not contain the reference signal s(x), the correlator output will be low
! If the input contains multiple replicas of the reference signal, resulting cross-correlation contains multiple peaks at locations corresponding to input positions.
( ) ( ) ( )c r x s x dxτ τ= −∫
12
Vijayakumar Bhagavatula
Cross-Correlation Via Fourier Transforms
InputScene
FT
CorrelationFilter
IFTCorrelationOutput
ReferenceIm age s(x)
FilterDesign
r(x)
R(f)
H (f)
c(τ)
! Fourier transforms can be done digitally or optically
13
Vijayakumar Bhagavatula
ToInput SLM
FourierLens
FourierLens
Correlationpeaks for objects
ToFilter SLM
CCD Detector
Laser Beam
FourierTransform
InverseFourierTransform
Optical Correlator
SLM: Spatial Light ModulatorCCD: Charge-Coupled Detector
14
Vijayakumar Bhagavatula
Correlation Filters
M atchNo M atch
DecisionTest Image
IFFT Analyze
Correlation output
FFT
Correlation Filter
Filter Design . . .Training Images
TrainingRecognition
15
Vijayakumar Bhagavatula
Peak to Sidelobe Ratio (PSR)
σmeanPeak
PSR−
=
1. Locate peak1. Locate peak
2. M ask a sm all 2. M ask a sm all pixel regionpixel region
3. Com pute the m ean and 3. Com pute the m ean and σσ in a in a bigger region centered at the peakbigger region centered at the peak
! PSR invariant to constant illumination changes
! Match declared when PSR is large, i.e., peak must not only be large, but sidelobes must be small.
18
Vijayakumar Bhagavatula
Using sam e Filter trained before,
Perform cross-correlation on cropped-face shown on left.
19
Vijayakumar Bhagavatula
••CO RRELATIO N FILTERS ARE SHIFT-INVARIANT
•Correlation output is shifted down by the sam e am ount of the shifted face im age, PSR rem ains SAM E!
20
Vijayakumar Bhagavatula
•Using SO M EO NE ELSE’S Filter,… . Perform cross-correlation on cropped-face shown on left.
•As expected very low PSR.
22
Vijayakumar Bhagavatula
Correlation Plane Contour M ap Correlation Plane Contour M ap
Correlation Plane SurfaceCorrelation Plane Surface
M 1A1 in the open M 1A1 near tree line
SAIP ATR SDF Correlation Perform ance for Extended Operating
Conditions
Courtesy: Northrop Grum m an
Adjacent trees cause some correlation noise
25
Vijayakumar Bhagavatula
Facial Expression Database
! Facial Expression Database (AMP Lab, CMU)! 13 People! 75 images per person! Varying Expressions! 64x64 pixels! Constant illumination
! 1 filter per person made from 3 training images
26
Vijayakumar Bhagavatula
PSRs for the Filter Trained on 3 Images
Response to Training Images Response to
Faces Images from Person A
M ARGIN OF SEPARATION
Response to 75 face images of the other 12 people=900 PSRs
PSR
28
Vijayakumar Bhagavatula
PIE Database Illumination Variations
! Simulations using 65 people from the Pose, Illumination and Expression (PIE) Database.
! Each person (with and without background lighting) has 21/22 face images respectively at frontal view.
30
Vijayakumar Bhagavatula
Training Image selection
! We used three face images to synthesize a correlation filter ! The three selected training images consisted of 3 extreme
cases (dark left half face, normal face illumination, dark righthalf face).
n = 3 n = 7 n = 16
33
Vijayakumar Bhagavatula
Reject Reject
AuthenticateAuthenticateThresholdThreshold
EER using Filter with Background illumination
36
Vijayakumar Bhagavatula
Iris Verification
! High-quality iris images yield low error rates
! Correlation filters yield zero verification errors for the 9 iris images
! Challenge is to acquire high-quality iris images
Source: National Geographic Magazine
Source: Dr. J. Daugman’s web site
37
Vijayakumar Bhagavatula
Features of Correlation Filters
! Shift-invariant; no need for centering the test image! Graceful degradation! Can handle multiple appearances of the reference image in
the test image! Closed-form solutions based on well-defined metrics
48
Vijayakumar Bhagavatula
Motivation for the Book
! Most pattern recognition researchers are not able to take advantage of the power of correlation filters because of the diverse background needed! Signals and systems
! Probability theory and random variables
! Linear algebra! Optical processing
! Digital signal processing
! Detection and estimation theory
! Goal of the book: To provide the background and techniques for correlation pattern recognition and illustrate with applications.
49
Vijayakumar Bhagavatula
Book Chapters
! Introduction! Mathematical background ! Signals and systems! Detection theory! Basic correlation filters! Advanced correlation filters! Optics basics! Optical correlators! Application examples