m.s. thesis defenseroadmap • literature review (5 min) • research objectives (5 min) •...
TRANSCRIPT
![Page 1: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/1.jpg)
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
M.S. Thesis Defense________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Bryan David ThompsonMarch 29th, 2007
________________________________________________________________________________________________________________________________
Adviser:Dr. Mahmood R. Azimi-Sadjadi
Committee:Dr. Robert Liebler
Dr. Edwin K. P. Chong
![Page 2: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/2.jpg)
Roadmap• Literature Review (5 min)• Research Objectives (5 min)• Algorithm Review (10 min)• Practical Application (15 min)
– Implementation– Experimental Results
• Theoretical Contribution (10 min)– Development– Simulation Results
• Q&A (15 min)• Additional slides
![Page 3: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/3.jpg)
Literature Review
Buried Underwater Target Detection and Classification• Sensor development & Data acquisition
[Schock] BOSS, AUVs, SAS Processing, Imagery[Harbaugh] Synthesized data sets
• Feature Extraction[Maussong] SAS Imagery (mean-standard deviation)[Sternlicht] Sediment volume imagery (clustering and ellipse)[Azimi] Canonical Correlation Analysis
• Detection & Classification[Dobeck] Detection & Classification Algorithm Fusion[Sternlicht] 2D & 1D Fusion (Gaussian-Bayesian classifiers)[Azimi] HMMs, NNs, D/F-level Fusion, CMAC,
Success of CCA as a coherence-based Feature extraction procedure
![Page 4: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/4.jpg)
Research Objectives
• To implement a multi-channel extension of CCA as a feature extraction procedure
• To demonstrate its effectiveness via practical application
• To develop an iterative learning algorithm providing numerous advantages
![Page 5: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/5.jpg)
Roadmap• Literature Review (5 min)• Research Objectives (5 min)• Algorithm Review (10 min)• Practical Application (15 min)
– Implementation– Experimental Results
• Theoretical Contribution (10 min)– Development– Simulation Results
• Q&A (15 min)• Additional slides
![Page 6: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/6.jpg)
Algorithm Review: Multi-channel Coherence Analysis
Assume n data channels
Composite data channel vector
Corresponding correlation matrix
![Page 7: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/7.jpg)
MCA Review (2)
![Page 8: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/8.jpg)
MCA Review (3)CCA: 1st coordinate pair and correlation
MCA: 1st coordinate set and correlations
![Page 9: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/9.jpg)
MCA Review (4)Constraint (1) Unit Variance:
SUMCOR + Unit Variance = No unique solution (thesis)
Constraint (2) Unit Trace:
SUMCOR + Unit Trace = Generalized eigensystem
![Page 10: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/10.jpg)
MCA Review (5)CCA: Higher order coordinates/correlations
MCA: Higher order coordinates/correlations
s.t.
![Page 11: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/11.jpg)
Roadmap• Literature Review (5 min)• Research Objectives (5 min)• Algorithm Review (10 min)• Practical Application (15 min)
– Implementation– Experimental Results
• Theoretical Contribution (10 min)– Development– Simulation Results
• Q&A (15 min)• Additional slides
![Page 12: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/12.jpg)
Practical Application• Date: May of 2004 • Platform: Disk BOSS• Location: St. Andrews Bay, Panama City, FL • 11 objects
– 5 mine-like (ML)• Boundary markers• Bomb-shaped object• Large cylinders
– 6 non-mine-like (NML)• Artillery shells• Small cylinder• Bullet-shaped objects• Spheres
• Naturally occurring objects (rocks / clutter)
![Page 13: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/13.jpg)
Implementation
![Page 14: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/14.jpg)
Implementation (2)Experimentally determined ping separations
![Page 15: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/15.jpg)
Experimental Results
• MCA-based single-ping classifier – Outperforms CCA-based single-ping classifier– Outperforms Decision-level Fusion for testing data
• Uses 67% more sonar returns per decision
– Highest generalization ability
![Page 16: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/16.jpg)
Experimental Results
![Page 17: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/17.jpg)
Roadmap• Literature Review (5 min)• Research Objectives (5 min)• Algorithm Review (10 min)• Practical Application (15 min)
– Implementation– Experimental Results
• Theoretical Contribution (10 min)– Development– Simulation Results
• Q&A (15 min)• Additional slides
![Page 18: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/18.jpg)
Theoretical Contribution
An Iterative Learning Algorithm for MCA• Data-driven• Real-time• Circumvented calculations:
– Calculating sample covariance matrices for all data channel pairs
– Solving generalized eigenvalue problems• Recent IJCNN 2007 acceptance
– Best Paper Award nomination by one reviewer
![Page 19: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/19.jpg)
Development1st coordinate set
Iterative method of steepest descent (or ascent… maximizing)
Concisely
![Page 20: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/20.jpg)
Development (2)1st correlation sum
Higher order (ith) coordinate sets / correlation sums
subject to:
![Page 21: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/21.jpg)
Development (3)Lagrange multiplier method with penalty parameter
Iterative method of steepest descent (ascent)
![Page 22: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/22.jpg)
Development (4)Conversion to Least-Mean-Square (LMS) based updating
For the composite coordinate mapping vector
For the correlation sum
![Page 23: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/23.jpg)
Simulation Results
• Simulated three-channel data set– -
• Guided by the linear model
![Page 24: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/24.jpg)
Simulation Results (2)
Squared estimation error plots vs. iteration
![Page 25: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/25.jpg)
Roadmap• Literature Review (5 min)• Research Objectives (5 min)• Algorithm Review (10 min)• Practical Application (15 min)
– Implementation– Experimental Results
• Theoretical Contribution (10 min)– Development– Simulation Results
• Q&A (15 min)• Additional slides
![Page 26: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/26.jpg)
Q&A
• Supported by ONR– Contract #N00014-05-1-0014
• Special Thanks– Dr. Azimi, Dr. Liebler, and Dr. Chong– Gordon, Nicholas, Jered, Amanda, Neil, Ali,
Jaime, Makoto, Derek, and Michael.– Family and friends– Coworkers at Agilent Technologies, Inc.
• Incredible experience because of the people
![Page 27: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/27.jpg)
Roadmap• Literature Review (5 min)• Research Objectives (5 min)• Algorithm Review (10 min)• Practical Application (15 min)
– Implementation– Experimental Results
• Theoretical Contribution (10 min)– Development– Simulation Results
• Q&A (15 min)• Additional slides
![Page 28: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/28.jpg)
Experimental Results
![Page 29: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/29.jpg)
Additional: MCA
FONC: (interior case)
SOSC: Satisfied because A (the Hessian) is positive semi-definite
![Page 30: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/30.jpg)
Additional: MCA (2)
![Page 31: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/31.jpg)
Additional: Mutual Information and Coherence
![Page 32: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/32.jpg)
Additional: BOSS Data set
![Page 33: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/33.jpg)
Additional: Preprocessing
![Page 34: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/34.jpg)
Additional: Concatenation
• For each channel, a preprocessed sonar return yields a a time series of 200x1
• Concatenated together to yield 9600x1• Partition into overlapping blocks 40x1
– 50% overlap– Statistically rich
![Page 35: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/35.jpg)
Additional: Feature Extraction• To capture ML vs NML discriminatory factors in sonar
returns using coherence-based feature extraction• CCA
– Input two preprocessed sonar ping partitions (each of 40x1)– Output vector of canonical correlations 40x1
• MCA– Input three preprocessed sonar ping partitions (each of 40x1)– Output vector of correlation sums 40x1
• Use dominate 20 as feature vector– To simplify classifier complexity
![Page 36: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/36.jpg)
Additional: Clarity• Preprocessing
– To remove undesirable portions of the signal– Matched filtering (requires incident signal)– Windowing– Inverse Matched Filtered
• Concatenation– Concatenate kth return for each channel into one time series– Form overlapping blocks with 50% overlap– Statistically rich
• Feature extraction– To capture ML and NML discriminatory factors in sonar
returns using coherence-based feature extraction– Canonical Correlation Analysis (two-channel)– Multi-channel Coherence Analysis (three-channel)
![Page 37: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/37.jpg)
Additional: Clarity (2)
• Single-ping classification– Back propagation neural network (BPNN)– Training data set: ½ of Line 4– Validation data set: ½ of Line 4– Testing data set: Lines 2 and 8
• Decision-level fusion– Nonlinear (BPNN-based)
![Page 38: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/38.jpg)
Additional: Single-ping Classifier• To capture ML vs. NML discriminatory factors in feature vectors• Batch process training using entire set of training patterns
– 20 dominant correlations (sums) extracted via CCA (MCA)– Equally split ML and NML features between training and validation
data• Back propagated error (between the desired and actual outputs)
through the system and the various weight layers are adjusted. – Mine-like output is [1 0]T– Non-mine-like output is [0 1]T
• Structure – Several different two- and three-layer BPNNs were tested– Two-layer implementation determined best– Tested ranged from 40 to 50 hidden layer neurons– Randomly initialized layer weights and biases (several times)– Optimal structure: 20 inputs, 50 hidden layer neurons, 2 output
neurons
![Page 39: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/39.jpg)
Additional: Single-ping Classifier (2)
• Optimal classifier chosen based on training and validation data set classification performance– ML classification performance favored over NML
• Training– Fixed learning rate (heuristically determined)– Levenberg-Marquardt (LM) learning rule [54]– No momentum factor
• Line 2 and 8 testing data sets determined classifiers’ generalization ability
![Page 40: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/40.jpg)
Additional: Decision-level Fusion
• To further exaggerate the captured ML vs. NML discriminatory factors of previously made decisions
• Similarly training as BPNN-based Single-ping classifiers
• Similarly obtained optimal structure– Optimal structure: 6 inputs, 6 hidden layer
neurons, 2 output neurons
![Page 41: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/41.jpg)
Additional: Implementation
![Page 42: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/42.jpg)
Additional: Implementation (2)
![Page 43: M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) • Algorithm Review (10 min) • Practical Application (15 min) – Implementation – Experimental](https://reader033.vdocument.in/reader033/viewer/2022051812/602e257d31c273515f44cb50/html5/thumbnails/43.jpg)
No more additional slides