ece 6950 adaptive filters and systems - western...
TRANSCRIPT
ECE 6950Adaptive Filters and Systems
Dr. Bradley J. BazuinAssociate Professor
Department of Electrical and Computer EngineeringCollege of Engineering and Applied Sciences
ECE 6950 2
Course/Lecture Overview
• Syllabus• Personal Intro.• Textbook/Materials Used• Additional Reading• ID and Acknowledgment of Policies
• Textbook• Chapter1
ECE 6950 3
Syllabus
• Everything useful for this class can be found on Dr. Bazuin’s web site!– http://homepages.wmich.edu/~bazuinb/
• The class web site is at– http://homepages.wmich.edu/~bazuinb/ECE6950Adaptive/ECE6950_Sp14.html
• The syllabus …– http://homepages.wmich.edu/~bazuinb/ECE6950Adaptive/Syl_6950Adapt.pdf
ECE 6950 4
Who am I?
• Dr. Bradley J. Bazuin– Born and raised in Grand Rapids Michigan– Undergraduate: BS in Engineering and Applied Sciences,
Extensive Electrical Engineering from Yale University in 1980– Graduate: MS and PhD in Electrical Engineering from Stanford
University in 1982 and 1989, respectively.– Industrial Experience – Digital, ASIC, System Engineering
• Part-time ARGOSystems, Inc. (purchased by Boeing) 1981-1989• Full-time ARGOSystems, Inc. 1989-1991• Full-time Radix Technologies 1991-2000
– Academic Experience – Electrical and Computer Engineering• Term-appointed Faculty, WMU ECE Dept. 2000-2001• Tenure track Assistant Professor, WMU ECE Dept. 2001-2007• Tenured Associate Professor, WMU ECE Dept. 2007- present
Research Activities and Interests
• Sunseeker – Adviser to solar car team– Electrical Systems: Li battery protection system, Controller Area Network (CAN)
based sensors and controllers, Solar Array Energy Collection and Conversion• Center for the Advancement of Printed Electronics (CAPE)
– Printed electronic device design, fabrication and testing– Semiconductor Physics
• Physical Layer Communication Signal Processing– Software Defined Radios (SDR)– Mulitrate Signal Processing (digital channel bank analysis and synthesis, filter-decimation and
interpolation-filter design methods)– Adaptive Filtering and Systems (channel equalization, smart-antenna spatial beamforming)
• Communication-based Digital Signal Processing Algorithm Implementation– Xilinx programmable devices– Parallel processing, hosts including NVIDIA GPUs with CUDA and multithreaded applications
ECE 6950 5
ECE 6950 6
Required Textbook/Materials
• Ali. H. Sayed, “Fundamentals of Adaptive Filtering,” Wiley & Sons, Hoboken, NJ, 2003, ISBN: 978-0-471-46126-5.
• MATLAB, Student Edition• MATLAB Signal Processing Toolbox & DSP Toolbox
– The MATH Works,MATLAB and Toolboxeshttp://www.mathworks.com/
ECE 6950 7
Supplemental Books and Materials
• Ali. H. Sayed, “Adaptive Filters,” Wiley & Sons, Hoboken, NJ, 2008, ISBN: 978-0-470-25388-5.
• Ali. H. Sayed, “Adaptive Filters,” IEEE-Wiley eBooks Library Title, http://ieeexplore.ieee.org/xpl/bkabstractplus.jsp?bkn=5237520
• S. Haykin, Adaptive Filter Theory, 5th ed., Prentice-Hall, 2014.• J.R. Treicher, C. R. Johnson, Jr., M.G. Larimore, “Theory and Design of
Adaptive Filters”, Prentice-Hall, Upper Saddle River, NJ, 2001. ISBN: 0-13-040265-6.
• T. Kailath, A.H. Sayed, B. Hassibi, “Linear Estimation”, Prentice Hall, 2000. ISBN: 0-13-022464-2.
• G. Strang, “Linear Algebra and Its Applications, 2nd ed.”, Academic Press, 1980. ISBN: 0-12-673660-X.
• G.H. Golub and C.F. Van Loan, “Matrix Computations”, 3rd ed., Johns Hopkins Univ. Press, 1996. ISBN: 978-0-801-85414-9.
ECE 6950 8
Supplemental Materials
• ftp://ftp.wiley.com/public/sci_tech_med/filtering/– MATLAB programs to solve all computer projects
• Ali. H. Sayed, on-line course lectures.– UCLA: EE210A Adaptive Filtering– Additional Course Website:
• http://asl.ee.ucla.edu/index.php?option=com_content&task=view§ionid=10&id=214
– On-line lectures using his other textbook• A newer version that is designed for a graduate class.
It does not include as much material as this one and does not have some of the supporting material present in this text.
ECE 6950 9
Identification and Acknowledgement
• Identification for Grade Posting, Course and University Policies, and Acknowledgement
• Please read, provide unique identification, sign and date, and return to Dr. Bazuin.
ECE 6950 10
Course/Text Coverage Goals
Linear EstimationChapter 1: OPTIMAL ESTIMATION
Chapter 2: LINEAR ESTIMATION
Chapter 3: CONSTRAINED LINEAR ESTIMATION
Stochastic Gradient Adaptive MethodsChapter 4: STEEPEST-DESCENT ALGORITHMS
Chapter 5: STOCHASTIC-GRADIENT ALGORITHMS
Chapter 10: BLOCK ADAPTIVE FILTERS
Performance AnalysisChapter 6: STEADY-STATE PERFORMANCE OF ADAPTIVE FILTERS
Chapter 7: TRACKING PERFORMANCE OF ADAPTIVE FILTERS
Chapter 8: FINITE PRECISION EFFECTS (brief)
Chapter 9: TRANSIENT PERFORMANCE OF ADAPTIVE FILTERS
Least-Squares Adaptive MethodsChapter 11: THE LEAST-SQUARES CRITERION (brief)
Chapter 12: RECURSIVE LEAST-SQUARES
Chapter 13: RLS ARRAY ALGORITHMS (if time permits)
Chapter 14-17 Not Covered
Text Key Sections
• The key sections listed in the preface will be followed:– See Table P.4 on page xxvi.
• The lecture plan is:– to cover the material suggested, – include important aspects of in the chapter appendixes– Include example problems when the text and the homework
“degree of difficulty” is significantly different.
ECE 6950 11
Impression from 2009
• Chap 1-4 is based on prerequisite mathematical concepts. Chap 5, with 4 as a setup, contains the dominant adaptive algorithms. Chap 6-9 provide steady state, transient steady state, numerical precision, and transient analysis. The use of ensemble average performance needs to be reviewed and included in Chap. 5 as a simulation technique for algorithm validation. Chap. 10 can be dealt with as prerequisite least-squares material with Chap 11 using the recursive least-squares algorithm.
• Block approaches to optimal filter generation are not specifically addresses. They would also help motivate linear algebra manipulations such as Cholesky factorization and QR decomposition. Cholesky and QR are introduced for LS methods and the RLS algorithm, but was only used as final exam material this time.
• A significant number of simulations are based on purely random input signals. More applicable communications or test signals need to be developed. An increase emphasis on blind-adaptive algorithms and analysis would be useful. There are very few examples with non-random signal examples of blind-adaptation. (PM or FM or an alternate constant modulus test signal must be developed.)
• One student had taken MATH 6050 Optimization which greatly enhanced their understanding of the introductory material.
ECE 6950 12
Impression from 2011
• Set up classic examples of adaptive systems for continuing simulations. Attempt to expand beyond channel estimation, equalization, and noise cancellation.
• Incorporate more blind adaptive examples. Spatial beamforming examples would help.
• Block approaches to optimal filter generation are not specifically addresses. They would also help motivate linear algebra manipulations such as Choleskyfactorization and QR decomposition. Cholesky and QR are introduced for LS methods and the RLS algorithm.
• Frequency domain processing was not presented well by the text. Investigate the appendix material for more practical examples and knowledge. Improve on the MATLAB simulation example (based on appendix material).
• Inclusion of chapter 11/12 material in the initial discussion of RLS may be useful. It helps to introduce alternate cost functions.
• Projects definition and activity needs more work. Generally, what I was hoping for was accomplished, but more structure for getting started is needed.
ECE 6950 13
ECE 6950 14
Course PlanExam 1
Chapter 1: OPTIMAL ESTIMATION
Chapter 2: LINEAR ESTIMATION
Chapter 3: CONSTRAINED LINEAR ESTIMATION
Exam 2Chapter 4: STEEPEST-DESCENT ALGORITHMS
Chapter 5: STOCHASTIC-GRADIENT ALGORITHMS
Chapter 6: STEADY-STATE PERFORMANCE OF ADAPTIVE FILTERS
Chapter 7: TRACKING PERFORMANCE OF ADAPTIVE FILTERS
Chapter 8: FINITE PRECISION EFFECTS (brief)
Chapter 9: TRANSIENT PERFORMANCE OF ADAPTIVE FILTERS
Final ExamChapter 10: BLOCK ADAPTIVE FILTERS
Chapter 11: THE LEAST-SQUARES CRITERION
Chapter 12: RECURSIVE LEAST-SQUARES
Chapter 13: RLS ARRAY ALGORITHMS
Motivations
ECE 6950 15
Estimation Theory
• Estimation theory is a branch of statistics that deals with estimating the values of parameters based on measured/empirical data that has a random component. The parameters describe an underlying physical setting in such a way that their value affects the distribution of the measured data. An estimator attempts to approximate the unknown parameters using the measurements.
http://en.wikipedia.org/wiki/Estimation_theory
ECE 6950 16
Three Basic Kinds of Estimation
17
• Estimator Information Processing Tasks:– Filtering
– Smoothing
– Prediction
• Linear Optimal Filters– Requires a priori statistical/probabilistic information about the signal and environment.– Matched filters, Wiener filters or Kalman filters
• Adaptive filters – Self-designing filters that “internalize” the statistical/probabilistic information using recursive
algorithm that, when well design, approach the linear optimal filter performance.– Applied when complete knowledge of environment is not available a priori
S. Haykin, Adaptive Filter Theory, 5th ed., Prentice-Hall, 2014
Four Classes of Application
• Identification• Inverse Modeling• Prediction• Interference Cancellation
ECE 6950 18
Identification
• The mathematical Model of an “unknown plant” • In state space control system this is an adaptive observer of
the Plant– Examples: Seismology predicting earth strata
ECE 6950 19S. Haykin, Adaptive Filter Theory, 5th ed., Prentice-Hall, 2014
Inverse Modeling
• Providing an “Inverse Model” of the plant• For a transmission medium, the inverse model corrects
non-ideal transmission characteristics. – An adaptive equalizer
ECE 6950 20S. Haykin, Adaptive Filter Theory, 5th ed., Prentice-Hall, 2014
Prediction
• Based on past values, provide the best prediction possible of the present values.– Positioning/Navigation systems often need to predict where an
object will be based on past observations
ECE 6950 21S. Haykin, Adaptive Filter Theory, 5th ed., Prentice-Hall, 2014
Interference Cancellation
• Cancellation of unknown interference that is present along with a desired signal of interest.– Two sensors of signal + interference and just interference– Reference signal (interference) is used to cancel the interference in
the Primary signal (noise + interference)– Classic Examples: Fetal heart tone monitors, spatial beamforming
ECE 6950 22S. Haykin, Adaptive Filter Theory, 5th ed., Prentice-Hall, 2014
ECE 6950 23
Chapter 1: Optimal Estimation1 OPTIMAL ESTIMATION
1.1 Variance of a Random Variable 1.2 Estimation Given No Observations 1.3 Estimation Given Dependent Observations
1.3.1 Mean-Square-Error Criterion 1.3.2 Orthogonality Principle 1.3.3 Gaussian Random Variables
1.4 Estimation in the Complex and Vector Cases 1.4.1 Complex-Valued Random Variables 1.4.2 Vector-Valued Random Variables 1.4.3 Optimal Estimator in the Vector Case 1.4.4 Equivalent Optimization Criterion 1.4.5 Spherically Invariant Gaussian Variables
1.5 Summary of Main Results 1.6 Bibliographic Notes 1.7 Problems 1.8 Computer Project l.A Hermitian and Positive-Definite Matrices l.B Gaussian Random Vectors