PRENTICE HALL SIGNAL PROCESSING SERIES

Alan V. Oppenheim, Series Editor

ANDREWS AND HUNT BRIGHAM The Fast Fourier Tmnsform BRIGHAM BURDIC CASTLEMAN Digital Image Processing COWAN AND GRANT Adaptive Filters CROCHIERE AND RABINER DUDGEON AND MERSEREAU HAMMING Digital Filters, 3 /E HAYKIN, ED. HAYKIN, ED. Array Signal Processing JAYANT AND NOLL JOHNSON A N D DUDGEON KAY KAY Modern Spectral Estimation KINO LEA, ED. LIM LIM, ED. Speech Enhancement LIM AND OPPENHEIM, EDS. MARPLE MCCLELLAN AND RADER MENDEL OPPENHEIM, ED. OPPENHEIM AND NAWAB, EDS. OPPENHEIM, WILLSKY, WITH YOUNG OPPENHEIM AND SCHAFER Digital Signal Processing OPPENHEIM AND SCHAFER Discrete- Time Signal Processing QUACKENBUSH ET AL. Objective Measures of Speech Quality RABINER AND GOLD RABINER AND SCHAFER Digital Processing of Speech Signals ROBINSON AND TREITEL STEARNS AND DAVID STEARNS AND HUSH TRIBOLET VAIDYANATHAN WIDROW AND STEARNS

Digital Image Restomtion

The Fast Fourier Transform and Its Applications Underwater Acoustic System Analysis, 2/E

Multimte Digital Signal Processing Multidimensional Digital Signal Processing

Advances in Spectrum Analysis and Array Processing, Vols. I 5 II

Digital Coding of waveforms Array Signal Processing: Concepts and Techniques

Fundamentals of Statistical Signal Processing: Estimation Theory

Acoustic Waves: Devices, Imaging, and Analog Signal Processing Trends in Speech Recognition

Two-Dimensional Signal and Image Processing

Advanced Topics in Signal Processing Digital Spectral Analysis with Applications

Lessons in Digital Estimation Theory Number Theory an Digital Signal Processing

Applications of Digital Signal Processing Symbolic and Knowledge-Based Signal Processing

Signals and Systems

Theory and Applications of Digital Signal Processing

Geophysical Signal Analysis Signal Processing Algorithms

Digital Signal Analysis, 2/E Seismic Applications of Homomorphic Signal Processing

Multimte Systems and Filter Banks Adaptive Signal Processing

Fundamentals of Statistical Signal Processing:

Est imat ion Theory

Steven M. Kay University of Rhode Island

For book and bookstore information

I I http://wmn.prenhrll.com gopher to gopher.prenhall.com

Upper Saddle River, NJ 07458

Contents

Preface xi

1 Introduction 1 1.1 Estimation in Signal Processing . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 The Mathematical Estimation Problem . . . . . . . . . . . . . . . . . . 7 1.3 Assessing Estimator Performance . . . . . . . . . . . . . . . . . . . . . . 9 1.4 Some Notes to the Reader . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2 Minimum Variance Unbiased Estimation 15 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.3 Unbiased Estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.4 Minimum Variance Criterion . . . . . . . . . . . . . . . . . . . . . . . . 19 2.5 Existence of the Minimum Variance Unbiased Estimator . . . . . . . . . 20 2.6 Finding the Minimum Variance Unbiased Estimator . . . . . . . . . . . 21 2.7 Extension to a Vector Parameter . . . . . . . . . . . . . . . . . . . . . . 22

3 Cramer-Rao Lower Bound 27 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.3 Estimator Accuracy Considerations . . . . . . . . . . . . . . . . . . . . . 28 3.4 Cramer-Rao Lower Bound . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.5 General CRLB for Signals in White Gaussian Noise . . . . . . . . . . . . 35 3.6 Transformation of Parameters . . . . . . . . . . . . . . . . . . . . . . . . 37 3.7 Extension to a Vector Parameter . . . . . . . . . . . . . . . . . . . . . . 39 3.8 Vector Parameter CRLB for Transformations . . . . . . . . . . . . . . . 45 3.9 CRLB for the General Gaussian Case . . . . . . . . . . . . . . . . . . . 47 3.10 Asymptotic CRLB for WSS Gaussian Random Processes . . . . . . . . . 50 3.1 1 Signal Processing Examples . . . . . . . . . . . . . . . . . . . . . . . . . 53 3A Derivation of Scalar Parameter CRLB . . . . . . . . . . . . . . . . . . . 67 3B Derivation of Vector Parameter CRLB . . . . . . . . . . . . . . . . . . . 70 3C Derivation of General Gaussian CRLB . . . . . . . . . . . . . . . . . . . 73 3D Derivation of Asymptotic CRLB . . . . . . . . . . . . . . . . . . . . . . 77

vii

Fundamentals of Statistical Signal Processing: Estimation Theory