nonlinear mmse estimation and soft decisions stefano galli dr. stefano galli telcordia technologies,...
TRANSCRIPT
Nonlinear MMSE Estimation and Soft Decisions Nonlinear MMSE Estimation and Soft Decisions
Stefano GalliStefano Galli
Dr. Stefano GalliTelcordia Technologies, Inc.Room: MCC-1J124B445 South StreetMorristown, NJ 07960-6438Tel. : (973) 829-4980Fax : (973) 829-5886Email: [email protected] Copyright © 2003 Telcordia Technologies. All Rights Reserved
John Hopkins University, April 17, 2003.John Hopkins University, April 17, 2003.
Telcordia Technologies Proprietary - Copyright 2003.
Telcordia Technologies Applied ResearchCore Competencies
Network and Services Management– Services Planning and Provisioning– Broadband Network Management– Wireless Network Management– Global Services Management
Software Technology and Engineering– High Availability Software– Distributed Systems– Scalable Systems– Software Architecture and Testing
Telcordia Technologies Proprietary - Copyright 2003.
Internet Evolution– Internet Architecture/Performance/Economics– Quality of Service (QoS) in Converged Networks
Next-Generation Networks– Voice over Packet (VoP)/Voice over IP (VoIP)– Multi-Protocol Label Switching (MPLS)– Integrated Access Networking– Internet Appliances and Premises Interworking– Next-Generation Signaling and Control
Telcordia Technologies Applied ResearchCore Competencies, Cont’d
Telcordia Technologies Proprietary - Copyright 2003.
Wireless Networking– 3rd Generation (3G)– RF Technology– Digital Signal Processing for Wireless Comms– Wireless Architecture and Middleware– Wireless Applications/Support (WAP)
Optical Networking– Dense Wavelength Division Multiplexing (DWDM)– Routing in All-Optical Networks– Optical Network Engineering and Management– Quantum Computing and Quantum Cryptography
Telcordia Technologies Applied ResearchCore Competencies, Cont’d
Telcordia Technologies Proprietary - Copyright 2003.
E-Business and Information Assurance– E-Commerce for Mobile Users (M-Commerce)– Data Mining and Information Extraction/Integration– Virtual Private Networks– Network Security
Plus Specialized Expertise In:– Speech Technology and Applications– Mathematical Sciences/Statistics (Algorithms, Network
Traffic Modeling, Cryptography, etc.)– Cable Interconnection Technology
Telcordia Technologies Applied ResearchCore Competencies, Cont’d
Telcordia Technologies Proprietary - Copyright 2003.
Telcordia Technologies Applied ResearchBroadband Networking Group
David Waring, Kenneth Kerpez, Thomas Banwell, Stefano Galli
Various aspects of DSL– Standardization efforts;– Loop and crosstalk identification;– Crosstalk modeling;– Dynamic Spectrum Management.
Power Line Communications
Home Networking (wired/wireless)
Telcordia Technologies Proprietary - Copyright 2003.
Summary of presentation
1) Hard and Soft Decisions in Adaptive Equalization
2) Linear and Nonlinear MMSE Channel Estimation
3) The Proposed Approach to Soft Detection
4) Practical Applications
5) Conclusions
Telcordia Technologies Proprietary - Copyright 2003.
• Hard decisions (Hard-Statistics) are extensively used communications, e.g. in adaptive receivers for channel estimation and tracking, in the feedback section of a DFE, etc.
• Hard-Decisions don’t give us an index of the reliability of the decisions but are the best we can do, if the decisions are correct.
• If the decisions are wrong, the use of Hard-Decisions may cause severe performance degradation (e.g. channel tracking loss, catastrophic error propagation, etc.).
Hard-Statistics and Soft-Statistics
Telcordia Technologies Proprietary - Copyright 2003.
• Soft-Decisions (Soft-Statistics) are decisions on a transmitted symbol which also contain an index of the reliability of the decision.
• In general, Soft-Decisions contain all the information contained in Hard-Decisions, whereas the viceversa is not true.
Hard-Statistics and Soft-Statistics (cont.)
Recently, Soft-Decisions have been proven to be a useful tool in the following areas:
• Channel estimation and tracking;
• Combined adaptive equalization and decoding;
• Blind equalization based on second order statistics;
• Enhanced DFE with soft feedback section;
• Iterative decoding of parallel/serial concatenated coded streams.
Telcordia Technologies Proprietary - Copyright 2003.
Essentially three kinds of soft-information:Essentially three kinds of soft-information:
• Nonlinear function applied to an estimate of the symbol.
• A Posteriori Probabilities (APPs);
• Estimate (usually, MMSE) of transmitted symbols;
Is there any formal justification for their use?Is there any formal justification for their use?
Are they related in any way?Are they related in any way?
Hard-Statistics and Soft-Statistics (cont.)
Telcordia Technologies Proprietary - Copyright 2003.
The Considered Communications System
Time Division Multiple Access
Transmission of independent time slots with preamble or midamble
a(i)
a(i-D)
H (f)T
exp(j2 f t)
xx
n(t) exp(j2 f t) +
QAM Modulator
H (f) x
q(t)
y(i)R
AdaptiveReceiver
(t-iT )Si
0
0
exp(-j2 f t)0
Radio
Channel
r(t) exp(j2 f t)0
QAM Demodulator
The System Model
Telcordia Technologies Proprietary - Copyright 2003.
The System Model (cont.)
Model of the observations (channel as linear filter, convolution):
where:
(vector of the TS-sampled input delay-spread function g(t;));
(ISI channel state vector);
(complex zero-mean Gaussian noise sequence);
)()()()()(v)();()(1
0ivizivixiGikiskigiy T
L
k
C
)(
CC
LTLigigiG C )1;( ... )0;()(
C
C
L
STLisisix A )1( ... )()(
)()()( C isjvicviv
Telcordia Technologies Proprietary - Copyright 2003.
Adaptive Receivers Based on Hard-Statistics
Goal of an adaptive receiver
• Recover the transmitted data stream after the distortion caused by the channel, by continuously adapting to the time-varying channel.
Basic components of an adaptive receiver
• Adaptive channel estimator: estimates on the basis of a certain criterion (LMS, MMSE, etc.) the distortion introduced by the channel and provides this estimate to the symbol/sequence detector.
• Symbol/sequence detector (equalizer): using the estimation of the channel, decides on the basis of a certain criterion (MAP, ML, ZF, etc.) what symbols were transmitted and then feeds them to the channel estimator.
Telcordia Technologies Proprietary - Copyright 2003.
Adaptive Receivers Based on Hard-Statistics: major problems
• Decisions on symbols are best made (minimization of probability of error) if the detector “waits” and gathers more information from the received signal before deciding on the transmitted symbols. The amountof “waiting” is the decision delay.Example: Viterbi algorithm optimal if decision delay is infinite, but in practice a delay of five times the memory of the channel L is sufficient.
• Channel estimators process both the received signal and the decisions of the symbol/sequence detector to estimate the channel and output the estimate of the channel at the time the decisions were made.
If the decision delay is too large, channel estimate is old.
If decision delay is too small, decisions are not reliable.
Telcordia Technologies Proprietary - Copyright 2003.
MLS detection and decision driven channel estimator
Low-delay tentative hard-decisions (d(L-1)) for channel estimation.Final hard-decisions at large delay (D5(L-1)).Trade-off between decision delay and prediction order of the channel estimate.
r(i) a(i-D)^
Observation MLSE-VA
Channel estimator
Channelestimates
with delay d
Final decisionswith high delay D
Tentative decisionswith low delay d
a(i-d)
B
Trainingsymbols
A
S
Adaptive Receivers Based on Hard-Statistics: examples
Telcordia Technologies Proprietary - Copyright 2003.
Effects of predicted channel estimates on system performance(BPSK over Rayleigh channel with Land Mobile fading spectrum)
Adaptive Receivers Based on Hard-Statistics: examples
Telcordia Technologies Proprietary - Copyright 2003.
Receivers based on the Per Survivor Processing (PSP) principle
Kubo, Murakami, Fujino (1994) and Polydoros, Raheli (1995)
• There are S(L-1) distinct estimators for the S(L-1) states, respectively;• Higher tracking capability at the expense of a much larger implementation complexity.
MLSE detector
S distinct tentative decisions, zero delay
large-delay final decisions
channel estimator #1
channel estimator #2
channel estimator #S
...
...
.........
...S distinct
channel estimates
L-1
L-1
L-1
received �data y(i)
Adaptive Receivers Based on Hard-Statistics: examples
Telcordia Technologies Proprietary - Copyright 2003.
Channel ModelChannel Model:
ObservationObservation :
GoalGoal : , i.e. MMSEMMSE estimation of the channel impulse response
The Linear MMSE recursive estimate of the channel impulse response:
)()1()( idiGiG
)()()()( inixiGiy T
iyiGEiiG 1/)()/(ˆ
)i(vix)i(G)i()ki(s)k;i(g)i(y TL
k
C
)(
1
0v
)i/i(y)i(yc)i/i(Gy|)i(GE)i/i(G ii 1111
MMSE Channel Estimation: Linear or Nonlinear?
Telcordia Technologies Proprietary - Copyright 2003.
The one-step MMSE prediction of the observation
Common semplification: assume correct correct hard decisions.
,y|)i(x)i/i(GE
y|)i(yE)i/i(y
iT
i
1
1
11
11
MMSE Channel Estimation: Linear or Nonlinear?
1111 iT y),i(x|)i/i(GE)i/i(y
pdf of pdf of GG((ii) conditioned to ) conditioned to yy((ii) ) andand xx((ii) is Gaussian) is Gaussian
Linear MMSE estimation (Kalman) of Linear MMSE estimation (Kalman) of GG((ii) is optimal) is optimal
Telcordia Technologies Proprietary - Copyright 2003.
However, the problem is intrinsecally non-Gaussian:
pdf of pdf of xx((ii) or ) or GG((ii) conditioned to ) conditioned to yy((ii) is ) is notnot Gaussian Gaussian
Optimal MMSE channel estimator is nonlinearOptimal MMSE channel estimator is nonlinear
MMSE Channel Estimation: Linear or Nonlinear?
Goal: , nonlinear nonlinear MMSE estimation of the channel impulse response
iyiGEiiG 1/)()/(ˆ
Telcordia Technologies Proprietary - Copyright 2003.
In estimation theory, it is useful to define the following sequences:
1
1/)()1/(ˆ iyiGEiiG
1
1/)()1/(ˆ iyiyEiiy
In fact, the following sequences:
(estimate innovation sequence)
(observation innovation sequence)
have special properties depending on the nature of the estimator:
Linear estimation : uncorrelated sequences.
Nonlinear estimation : Martingale Difference (MD) sequences.
)1/(ˆ)/(ˆ)( iiGiiGi
)1/(ˆ)()( iiyiyi
Nonlinear MMSE Channel Estimation
Telcordia Technologies Proprietary - Copyright 2003.
M D R e p r e s e n t a t i o n T h e o r e m
T h e M D s e q u e n c e ( i ) c a n b e e x p r e s s e d a s a t r a n s f o r m a t i o n o f t h e
M D s e q u e n c e ( i ) a s i n t h e f o l l o w i n g :
)()/()( iiiCi
w h e r e t h e o b s e r v a t i o n d e p e n d e n t f i l t e r i n g g a i n C ( i / i ) i s a n a d a p t e d
s e q u e n c e a n d i s g i v e n b y :
ii yiiEyiiEiiC 11 /)(*)( //)(*)()/(
Nonlinear MMSE Channel Estimation (cont.)
Telcordia Technologies Proprietary - Copyright 2003.
Given a probability space (,F,P), the sequence {C} is said to be adaptedadapted to {B}, if C(i/i) is measurablemeasurable with respect to Bi for any i, where {B}={Bi , i=1, 2, ...} is an increasing sequence of -algebras of subsets in F.
Since Bi can be viewed as containing the past of all sequences of interest up to instant i,
the property of being an adapted sequence implies that the filtering gain C(i/i) is not recursively computablenot recursively computable.
ii yiiEyiiEiiC 11 /)(*)( //)(*)()/(
Nonlinear MMSE Channel Estimation (cont.)
Telcordia Technologies Proprietary - Copyright 2003.
In order to have a recursive estimator, it would be necessary that the gain sequence C(i/i) be predictablepredictable not adapted or, equivalently, that C(i/i) is measurablemeasurable with respect to Bi-1 for any i. Therefore:
Unfortunately, the MD-Representation theorem has been proven to be falsefalse in the predictable form for the case of
1
11
1 /)(*)( //)(*)()1/()/( ii yiiEyiiEiiCiiC
discrete-time observations in white Gaussian noise!!!discrete-time observations in white Gaussian noise!!!
Nonlinear MMSE Channel Estimation (cont.)
Telcordia Technologies Proprietary - Copyright 2003.
Non Linear MMSE estimation:Non Linear MMSE estimation:E. Baccarelli, R. Cusani, S. Galli, “A Novel Adaptive Receiver with Enhanced Tracking Capability for TDMA-Based Mobile Radio Communications”, IEEE JSAC-Special Issue on Wireless Communications (Part II), Vol. 16, No. 8, Dec. 1998.
• Formal approach via martingales is not trivial;
• The exact solution does not exist in the recursive and finite dimensional form;
• Structure of the estimator is fixed: Kalman-like;
• Computational complexity issues;
Is there another way?Is there another way?
Nonlinear MMSE Channel Estimation (cont.)
Telcordia Technologies Proprietary - Copyright 2003.
Let us consider again the linear MMSE recursive estimate of the channel impulse response:
)i/i(y)i(yc)i/i(Gy|)i(GE)i/i(G ii 1111
11 11 iT y|)i(x)i/i(GE)i/i(y
1 11 iT y|)i(xE)i/i(G
Conjecture: this approximation is less harsh than assuming correct hard decisions.
Soft-Decision Based Channel Estimation
Telcordia Technologies Proprietary - Copyright 2003.
1111 iT y|)i(xE)i/i(G)i/i(y
NL-MMSENL-MMSE prediction of the states of the ISI channel prediction of the states of the ISI channeloror
NL-MMSE filtered and fixed-lag smoothed estimates NL-MMSE filtered and fixed-lag smoothed estimates of the transmitted symbolsof the transmitted symbols
11
11
11
11
|)1(
|)1(
|)(
|)(
i
i
i
i
yLisE
yisE
yisE
yixE
Soft-Decision Based Channel Estimation (cont.)
Telcordia Technologies Proprietary - Copyright 2003.
Goal: Non-linear MMSE estimation of transmitted symbol Goal: Non-linear MMSE estimation of transmitted symbol ss((ii))
iMMSENL yisEiis 1|)()/(ˆ
NL-MMSENL-MMSE estimation of the channel impulse response estimation of the channel impulse response
NL-MMSENL-MMSE estimate of the ISI channel state vector estimate of the ISI channel state vector
11
11
11
11
|)1(
|)1(
|)(
|)(
i
i
i
i
yLisE
yisE
yisE
yixE
Soft-Decision Based Channel Estimation (cont.)
Telcordia Technologies Proprietary - Copyright 2003.
NL- MMSE estimation of transmitted symbolsNL- MMSE estimation of transmitted symbols
Tarköy (ISIT ’95), Wang & Poor (IEEE Trans. Comm. ’99)
Only filtered estimates may be obtained.Therefore, it is optimal only for channels with no memory.
kkk
MMSENLiysisPs
APPsiis
1|)(
symbols message theof )/(ˆ f
Soft-Decision Based Channel Estimation (cont.)
Telcordia Technologies Proprietary - Copyright 2003.
NL- MMSE estimation of transmitted symbolsNL- MMSE estimation of transmitted symbols
(ISIT ’00; IEEE Trans. on Comm., Dec. 2002)
New approachNew approach:
where 0 D L-1
Both filtered and fixed-lag smoothed estimates may be obtained.
Therefore, it is optimal also for channels with memory.
vectorstate channel ISI theof )/(ˆ APPsDiis MMSENL f
Soft-Decision Based Channel Estimation (cont.)
Telcordia Technologies Proprietary - Copyright 2003.
)(
|)(
|)(
|)(
)/1(ˆ
)/1(ˆ
)/(ˆ
)/(ˆ
1
12
11
)()2()1(
)(2
)2(2
)1(2
)(1
)2(1
)1(1
i/i
yixP
yixP
yixP
iLis
iis
iis
iis
iN
i
i
NLLL
N
N
MMSENL
MMSENL
MMSENL
MMSENL
Ξ
Optimal NL-MMSE symbol estimation as a linear transformation of the APP vector of the channel state.
Soft-Decision Based Channel Estimation (cont.)
Telcordia Technologies Proprietary - Copyright 2003.
The channel estimator is fed by the more informative NL-MMSE estimates of the transmitted symbols and not by the usual hard-decided data.
The APPs are recursively computed and delivered to the channel estimatorwith no delay.
Practical applications: adaptive equalization
Telcordia Technologies Proprietary - Copyright 2003.
A MLS equalizer can operate efficiently at a high decision delay, e.g. at a decision delay equal to the length of the TDMA-slot.
The VA can build the trellis with more reliable zero-delayed channel estimates and in parallel with channel tracking.
Practical application: adaptive equalization (cont.)
Telcordia Technologies Proprietary - Copyright 2003.
Six equal-powered taps with land mobile fading spectrum
(BPSK modulation - Lp=12, Lf=60 - BDTS = 10-4)
Practical application: adaptive equalization (cont.)
Telcordia Technologies Proprietary - Copyright 2003.
Six equal-powered taps with land mobile fading spectrum
(BPSK modulation - Lp= 20, Lf= 100 - BDTS = 10-3)
Practical application: adaptive equalization (cont.)
Telcordia Technologies Proprietary - Copyright 2003.
HF link: CCIR Moderate conditions (= 1 ms;BD=0.5 Hz)
(QPSK modulation - Lp=15, Lf=50 - BDTS = 4.17·10-4)
Practical application: adaptive equalization (cont.)
Telcordia Technologies Proprietary - Copyright 2003.
Practical application: enhanced DFE
Digital Feedback Equalization (DFE)Digital Feedback Equalization (DFE)
Pros: optimum (MMSE sense), cheap, adaptive channel estimation.
Cons: risk of catastrophic error propagation.
MAP or MLSE are better than the MAP or MLSE are better than the idealideal DFE (at least 3 dB) DFE (at least 3 dB)
If channel impulse response is short:
MAP or MLSE are the preferred choice.
If channel impulse response is long:
DFE is the only practical approach unless reduced state techniques are employed for the MAP or MLSE solutions
Telcordia Technologies Proprietary - Copyright 2003.
Practical application: enhanced DFE (cont.)
The soft approachThe soft approach(Globecom 1999)(Globecom 1999)
• Reduced state MAP or MLSE receivers with feedback filter to shorten the long impulse response;
• Relax the overly optimistic assumption of error free decisions;
• Do not use hard decisions in the feedback section but the non-linearnon-linear MMSE MMSE estimates of the transmitted symbols;
• The non-linear MMSE estimates are computed on the basis of the A Posteriori ProbabilitiesA Posteriori Probabilities (APP, soft statistics) delivered at small delay by a MAP receiver;
Telcordia Technologies Proprietary - Copyright 2003.
Fig. 5
1.00E-06
1.00E-05
1.00E-04
1.00E-03
1.00E-02
1.00E-01
1.00E+00
7 8 9 10 11 12 13 14 15 16 17 18 19
SNR (dB)
BE
R
ZF-DFE
MAP/SDF[3/121]
MAP/HDF[3/121]
MAP/SDF[2/121]
MAP/HDF[2/121]
Practical application: enhanced DFE (cont.)
Telcordia Technologies Proprietary - Copyright 2003.
Practical application: multiuser detection
Successive Cancellation MUDSuccessive Cancellation MUD • Orders the users from the strongest to the weakest: P1>P2> … >PK;
• Detect the data sequence of the first (strongest) user;
• Subtract the decoded stream (hard decisions) from the observations;
• Detect the data sequence of the second user and so on;
KkdttriTtsib kkk ..., ,2 ,1 , )()(sgn)(ˆ
1
1
)(ˆ)()(k
llllk iTtsbPtrtr
Telcordia Technologies Proprietary - Copyright 2003.
Practical application: multiuser detection (cont.)
The soft approachThe soft approach(DARPA MIMO project, 2002)
• Do not use hard decisions in the subtraction of the decoded users but the non-linear MMSEnon-linear MMSE estimates of the transmitted symbols;
• Better performances are obtained if the fixed-lag smoothed estimates are used in place of the filtered ones;
w0 Detector
0ˆks
hk0
Hard-decisions
Conventional Hard SIC
w0APP
Computer
hk0
LinearTransformation
0
~ks
NL-MMSE Fixed-lag Smoothed Estimates of the Transmitted Symbol
Proposed Soft SIC
Telcordia Technologies Proprietary - Copyright 2003.
s(i)Unknown Channel
g(i) +
n(i)
y(i)Blind Equalizer s(i)^
Sufficient condition for channel identification: Probability distribution of must be equal to the probability distribution of {s(i)}.
This condition implies that the following cost function must be minimized:
, with p>2
Sato (‘75), Godard (‘80), Benveniste-Goursat (‘80), Picchi-Prati (‘80), Shalvi-Weistein (‘90)
Main disadvantages:•Slow convergence: several thousands of observation samples are needed to achieve channel identification;•The channel is estimated with a high residual MSE due to the use of nonconvex cost functions.•The channel estimate is not always available
)(ˆ is
pp ksksE )(ˆ)(
)(ˆ ig
Practical Applications: Blind Equalization
Telcordia Technologies Proprietary - Copyright 2003.
y(i)APPs
Computer
ChannelEstimator
G(i)^
(i/i)
Proposed blind equalizer
+
v(i)
z(i)Channel{g(k)}
MAPDetector
s(i-L+1)^
QAMModulator
s(i)
QAMDemodulator
S1S2
a(i-L+1)^
DifferentialEncoder
a(i)
DifferentialDecoder
The channel-estimator is fed with the soft information given by the APPs of the channel
Practical Applications: Blind Equalization (cont.)
IEEE Trans. on Signal Processing, July 2001
Telcordia Technologies Proprietary - Copyright 2003.
Channel: g=[1, 0, -1]
1.00E-05
1.00E-04
1.00E-03
1.00E-02
1.00E-01
1.00E+00
4 5 6 7 8 9 10 11 12 13 14 15 16
Eb/No (dB)
Bit
Err
or R
ate
No ISI (AWGN only)Known Channel with MAP detection (D=2)Blind MAP detection (D=2)Soft procedure
Practical Applications: Blind Equalization (cont.)
Telcordia Technologies Proprietary - Copyright 2003.
• As the transmitted bit-rate increases, ISI increases and powerfulAs the transmitted bit-rate increases, ISI increases and powerful
channel estimators become more necessary. channel estimators become more necessary.
• Hard-decision driven channel estimators are sub-optimal.Hard-decision driven channel estimators are sub-optimal.
• Optimal channel estimation implies the computation of the non-Optimal channel estimation implies the computation of the non-
linear filtered and fixed-lag smoothed estimates of the transmitted linear filtered and fixed-lag smoothed estimates of the transmitted
symbols. symbols.
• A new and simpler method for generating NL-MMSE filtered andA new and simpler method for generating NL-MMSE filtered and
fixed-lag smoothed estimates of the transmitted symbols via APPs fixed-lag smoothed estimates of the transmitted symbols via APPs
has been proposed. has been proposed.
Conclusions
Telcordia Technologies Proprietary - Copyright 2003.
Conclusions
• NL-MMSE filtered and fixed-lag smoothed estimates of theNL-MMSE filtered and fixed-lag smoothed estimates of the
transmitted symbols can be seen as transmitted symbols can be seen as optimaloptimal soft information, and soft information, and
their use is a consequence of the correct statement of the problem their use is a consequence of the correct statement of the problem
of MMSE estimation. of MMSE estimation.
•The proposed method makes SbS-MAP receivers very appealing.The proposed method makes SbS-MAP receivers very appealing.
• The proposed approach can be applied to all those problems thatThe proposed approach can be applied to all those problems that
admit a space-state representation. admit a space-state representation.
• Several fields of application, especially all those situations whereSeveral fields of application, especially all those situations where
hard decisions are employed despite their low reliability. hard decisions are employed despite their low reliability.