2011/7/16

Rong-Hui Peng

Efficient Markov Chain Monte CarloAlgorithms For MIMO and ISI channels

2011/7/162

Summary of Past work

• Efficient MCMC algorithms for communication– Application to MIMO channel [Chen-Peng-Ashikhmin-Farhang]– Application to noncoherent channel [Chen-Peng]– Application to ISI channel and extend to underwater channel [Peng-Chen-

Farhang]– Achieve near optimal performance with reduced complexity– Solve the slow convergence problem

• MIMO-HARQ schemes and combining algorithm [Peng-Chen]– Propose new retransmission scheme– Propose new combining algorithm– Increase the throughput significantly– Applicable to Wimax and LTE

2011/7/163

Summary of Past work

• Nonbinary LDPC codes [Peng-Chen]– Nonbinary LDPC coded MIMO system, achieve good performance with

reduced complexity– Code design based on EXIT chart– Hardware-friendly construction

• Low encoding complexity• Parallel architecture• Low error floor

• Intern at MERL– Low complexity MIMO detection algorithm for MIMO-OFDMA– Wimax link-level simulation– 3G LTE antenna selection– 1 paper, 1 patent

Summary of Past work

• Consultant at WiderNetworks– Synchronization and signal detection for 3G LTE driver scanner

• Work at SpiralGen– Vectorizing physical layer algorithm and map it to multiple-core processor– Implement physical layer software components in software

communication architecture (SCA)

2011/7/164

2011/7/165

Outline

• Background and motivation

• Introduction to MCMC technology

• MCMC MIMO detection

• MCMC ISI equalization

• Conclusion

2011/7/166

Background and motivation

ky ˆkuENC MOD CHANNEL DEM/DET DEC

k̂bku kb kdkh

y Hd nMAP detection:

ˆ max ( | , ) ( | , ) ( | , ) ( )

k

k k

k

b P bP b p P

b

y Hy H Y H d d

MAP detector : Optimal performance but exponential complexity

Reduction in computational cost is essential for application tofuture system.

JDD

2011/7/167

Our contribution

• Previous work on MCMC methods for signal processingand communication [Doucet-Wang’05]– Bit-counting: use MCMC to determine the frequency over which a

bit occurs.– Many samples are needed.– Requires a burning period to allow Markov chain to converge.– Do not consider convergence problem

• Our proposed MCMC methods– Do not use bit-counting– No burning period is needed– Fewer samples even for large systems– Solve the slow-convergence problem

2011/7/168

Markov Chain Monte Carlo (MCMC)

• MCMC obtains the statistical inference by sampling froma posterior distribution through Markov chain

• MCMC is suitable for addressing problems involving high-dimensional summations or integrals

• Instead of evaluating all summation terms (exponentialcomplexity), average over the samples from the complexdistribution

X

f(x)

2011/7/169

Gibbs sampler

• A MCMC method sampling from a multivariate distributionIdea: Sample from the conditional of each variable given the settings

of the other variables

Repeatedly:

1) pick i (either at random or in turn)2) replace by a sample from the conditional

distribution

Gibbs sampling is feasible if it is easy to samplefrom the conditional probabilities.

This creates a Markov Chain

),,,,,|( 111 niii xxxxxp

ix

)3()2()1( xxx

Example: 20 iterationsof Gibbs sampling on abivariate Guassian

2011/7/1610

Why Gibbs sampling works

• Retains elements of the greedy approach– weighing by conditional PDF makes likely to move towards locally

better solutions

• Allows for locally bad moves with a small probability, toescape local maxima

2011/7/1611

Limitations

• May have slow convergence problem– High posterior correlation, hard to move in other directions– Multi-modal distribution, proposing small changes causes the

move between modes to become rare

P(x)

x

Trapped inlocal states

High posterior correlation Multi-model distribution

2011/7/1612

MIMO channel

…TX

0d

1d

1tNd

+

+

+

RX

0n

1n

1rNn

0y

1rNy

1y

antennasrxand txofnumber theis, vectornoiseadditiveanis

matrixgainchannel theis

signalreceivedof vectoraissymbols transmitof vectorais

where

rt

N

NN

N

N

NN

r

tr

r

t

CnCHCyCd

nHdy

System Diagram of MIMO system

2011/7/1613

SDD: Separate detection and decodingIDD : Iterative detection and decoding

2011/7/1614

MIMO detection

• Maximal likelihood (ML) detection or Maximum a posteriori (MAP),optimal but exponential complexity

• Low complexity sub-optimal detectors (ZF, MMSE, VBLAST),performance gap can be 20 dB

• Tree search based (sphere decoding (SD), QRD-M)– Excellent performance at high SNR– SD has variable complexity depending channel condition– Exponential complexity at low SNR– Complexity increase quickly with large system

• Markov chain Monte Carlo (MCMC)– Constant complexity, not increased much for large system– Excellent performance at low SNR– High SNR problem

2011/7/1615

Low complexity detection

• Approximate optimal detectors

The searching is performed over a small subset (Important set) insteadof a large full set.

( 1, | )( 1| )ln ln( 1| ) ( 1, | )

max ( 1, | ) max ( 1, | ) ln ln

max ( 1, | ) max ( 1, | )

where

k

k

k k

k k

k kk

kk k k

k k k k

k k k k

k k

P bP bP b P b

P b P b

P b P b

b

b

b

b

b yyy b y

b y b y

b y b y

b

I

I

I

kI

2011/7/1616

MCMC MIMO detector

• MCMC detector finds important sets using Markov chainMonte Carlo– Create Markov chain with state space: d and stationary

distribution P(d|Y)– Run Markov chain using Gibbs sampler– After Markov chain converge, the samples are generated

according to P(d|Y). Those samples with large P(d|Y) generatedwith high probabilities

• The samples generated by Gibbs sampler are used tocompute the soft message for soft decoder

2011/7/1617

MCMC MIMO detector (bit-wise)

(0)

( ) ( 1) ( 1)0 0 1 1

( ) ( ) ( 1) ( 1)1

(

1 1 0 2

)1

1 to draw ( | , , , )

S1. Generate init

draw ( | , , , , )

ialS2. FOR

draw

t c

t c

t c

n n nN M

n n n nN M

nN M

n Lb p b b b b

b p b b b b b

b

b

y

y:

L

L

M

:

( )0

(

1( )

2

)

( | , , , )

Add to important set END FORS3. Calculate LLR

max ( 1, | )ln

max ( 1, | )

t c

k

k

k k

kk k

n nN M

n

p b

P b

P

b b

b

b

y

b

b y

b y

L:

I

I

I

~~

~

2011/7/1618

MCMC detector

• Calculate transition probability

( ) ( ) ( 1) ( 1)0 1 1 1

( ) ( ) ( 1) ( 1)0 1 1 1

( | , , , , , )

( | , , , , , ) ( )

( | ) ( )

t c

t c

n n n ni i i N M

n n n ni i i N M i

i

p b b b b b b

p b b b b b b p b b

p p b b

y

y

y d

L L

L L

Simulation results (Low SNR)

• MCMC with LDPC performs best in performance and complexity• Within 1.8 dB of capacity at 24 bits/channel use• Turbo LSD is 25 times higher in running time

2011/7/1619

2011/7/1619

Performance of turbo and LDPC coded TX8 64QAM systems, code length 18432

6 7 8 9 1010-4

10-3

10-2

10-1

Eb/N0 (dB)

BE

RLDPC Log-MAP MCMC 10x10 G=1Turbo Log-MAP MCMC 10x10 G=0.7Turbo Log-MAP LSD L=100 G=26

Capacitylimit

2011/7/1620

High SNR problem

• At high SNR, MCMC takes long time to converge, leadsperformance degradation, this is because of themultimodal property of channel PDF at high SNR

P(y|d)

dLow SNR

P(y|d)

dHigh SNR

Trapped inlocal states

2011/7/1621

Solutions

• Run L independent parallel Gibbs samplers and eachGibbs sampler run I iterations

• Generate good initial candidates using other lowcomplexity detector (warm start)– MMSE/ZF

• Hybrid QRD-MCMC

2011/7/1622

QRD-M

'4

'3

'2

'1

4

3

2

1

4,4

4,33,3

4,23,22,2

4,13,12,11,1

4

3

2

1

00000

0'

nnnn

dddd

rrrrrrrrrr

zzzz

H nRdyQz

• QR decomposition

Full path (MLD)

Select M path (QRD-M)

QPSK, M=1

2011/7/1623

• Combine QRD-M and MCMC– A QRD-M with a small M is running first to generate initial

important sets– The bit sequence with minimal path metric will be used to initialize

one of L parallel MCMC– The important set produced by the QRD-M detector is

incorporated by the MCMC detector– MCMC is running to generate refined important set– The soft message is computed using refined important set

Hybrid QRD-MCMC

2011/7/1624

Simulation Results

8x8 16QAM LDPC coded SDD and IDD systems, code length 2304

9 9.5 10 10.5 11 11.5 1210

-5

10-4

10-3

10-2

Eb/N0(dB)

BE

R

QR32QR8 MCMC5x5RND-MCMC10x5QR32QR8 MCMC5x5RND-MCMC10x5

SDDIDD

2011/7/1625

11 12 13 14 15

10-5

10-4

10-3

10-2

Eb/N0(dB)

BE

R

QR64QR16 MCMC10x5MCMC15x15QR64QR16 MCMC8x6MCMC12x12

IDD SDD

4x4 64QAM LDPC coded SDD and IDD systems, code length 2304

Simulation Results

2011/7/1626

Complexity

8x8 16QAM 4x4 64QAM

2011/7/1627

Summary of results

• MCMC detector for MIMO channel is studied

• Low complexity for large system

• Excellent performance at low SNR region

• High SNR problem can be solved by combining QRD-Mwith MCMC without increasing complexity

2011/7/1628

ISI channel

• Multipath fading channel

0( ) ( ) ( ) ( ),

for 0,1, , 1

L

ll

y m h m d m l n m

m N L

L

DD

+

)(md

)(0 mh 1 ( )h m 2 ( )h m ( )Lh m

)(mn )(my

…

2011/7/1629

ISI channel

• In matrix form

1

1

1

where : transmitting vector : received vector : channel matrix : dditive noise vector : Equalizer block length :

N

N L

N L N

N L

NL

y Hd n

d Cy CH Cn C

Channel memory

0

1 0

0

1 0

1

0 0 00 0

0 0 0

0 00 00 00 0

L

L L

L L

L

hh h

h h

h h h

h hh

H

L LL

M O O ML L

M O O ML LL O MLL L

2011/7/1630

Detection for ISI channel

• MAP detection is still optimal– Efficient implementation of MAP detection is Forward backward

algorithm (BCJR)– Still exponential complexity with channel memory

• Low complexity algorithm– MMSE– Decision-feedback– …

2011/7/1631

Bit-wise MCMC detector

• Transition probability

1

:0

1 1

: : :0 1

:

( ) ( )0 1

( | , ) ( | ) ( )

( | ) ( )

( | ) ( )

( | ) ( | ) ( | ) ( )

( | ) ( )

{ , , , ,

{ }

ak k k

ak

N La

j j L j kj

i N L i La a a

j j L j j j L j j j L j kj j i L j i

i La

j j L j kj i

a n nk

P b a p P b ap P b a

p y P b a

p y p y p y P b a

C p y P b a

b b a

b y y by d

d

d d d

d

b

g

L ( 1) ( 1)1 1

denote the symbol vector corresponding to

,

is mapped

, },

to

b

n nk M

k

N

a a

ib d

b b

d b

L

2011/7/1632

Bit-wise MCMC detector

• Computing the a posteriori LLR– Accurate posteriori LLR involve computations over the whole

block (may be very cumbersome since N can be very large)– Truncated window to approximate

2

1 2,0

1: :1 2 1 2

2

1 2,1

1: :1 2 1 2

1 2

: :

1, 2,

: :

:

1 2

ln

the set which truncate seque

( | ) ( )

( | ) ( )

n

with bits {

ces in

, }

ki i i i

ki i i i

i

i i L i i ll ie e

k ki

i i L i i ll i

i i

k

p P b

p P b

b i k i

b

b

y b

y b

I

I

I I

2011/7/1633

Severe ISI channel

• For channels with severe ISI,bit-wise MCMC suffers fromslow convergence problem

• We found slow convergenceproblem for ISI channel ismainly caused by highposterior correlation

0 0.2 0.4 0.6 0.8 1-60

-50

-40

-30

-20

-10

0

10

Normalized Frequency

PSD

, dB

C1C2

1

2

[ ] 0.227 [ ] 0.46 [ 1] 0.688 [ 2] 0.46 [ 3] 0.227 [ 4]

[ ] (2 04 ) [ ] (15 18 ) [ 1] [ 2] (12 13 ) [ 3] (08 16) [ 4]

h n n n nn n

h n i n i n ni n n

2011/7/1634

Severe ISI channel

• Our solution– Grouping (blocking) the highly correlation variables in Gibbs

sampler

1st iteration:

2nd iteration:

3rd iteration:

4th iteration:

Shift grouping different adjacent symbols over iterationsto speed up the mixing rate of Gibbs sampler

2011/7/1635

Transition probability

• Grouping multiple variables allow a large sample spacecontaining values

0 00

1 0

1 1

0

1 0

1 1

1

1 1

0 0 00 0

0 0 0

0 00 00 00 0

i i

i iL

i P i GL L

i P i G

L L

N L NL

y dhh h

y dy dh h

y dh h hy d

h hy dh

L LM ML

M O O ML L

M MM O O ML LL O M

M MLL L

0

1

1

1

i

i

i P

i P

N L

n

nn

nn

n

M

M

M

2 bM Gr

1 LGP

2011/7/1636

Transition probability

• Can be considered as received signals of a MIMOchannel with G transmit antenna and P+1 receiveantenna

• QRD-M is applied to find important states withlarge APPs

• Gibbs sampler randomly jump to one state from thoseimportant states

: 1 :: : : : 1 : :

::

:

: 1 :

i L i i G i Pi i P i i P i i P i L i i i P i G i P i i P

i i Gi G i Pi i P i i P

y y H d H dH

nd n

%

rr 0

2011/7/1637

Simulation results

Performance comparison of MAP, MMSE, g-MCMC equalizer for a strong ISI channel

• A channel with strong ISI

• g-MCMC significantly outperforms MMSE equalizer (Tuchler-Singer-Koetter’02)

• b-MCMC does not work for such channel with strong ISI

]4[227.0]3[46.0]2[688.0]1[46.0][227.0][ nnnnnnh

2011/7/1638

Summary of results

• MCMC equalizer for ISI channel is studied

• Near optimal performance can be obtained

• Slow convergence is mainly caused by high posteriorcorrelation

• QRD-M algorithm is applied to reduce the complexity ofgroup-wise MCMC equalizer

• Parallel implementation is proposed

2011/7/1639

Conclusions

• Efficient MCMC algorithms for MIMO and ISI channel are studied– For MIMO, MCMC works well at low SNR region– For ISI, MCMC works well for moderate ISI

• Solutions for slow convergence are proposed– For MIMO, use QRD-M as good start point– For ISI,

• Use group-wise Gibbs sampler to group high correlated variables• Use QRD-M to reduce the complexity of group-wise Gibbs sampler

• Future work– Study the sensitivity of proposed MIMO detector with more practical

channel model– Study the time-varing ISI channel and adaptive grouping scheme for

group-wise MCMC– Hardware implementation of MCMC

2011/7/1640

Thank you !