2011/7/16

Rong-Hui Peng

Department of Electrical and Computer EngineeringUniversity of Utah

Efficient Markov Chain Monte CarloAlgorithms For MIMO and ISI channels

2011/7/162

Summary of PhD work

• Efficient MCMC algorithms for communication– Application to MIMO channel ([Chen-Peng-Ashikhmin-Farhang], To

appear IEEE Trans. Comm.)– Application to noncoherent channel ([Chen-Peng], to appear IEEE Trans.

Comm.’09)– Application to ISI channel and extend to underwater channel ([Peng-

Chen-Farhang], To appear IEEE Trans. Signal Process.)– Achieve near optimal performance with reduced complexity– Solve the slow convergence problem

• MIMO-HARQ schemes and combining algorithm ([Peng-Chen], to besubmitted to IEEE Trans. Comm.)– Propose new retransmission scheme– Propose new combining algorithm– Increase the throughput significantly– Applicable to Wimax and LTE

2011/7/163

Summary of PhD work

• Nonbinary LDPC codes ([Peng-Chen], IEEE Trans. WirelessComm.’08)– Nonbinary LDPC coded MIMO system, achieve good performance

with reduced complexity– Code design based on EXIT chart– Hardware-friendly construction

• Low encoding complexity• Parallel architecture• Low error floor

• Intern at MERL– Low complexity MIMO detection algorithm for MIMO-OFDMA– Wimax link-level simulation– 1 paper, 1 patent

2011/7/164

Outline

• Background and motivation• Introduction to MCMC technology• MCMC MIMO detection

– Review of MIMO detection– QRD-M and MCMC detector– Hybrid MIMO detector– Complexity analysis

• MCMC ISI equalization– Bit-wise MCMC equalizer– Group-wise MCMC equalizer

• Conclusion

2011/7/165

Background and motivation

ky ˆkuENC MOD CHANNEL DEM/DET DEC

k̂bku kb kdkh

y Hd nMAP detection:

ˆ max ( | , ) ( | , ) ( | , ) ( )

k

k k

k

b P bP b p P

b

y Hy H Y H d d

MAP detector has optimal performance but the complexity isexponential with the dimension of

Our objective: reduce the complexity but still obtain nearoptimal performance

kb

IDD

2011/7/166

Markov Chain Monte Carlo

• MCMC is a class of algorithms for sampling fromprobability distributions based on constructing a Markovchain that has the desired distribution as its stationarydistribution.

• The state of the chain after a large number of steps isthen used as a sample from the desired distribution.

• MCMC is suitable for addressing problems involving high-dimensional summations or integrals

X

f(x)

2011/7/167

Gibbs sampling

• One kind of the MCMC methods.• The point of Gibbs sampling is that given a multivariate distribution it

is simpler to sample from a conditional distribution rather thanintegrating over a joint distribution.

• Generate an initial sample as start state• Using conditional distribution as state transition probability• Each state jumping allows only one variable change• Return best state seen across all iterations (may not be the last one)• Stop after a fixed number of iterations• Solution is sensitive to the starting state, so we typically run the

algorithm several times from different starting points

2011/7/168

State transition in Gibbs sampling

d d1

d2

d3

State d3 d2 d1

S0 -1 -1 -1

S1` -1 -1 +1

S2 -1 +1 -1

S3 -1 +1 +1

S4 +1 -1 -1

S5 +1 -1 +1

S6 +1 +1 -1

S7 +1 +1 +1

y Hd n

),,|(PMFlconditionausing),|( toaccordingSampling

iidpp

dHyHydd

2011/7/169

Gibbs sampler

(0)

( ) ( 1) ( 1) ( 1)0 0 1 2 1( ) ( ) ( 1) ( 1)1 1 0 2 1

Generate initial randomlyfor n= 1 to generate from distribution ( | , , , , )

generate from distribution ( | , , , , )

g

n n n nN

n n n nN

Id p d b d d dd p d b d d d

d

yy

L

LM

( ) ( ) ( ) ( )1 1 0 1 2enerate from distribution ( | , , , , )

end for

n n n nN N Nd p d b d d d yL

2011/7/1610

Why Gibbs sampling works

• Retains elements of the greedy approach– weighing by conditional PDF makes likely to move towards locally

better solutions

• Allows for locally bad moves with a small probability, toescape local maxima (with limitations)

2011/7/1611

ISI channel

• Multipath fading channel

0( ) ( ) ( ) ( ),

for 0,1, , 1

L

ll

y m h m d m l n m

m N L

L

DD

+

)(md

)(0 mh 1 ( )h m 2 ( )h m ( )Lh m

)(mn )(my

…

2011/7/1612

ISI channel

• In matrix form

1

1

1

where : transmitting vector : received vector : channel matrix : dditive noise vector : Equalizer block length :

N

N L

N L N

N L

NL

y Hd n

d Cy CH Cn C

Channel memory

0

1 0

0

1 0

1

0 0 00 0

0 0 0

0 00 00 00 0

L

L L

L L

L

hh h

h h

h h h

h hh

H

L LL

M O O ML L

M O O ML LL O MLL L

2011/7/1613

Detection for ISI channel

• MAP detection is optimal– Efficient implementation of MAP detection is Forward backward

algorithm (BCJR)– Still exponential complexity with channel memory

• Low complexity algorithm– MMSE– Decision-feedback– …

2011/7/1614

Bit-wise MCMC detector

• Transition probability

1

:0

1 1

: : :0 1

:

( ) ( )0 1

( | , ) ( | ) ( )

( | ) ( )

( | ) ( )

( | ) ( | ) ( | ) ( )

( | ) ( )

{ , , , ,

{ }

ak k k

ak

N La

j j L j kj

i N L i La a a

j j L j j j L j j j L j kj j i L j i

i La

j j L j kj i

a n nk

P b a p P b ap P b a

p y P b a

p y p y p y P b a

C p y P b a

b b a

b y y by d

d

d d d

d

b

g

L ( 1) ( 1)1 1

denote the symbol vector corresponding to

,

is mapped

, },

to

b

n nk M

k

N

a a

ib d

b b

d b

L

2011/7/1615

Bit-wise MCMC detector

• Computing the a posteriori LLR– Accurate posteriori LLR involve computations over the whole

block (may be very cumbersome since N can be very large)– Truncated window to approximate

2

1 2,0

1: :1 2 1 2

2

1 2,1

1: :1 2 1 2

1 2

: :

1, 2,

: :

:

1 2

ln

the set which truncate seque

( | ) ( )

( | ) ( )

n

with bits {

ces in

, }

ki i i i

ki i i i

i

i i L i i ll ie e

k ki

i i L i i ll i

i i

k

p P b

p P b

b i k i

b

b

y b

y b

I

I

I I

2011/7/1616

Severe ISI channel

• For channels with severe ISI,bit-wise MCMC suffers fromslow convergence problem

• We found slow convergenceproblem for ISI channel ismainly caused by highposterior correlation

0 0.2 0.4 0.6 0.8 1-60

-50

-40

-30

-20

-10

0

10

Normalized Frequency

PSD

, dB

C1C2

1

2

[ ] 0.227 [ ] 0.46 [ 1] 0.688 [ 2] 0.46 [ 3] 0.227 [ 4]

[ ] (2 04 ) [ ] (15 18 ) [ 1] [ 2] (12 13 ) [ 3] (08 16) [ 4]

h n n n nn n

h n i n i n ni n n

2011/7/1617

Severe ISI channel

• Our solution– Grouping (blocking) the highly correlation variables in Gibbs

sampler

1st iteration:

2nd iteration:

3rd iteration:

4th iteration:

Shift grouping different adjacent symbols over iterationsto speed up the mixing rate of Gibbs sampler

2011/7/1618

Transition probability

• Grouping multiple variables allow a large sample spacecontaining values

0 00

1 0

1 1

0

1 0

1 1

1

1 1

0 0 00 0

0 0 0

0 00 00 00 0

i i

i iL

i P i GL L

i P i G

L L

N L NL

y dhh h

y dy dh h

y dh h hy d

h hy dh

L LM ML

M O O ML L

M MM O O ML LL O M

M MLL L

0

1

1

1

i

i

i P

i P

N L

n

nn

nn

n

M

M

M

2 bM Gr

1 LGP

2011/7/1619

Transition probability

• Can be thought as received signals of a MIMO channelwith G transmit antenna and P+1 receive antenna

• Can apply QRD-M to find samples with large APPs• Gibbs sampler randomly chooses one sample from

samples

: 1 :: : : : 1 : :

::

:

: 1 :

i L i i G i Pi i P i i P i i P i L i i i P i G i P i i P

i i Gi G i Pi i P i i P

y y H d H dH

nd n

%

0r r=

0r

2011/7/1620

Simulation results

Performance of the channel with strong ISI

2011/7/1621

Summary of results

• MCMC equalizer for ISI channel is studied• Near optimal performance can be obtained• Slow convergence is mainly caused by high posterior

correlation• QRD-M algorithm is applied to reduce the complexity of

group-wise MCMC equalizer• Parallel implementation is proposed

2011/7/1622

Conclusions

• Efficient MCMC algorithms for MIMO and ISI channel are studied– For MIMO, MCMC works well at low SNR region– For ISI, MCMC works well for moderate ISI

• Solutions for slow convergence are proposed– For MIMO, use QRD-M as good start point– For ISI,

• Use group-wise Gibbs sampler to group high correlated variables• Use QRD-M to reduce the complexity of group-wise Gibbs sampler

• Future work– Study the sensitivity of proposed MIMO detector with more practical

channel model– Study the time-varing ISI channel and adaptive grouping scheme for

group-wise MCMC– Hardware implementation of MCMC

2011/7/1623

Motivation

• Optimal binary code has been designed to approach channel capacity.– Long codes– Irregular

• Nonbinary LDPC code design has been studied for AWGN and showsbetter performance than binary codes.

– Shorter codes– More regular

2011/7/1624

Contribution

• Apply nonbinary LDPC codes to fading channels and MIMO channelsand provide comparison with optimal binary LDPC coded systems

• Propose modified nonbinary LDPC decoding algorithm.• Extend EXIT chart to nonbinary code design• Propose parallel sparse encodable nonbinary code with low encoding

and decoding complexity

2011/7/1625

Introduction of binary LDPC codes

• A subclass of linear block codes• Specified by a parity check matrix (n-k)×n

n: code length k: length of information sequence

+

+

+

VariableNodes

checkNodes

x1

x2

x3

x4

x5

x6

x7

c1

c2

c3

100110101011100010111

3

2

1

721

0Hx

H

T

ccc

xxx

0:0:0:

74313

64322

53211

xxxxcxxxxcxxxxc

2011/7/1626

Definition of nonbinary LDPC codes

• For nonbinary codes, the ones in parity check matrix are replaced bynonzero elements in GF(q)

500720403065200030173

H

2011/7/1627

Application to fading channels

• Channel model

VHSX Mρ

Assume each entry of channel matrix is independent, followsRayleigh fading, and is known by receiver

2011/7/1628

System block diagram

Separate detection and decoding: the detection is performedonly once.

2011/7/1629

Performance comparison

Performance comparison of GF(256) SDD, GF(16) JDD and binary JDDsystem

GF16JDD

GF256SDD

BinaryJDD

802.16e

Performance:GF256>GF16>binary>802.16e

Complexity:GF16<GF256802.16e<binary

2011/7/1630

Construction of Parallel sparse encodable codes

• Motivation– Low encoding complexity– Allow parallel implementation

• Parallel sparse encodable (PSE) codes =Qausi cyclic (QC) cycle codes + tree codes– QC codes: parallel implementation– Cycle code: Low encoding complexity

2011/7/1631

Quasi-cyclic construction

• Quasi-cyclic structure

• Ai,j is a circulant: each row is a right cycle-shift of the row above it and the firstrow is the right cycle-shift of the last row

• The advantage of QC structure– Allow linear-time encoding using shift register– Allow partially parallel decoding– Save memory

nmmm

n

n

,2,1,

,22,21,2

,12,11,1

AAA

AAAAAA

H

2011/7/1632

A new QC structure for GF(q)

000

000

0000

000

2,

2,

,

,

,

'qji

ji

ji

ji

ji

A

• Ai,j is a nonbinary multiplied circulant permutation matrix

)GF(ofelementprimitiveais

},,,{fromchoosenrandomlyis)'GF(ofelementprimitiveais);GF()'GF(

1)1'/(1(10,

q

qqqqq

ji

2011/7/1633

Cycle codes

• With degree 2 variable nodes only• Can be represented by normal graph, every vertex imposes one linear

constraint– Columns => edges; rows => vertices

101100010110100011011001

mH

b0 b1 … b5

c0

c3

c0 c1

c2c3

b0

b2

b3 b1

b4 b5

Parity check matrixNormal graph representation

2011/7/1634

H based encoding of cycle codes

① Find the spanning tree:b0, b3, b4

② Information bits =>Edges outside SPb1=1, b2=1, b5=0

③ Compute coded bitsb3=b2+b5=1b4=b2+b1=0b0=b1+b5=1

c0 c1

c2c3

b0

b2

b3 b1

b4 b5

0 1

1

1 0

1

Not work for nonbinary cycle codes! Why?

For binary code: check c0 is always satisfied.b0+b3+b4=0

2011/7/1635

PSE codes

03c

02c

03b

01c

00c

04b

00b

01b0

2b

05b

13c

12c

13b

11c

10c

14b

10b

11b1

2b

23c

22c

23b

21c

20c

24b

20b

21b2

2b

15b

25b

8b

5c6b

7b

9b

PSE codes = QC cycle codes + tree codes

2011/7/1636

Encoding of PSE codes

• Parity check matrix based encoding• Parallel encoding for QC cycle subcode• Much lower encoding complexity than normal LDPC codes using

generator matrix based encoding because the generator matrix isusually dense

2011/7/1637

Simulation results

Performance comparison of PSE codes over GF(256)with QPP (QC) codes and PEG (non-QC) codes

2011/7/1638

Thank you !