,

\

PARTIAL RESPONSE SIGNALING WITH A MAXIMUM.t' .

LIKELIHOOD SEQUENCE ESTIMATION RECEIVER

by

N. Chan, B.Eng.

\

A Thesis .

Submitted to the Faculty of Graduate Studies

in Partial Fulfilmen~ of the Requirements

for the Degree

Master of Engineering

'~ ' ..

McMaster University

December 1980

MASTER OF .ENGINEERING {1980)Electrical and ~omputer Engineering

McMASTER UNIVERSITYHamilton~ Ontario

•~ .

TITLE: Partial Response Signaling with a Maximum LikelihoodSequence Estimatio~ Receiver

AUTHOR: N. Chan~ B.Eng. (Concordia University)

SUPERVISOR: Dr. J. B. Anderson

NUMBER OF PAGES: vii, 115

·...

ABSTRACT

This thesis evaluates a bandwidth efficient data transmitting

system woich is modelled as a PRS system. A maximum likelihood,

receiver implementing VA is assumed at,the receiving end. An algor­

i thm is developed to compute a fundamental performance parameter of

the system, called the free distance..

99% energy bandwidth and

intersy~bol in~erference (lSI) degradation are used to measure the

performance of the system.\

\

Nonlinear progr9mming and minimax methods

are applied to find the optimal channel codes under different criteria.

Three different set~ of optimal channel codes have been found; first,

the worst~case channel codes in terms of lSI degraQation, secondly, ,~

the minimum 99% energy bandwidth channel codes and finally the mini-

mum 99% energy bandwidth channel with fixed lSI degradation constraint.

Two PRS systems with different pulse shaping filter ar~ considered.

First, an ideal low pass filter with minimum- Nyquist band\"idth is

evaluated for channel lengths up to twelve. Then a spectra1 raised~

cosine filter with roll-off factor equal to one is evaluated for~

channel lengths up to four. The two PRS systems show that a longer

channel can have better performance in- consideration of both band­~

width and lSI degradation. the raised cosine filter causes no per-

formance penalty in narrow band channels.

; i i

).

ACKNOWLEDGEMENTS

The author wishes to 'thank Dr. J. B. Anderson for his. .

guidance and inspiration throughout the course of this program.

The author would also like to express ~is appreciation

from a number of useful discussions with Dr. M. R. M. Rizk.

The financial assi.stance provided by the Department of

~ Electrical and Computer Engineering is gratefully acknowledged.

Finally, the author also wishes to thank Ms. linda Hunter

for typing this thesis.

I

iv

, .

• •

· .

,

ITABLE OF CONTENTS

6

p~e

CHAPTER 1 INTRODUCTION

CHAPTER 2 ABANDWIDTH EFFICIENT SIGNALING SCHEME: PARTIAL ?RESPONSE SIGNALING

2.1 Partial Response Signaling System (PRS) 7

2.2 Choice of Pulse ~ha~ing Filter G(w) '10

2.3 Energy. and Bandwidth of PRS System 13

2.4 Finite State Machine 14

2.5 Tree and Trellis

CHAPTER '3 MAXIMUMt1RELIHOOD RECEIVER

3.1 A Nonlinear .Receiver Structure

15

17

17

3.2 Maximum Likelihood SequenGe Est}mation 20for PRS Systems .

\

3.3 Modifi€d Vjterbi Algorithm 22

\

3.4 Performance of the Receiv~r

3.5 Some Discussion

CHAPTER 4 COMPUTATION OF MINIMUM DISTANCE - dfree4.1 Represe~tation gf d?f

2 ree4.2 Properties of dfree

4.3 An'Algorithm for Computation of d~ree

4.4 Complexity and Efficiency of the ModifiedStack Algorithm

v

25

28

31<

31

33

35

38

I ~

page

CHAPTER 5 OPTIMISATIO~ TECHNIQUES

5.1 Problem Statement

43(43

5.2 The Kuhn-Tucker Conditions 45

5.3 Techniques to Solve the Constrained Problem 46, .

5.4 A Computer Package Fl~pt 5 48

~ 5.5 Application to Our Work 50

CHAPTER 6 THE MINIMUM FREE DISTANCES OF PRS SYSTEMS WITH 51CHANNEL LENGTHS UP TO 12

6.1 Normali~ed d2f and Degradation 51

'. ree6.2 Computation of the Worst Possible Degradation 53

.6.3 Analysis of the R€sults 55

CHAPTER 7 MINIMIZATION OF BANDWIDTH WITH AND WITHOUT POWER 63CONSTRAINTS

-\0

7.1 DefinitiDn of 99% Energy Bandwidth 63

7.2 Minimization of Bandwidth 66

7.3 Discussion of the Results 67

7.4 Minimization of Bandwidth with DB Constraint 73

1/

CHAPTER 8

7.5 Discussion of the Optimal Bandwidth' Channelwith DB Constraint

CONSIDERATION OF RAISED-COSINE FUNCTION FORPULSE SHAPING .

75

84

8.1 Bandwidth of a PRS System with Raised-Cosine 84Filter . I

8.2 Minimum Bandwidth of the PRS System with a' 88Raised-Cosine Filter

vi

f

page

8.3 Degradation of the PRS System

8.4 The Worst Possible Degradation

93

98

8.5 Minimization of Bandwidth with Degradation 98Constraint

CHAPTER 9 CONCLUSION 102\.t

tAPPENDIX A 107

APPENDIX B 109

""'-REFERENCES t 113

\

(l>

vii

\ '

CHAPTER 1

INTRODUCTION

1.1 A.Digital Commun~ation System

Increasing demand for faster, bandwidth efficient an~ reliable

data transmission has caused interest in various signaling schemes and

receivers to fulfil these objectives. For instance, on a satellite

communication ~ystem an~a mobile radio system, ?oth the transmission

bandwidth and signal to nciise ratio 1SNR) are of concern. Since the

deman~r communication·is increas4ng rapidly, the efficient use'of

channel bandwidth is important to reduce the channel cost directly

and conserve the radio spectrum. The immense transmission distance,

limited power supply and/or unfavourable channels of these c6mmun­

ication systems )increase the.im~ortance of reliable data transmission

with manageable transmission power.

The primary sources of distortion in a high data rate trans­

mission.system 'are intersymbol interference (lSI) and noise. lSI is

produced by the tail of a baseband pulse at each symbol period which

interferes with the neighbour pulses. Multipath interference has a

similar effect.

The purpose of this thesis is to design a bandwidth efficient•

transmitting system, a so called partial response signal.ing (PRS)

system or correlative encoder. A maximum likelihood (ML) receiver

employs the Viter?i Algorithm (VA) to decode the received sequence.

1

~'\

2

Fig~ 1.1 shows the digital communication system model that is used in

our work. !he inpJt data is assumed to be a sequence of impulse {O,l},

and the transmitting filter correlates the input sequence for spectral

shaping to achieve better bandwidth performance. A bandlimited channel

corrupteG by white Gausstan noise is. assumed. On the receiver1s side,

a matched filter and a maximum likelihood sequence estimator implementing'"

the V~ are used as an optimum receiver.,

Kabal and Pasupathy [lJ presented a detailed study ojY01fferent

PRS schemes from the viewpoint of spectral properties. PRS schemes are

based Dn the idea of introducing some correlation (or lSI) among the

data for spectral shaping so as to increase bandwidth efficiency.

Since the controlled amount of lSI is introduced to the adjacent

symbols in a finite discrete way, the lSI can be eliminated at the,

receiver by subtracting the previ~us weighted symbols from present

symbol.

Various receiver structures have been proposed to combat

the effect of lSI [2J-[5]~ One simple struct~re used in pract{ce is

called the Jinear equalizer. Its performance has been analysed by\

Lucky [2]. Another kind of receiver that implements a non-linear

technique, called the decision feedba~ ~qua1izer, was anglysed by

Price [3]'. Amore recent nonlinear r.eceiver was proposed by Forney

[5] which consists of a whitened matched filter, a sampler with symbol

rate and a recursive non1 inear p'rocesso'r, called the Viterbi Algorithm

(VA). The VA was devised by Vite:bi [6] as a decoding algorithm for

convolutional codes.' Omura [7J pointed out that the VA can be ~on-

.'

\\

\

\

sidered ilS a dynamic programming splution to ~ decoding problem. 3

Forn~y [3] and Kobayashi [8] . showed . that the A is ,indeed amaximum likelihood decoder performing maximum llkelihood estimation,.(MLSE) in pulse amplitude modulation (PAM) system wi h lSI. The

\performance analysis of a MLSE implementing the VA w S reported

, '

•by Forney [3J, [9J. A simil ar recei ver was proposed b Ungerboeck

[4], in which the VA was modified to deal with correla~ive noise

__directly without a pre-whitening filter. An _adapti.v.e"F-Ae~e for the

'receiving filter is ~lso proposed in the same paper. ~agee [10].- -~ --- - I'

. I

proposed another ~daptive ML receiver WhiCh'com~ines rorn~yls ~eceiver

with a cha~~el estimator so that it can be applied tQ\ an unknown slowly• • #' )

time-varying dispersive channel.

Forney showed that the probability of error of MLSE is domin­

ated by the minimum distance at moderate SNR (5]. Magee and Proakis,

[llJ suggested a method to estimate the worst possible performance- .

/ over a chennel with a fixed energy finite duration pulse response.

The results for estimated worst performance were shown for channel

lengths up to ten_ Anderson and Foschini [12J later provided a

procedure for computing the minimum distance for classes of a'few

hundred states. A combined functional analysis and computer search-.

l).

approach was used to find the minimum distance.~

Their results indic-

ated that Magee's results [11] were only true for the channe1 lengths

less than seven.

An ~val'uation on power, bandwidth and complexity in MLSE

was done by Wong [13J. The transmitter was modelled as a partial

4

.r~sponse signaling system, while Forney's receiver was applied at

the receiving end. 'A double dynamic programming technique was

developed and used for camputation of the minimum dis~~nce...

Power,

bandwidth and complexity were evaluated for ~hqgnel lengths up ,to

four by plotting the corresponding contour maps.'

,, .

\1.2­

I/

Organization of the Thesis

In_,Chapter 2--of this tflesi-s,~-PRS system.and it's equivalent

finite state ma.chine model is--d:i-s-c~ ...----fflChapter 3, -a ~1'1rrecei ver

proposed by Ungerboeck, [4) is presented and compared with Forney's

receiver [5].

Amodified stack algorithm i~ived in Chapter 4 for com-'. .

putation of the minimum distance.' Efficiency of the algorithm is

d~scussed and compared with the double dynamic programming method.

An optimization technique and the corresponding computer package are

introduced in Chapter 5. The computer package is used to solve the

problem of optimizing d2f . in Chapter 6 and optimizing bandwidth in- ree

Chapter 7 and 8.

Combining the\"algoritQm and technique proposed in Chapter 4

and 5, we have found the worst degradation of the MLSE for channel

lengtbs u~ to 12 in Chapter 6, while in Chapter 7, we optimize the- ----

bandwidth with and without the power constraint for channel lengths

up to 10. Finally, a spectral raised cosine function is introduced

for pulse shaping rather than the rectangu)ar spectral function used

-------iQJu.:evious chapters.. The same optimizatiqn problems in "Chapter

6 and· 7 are done for the modified system,;n Chapter 8.

)

­,

.. To' ..... l"-;.

•

5

.v

."

PRe; pilter

ITransmitting I

~Channel

~-Filter I FilterI

--I

-----. -- ------- --1.

II

Input Dal;;;:>o{Xd _,

0,1 I

L

---- --- --- - --I

+ ~Noise

r-1aXimun-likeliliOO'i. Receiver

VA

ML Decexlf>.r

IIIReceiving

Filter 1

I_____ -1

II

t-t--------..

r:~tirratedd

Sequence ~t--

{Xi} I /

~I -

Firy.~.l A diqitnl communication system.0"1

"I

("

)

CHAPTER 2

A BANDWIDTH EFFICIENT SIGNALING SCHEME:PARTIAL RESPONSE SIGNALING

In communication systems, most channels are bandlimited

in some sense, so that efficient use of bandwidth is one of the

objects of a transmission system. In this chapter, partial response

signaling is introduced as a bandwidth efficient signaling scheme.

2.1 Partia'l Response Signaling System (PRS)

Lender [14] first introduced PRS for data transmission. A

PRS system is based on the allowance of a controlled amount of inter-.

symbol interference (ISI)which is used for spectral shaping so as

to redistribute the spectral energy of the signal. Most of the

energy concentrates in low frequency region or some other frequency

region depending on the correlation introduced among the signals.

Since the lSI is known, its effect can be removed or diminished at

the receiver. In comparison, a conventional pulse amplitude mod-

ulation (PAM) system eliminates lSI by creating a large number of

signal levels. A PRS system can achieve high data rate by signaling

at a higher rate with fewer levels. Therefore, PRS system will have

better error rate performance than the conventional PAM system [2J.

A PRS system can be considered as a cascade of a digitalI

7

8

transversal filter F(w) and a pulse shaping filter G(w), as shown in

Fig. 2.1 [1],

The digital transversal filter F(w) is equivalent to a shist

register with Ktaps. It correlates each input datum with L = K- 1

succeeding input data. By controlling the tap coefficients, we can

achieve the desired spectrum at the output of F(w). In our study,

the tap coefficients can be any real numbers.

The transfer function of F(w) is

I:> LF(w) = [

i=O

-jwTf. e, (2.1)

where L is number of del~y units.

f. is a tap coefficient1

T is one tap delay time which is equal to one symbol period.

Its impulse response isL ..

F( D) r f. Oi (2.2)= i=O 1

,where 0 is the Huffman's delay pperator [5J.

F(w) is a periodic function with period liT. The analog

filter G(w) ~onverts the samples YK to an analog waveform and in so~

doing bandlimits the resulting system function, hopefully without too'

mu~h change to the sample values introduced by F(O). The choice of

G(w)' will be discussed in more detail in next section.

The analogy between PRS and convolutional coding was pointed '