[ieee 2012 international symposium on intelligent signal processing and communications systems...

6
Optimization of the Hybrid Asymmetric-FIR/Analog Square-Root Filters Chia-Yu Yao 1 , Yung-Hsiang Ho 2 , and Wei-Chun Hsia 3 Department of Electrical Engineering National Taiwan University of Science and Technology Taipei, Taiwan, ROC 1 [email protected], 2 [email protected], 3 [email protected] Abstract—This paper presents a method for designing asymmetric square-root (SR) FIR filters against the effects of jitter-induced in- terference at the receiver. The proposed method employs the linear programming to design the Nyquist filter coefficients for minimizing the power of the jitter-induced interference. The desired SR filter coefficients are obtained via the spectral factorization of the Nyquist filter polynomial. The proposed method takes the filter’s stopband attenuation, the system’s tolerable ISI, and the “roughness” of the analog/FIR-SR filter impulse response into account. Unlike previous publications, we show that the error-performance of a system employing a matched SR filter pair, in the presence of receiver timing jitter, is more strongly related to the roughness of the hybrid analog/FIR-SR filter’s impulse response, than it is to the eye width caused by the Nyquist pulse. The design example validates the proposition that the proposed method can lead to improved error performance of a high-order QAM communication system in the presence of receiver clock jitter. Keywords—square-root filter; Nyquist filter; inter-symbol interference; receiver clock jitter; QAM I. I NTRODUCTION A matched pair of square-root (SR) filters of a Nyquist filter, used in the transmitter and the receiver of a band-limited digital com- munication system, can provide zero inter-symbol interference (ISI). In practice, the SR filters are realized in FIR form. Conventionally, the SR filters are designed by directly designing the Nyquist filter with a nonnegative frequency response (i.e., ignoring the linear-phase factor), then getting the matched SR transmitter and receiver filters by performing a spectral factorization of the Nyquist filter polynomial [1]–[5]. Although zero ISI is theoretically desired, in practice, it is not always necessary. In [2]–[5], recognition of the "tolerable ISI" was proposed. The tolerable ISI should depend on the operating point (the error rate vs. signal-to-noise ratio, SNR) of a band-limited digital communication system. As long as the ISI is smaller than the tolerable ISI, the error performance of a communication system will not degrade significantly at the operating point. A small but non- zero ISI provides an increased degree of freedom in designing the SR filters. Since there are interference and noise in the channel, the recovered receiver clock signal must suffer from some timing jitter [6] in practical implementations of band-limited communication systems. This receiver timing jitter will deteriorate the error performance. Consider two cases shown in Figure 1. In Figure 1(a), the designing of a wide “eye width” Nyquist pulse can alleviate the jitter effect when the transmitted signal is shaped by the designed wide eye-width Nyquist filter h[n]. On the contrary, Figure 1(b) shows the case of having an SR filter h t [n] at the transmitter and having its matched SR filter h r [n] at the receiver. Since the samplers are placed at different This work was supported by the National Science Council of Taiwan, ROC, under Grant NSC 100-2221-E-011-095. Figure 1. (a) Nyquist filter at the transmitter. (b) A pair of matched SR filters in the system with the oversampling ratio T . locations, the ways how timing jitter affect the received signals in two cases of Figure 1 should be different even if h[n]= h t [n] h r [n]. In the Figure 1(a) system, the ‘eye height’ is controlled by the tolerable ISI, while in the Figure 1(b) system, the eye height is controlled not only by the tolerable ISI, but is also affected by the jitter of the recovered clock. However, [2], [5], [7], [8] assume that the design of the wide eye-width Nyquist pulse is applicable to the Figure 1(b) case. They assume that there is no jitter in the sampler of the Figure 1(b) system and the sampled signal is perfectly digitized without any jitter- induced interference. On the contrary, they assume that the jitter is introduced in the down sampler such that the design results of the wide eye-width Nyquist pulse can be directly applied. Although the eye diagrams shown in [2], [5], [7], [8] look good, but they are unrealistic. In the Figure 1(b) system, the clock recovery subsystem produces the required clock for the sampler. No matter how the clock recovery subsystem is realized, the clock definitely suffers from timing jitter [9]. Thus, the sampler is the source of the jitter-induced interference. On the other hand, the down sampler just outputs a digital sample for every T samples it receives. Therefore, no jitter- induced interference will be introduced in a digital down sampler. To investigate the effects of the jitter-induced interference, symbol- error-rate (SER) simulations are essential. The simulation result of [5] shows that only slight improvement is made for designing SR filters against receiver timing jitter using the idea of the wide eye- width Nyquist pulse. The even worse thing of [5] is that the length of the SR filter designed based on the wide eye-width Nyquist pulse criterion is much longer than the conventional SR filter. This result 2012 IEEE International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS 2012) November 4-7, 2012 395 978-1-4673-5082-2 ©2012 IEEE

Upload: wei-chun

Post on 13-Mar-2017

217 views

Category:

Documents


4 download

TRANSCRIPT

Optimization of the Hybrid Asymmetric-FIR/AnalogSquare-Root Filters

Chia-Yu Yao1, Yung-Hsiang Ho2, and Wei-Chun Hsia3

Department of Electrical EngineeringNational Taiwan University of Science and Technology

Taipei, Taiwan, [email protected], [email protected], [email protected]

Abstract—This paper presents a method for designing asymmetricsquare-root (SR) FIR filters against the effects of jitter-induced in-terference at the receiver. The proposed method employs the linearprogramming to design the Nyquist filter coefficients for minimizing thepower of the jitter-induced interference. The desired SR filter coefficientsare obtained via the spectral factorization of the Nyquist filter polynomial.The proposed method takes the filter’s stopband attenuation, the system’stolerable ISI, and the “roughness” of the analog/FIR-SR filter impulseresponse into account. Unlike previous publications, we show that theerror-performance of a system employing a matched SR filter pair, inthe presence of receiver timing jitter, is more strongly related to theroughness of the hybrid analog/FIR-SR filter’s impulse response, thanit is to the eye width caused by the Nyquist pulse. The design examplevalidates the proposition that the proposed method can lead to improvederror performance of a high-order QAM communication system in thepresence of receiver clock jitter.Keywords—square-root filter; Nyquist filter; inter-symbol interference;

receiver clock jitter; QAM

I. INTRODUCTION

A matched pair of square-root (SR) filters of a Nyquist filter, usedin the transmitter and the receiver of a band-limited digital com-munication system, can provide zero inter-symbol interference (ISI).In practice, the SR filters are realized in FIR form. Conventionally,the SR filters are designed by directly designing the Nyquist filterwith a nonnegative frequency response (i.e., ignoring the linear-phasefactor), then getting the matched SR transmitter and receiver filters byperforming a spectral factorization of the Nyquist filter polynomial[1]–[5].

Although zero ISI is theoretically desired, in practice, it is notalways necessary. In [2]–[5], recognition of the "tolerable ISI" wasproposed. The tolerable ISI should depend on the operating point(the error rate vs. signal-to-noise ratio, SNR) of a band-limiteddigital communication system. As long as the ISI is smaller thanthe tolerable ISI, the error performance of a communication systemwill not degrade significantly at the operating point. A small but non-zero ISI provides an increased degree of freedom in designing theSR filters.

Since there are interference and noise in the channel, the recoveredreceiver clock signal must suffer from some timing jitter [6] inpractical implementations of band-limited communication systems.This receiver timing jitter will deteriorate the error performance.Consider two cases shown in Figure 1. In Figure 1(a), the designingof a wide “eye width” Nyquist pulse can alleviate the jitter effectwhen the transmitted signal is shaped by the designed wide eye-widthNyquist filter h[n]. On the contrary, Figure 1(b) shows the case ofhaving an SR filter ht [n] at the transmitter and having its matched SRfilter hr[n] at the receiver. Since the samplers are placed at different

This work was supported by the National Science Council of Taiwan, ROC,under Grant NSC 100-2221-E-011-095.

Figure 1. (a) Nyquist filter at the transmitter. (b) A pair of matched SRfilters in the system with the oversampling ratio T .

locations, the ways how timing jitter affect the received signals in twocases of Figure 1 should be different even if h[n] = ht [n]∗hr[n]. In theFigure 1(a) system, the ‘eye height’ is controlled by the tolerable ISI,while in the Figure 1(b) system, the eye height is controlled not onlyby the tolerable ISI, but is also affected by the jitter of the recoveredclock.

However, [2], [5], [7], [8] assume that the design of the wideeye-width Nyquist pulse is applicable to the Figure 1(b) case. Theyassume that there is no jitter in the sampler of the Figure 1(b)system and the sampled signal is perfectly digitized without any jitter-induced interference. On the contrary, they assume that the jitter isintroduced in the down sampler such that the design results of thewide eye-width Nyquist pulse can be directly applied. Although theeye diagrams shown in [2], [5], [7], [8] look good, but they areunrealistic. In the Figure 1(b) system, the clock recovery subsystemproduces the required clock for the sampler. No matter how theclock recovery subsystem is realized, the clock definitely suffers fromtiming jitter [9]. Thus, the sampler is the source of the jitter-inducedinterference. On the other hand, the down sampler just outputs adigital sample for every T samples it receives. Therefore, no jitter-induced interference will be introduced in a digital down sampler.

To investigate the effects of the jitter-induced interference, symbol-error-rate (SER) simulations are essential. The simulation result of[5] shows that only slight improvement is made for designing SRfilters against receiver timing jitter using the idea of the wide eye-width Nyquist pulse. The even worse thing of [5] is that the lengthof the SR filter designed based on the wide eye-width Nyquist pulsecriterion is much longer than the conventional SR filter. This result

2012 IEEE International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS 2012) November 4-7, 2012

395978-1-4673-5082-2 ©2012 IEEE

Figure 2. Block diagram of a band-limited digital communication system.

Figure 3. A baseband channel model.

motivates us reexamine the design problem of the SR filters. Inthis paper, we derive the model for the jitter-induced interferenceintroduced by the sampler. By assuming that the jitter is small (thatis the case in practice), we employ a newly conceived measurementcalled the ‘roughness’ [10] that can indicate how bad the jitter-induced interference affect the error performance of the system.Different from [10], in which a symmetric SR filter is designed bylocally minimizing a non-convex problem, in this paper we proposea design method for the optimal asymmetric SR pulse-shaping filtersby globally minimizing the power of the jitter-induced interference.

On the other hand, the effect of analog front-ends should also beconsidered in designing the SR filter pairs [3]–[5], [11], [12]. Figure 2shows the block diagram of a band-limited digital communicationsystem. In Figure 2, Sm(s) and An(s) represent the transfer functionsof the analog smoothing filter and the analog anti-aliasing filter,respectively. Having the analog parts in the system, we are facing ahybrid FIR-filter/analog-filter pulse-shaping problem. The designingof ht [n] and hr [n] must consider compensating for the effects of theanalog parts.

This paper is organized in the following manner: In Section II, wederive a jitter model of the analog-to-digital converter (ADC) and amethod of compensating for the nonlinear phase response of analogparts. In Section III, the basic time-domain and frequency-domainresponses are formulated. In Section IV, the formulation of ISI andthe design strategy for SR filters in the presence of receiver timingjitter are introduced. In Section V, the proposed design procedureis presented. A Numerical design example is given in Section VI.Finally, a summary of conclusions is given in Section VII.

II. MODEL OF THE JITTER-INDUCED INTERFERENCE

Assume that the channel in Figure 2 is a wide-band linear-phase channel such that the channel model shown in Figure 3can be applied. Let a[m] denote the m-th transmitted data sample.The digital-to-analog converter (DAC) is modeled as a zero-order-hold block where T0 represents the sampling period. The ADCis simply modeled as a sampler that takes samples at multipletime-instants nT0 + ε + τn where ε denotes the deterministic timingoffset between the transmitter and the receiver, and τn representsthe random timing jitter of the ADC at the receiver. We assume

that 0 ≤ ε < T0. Let hsm(t) and han(t) be the impulse responses ofSm(s) and An(s), respectively, and let h(t) = hsm(t)∗han(t). Denotingr[n] = r(nT0 + ε+ τn), we can obtain

r[n] = ∑ma[m]

∫ (n−m)T0+ε+τn

[n−(m+1)]T0+ε+τnh(τ)dτ

= ∑ma[m]

{∫ (n−m)T0+ε

[n−(m+1)]T0+εh(τ)dτ+

∫ (n−m)T0+ε+τn

(n−m)T0+εh(τ)dτ−

∫ [n−(m+1)]T0+ε+τn

[n−(m+1)]T0+εh(τ)dτ

}. (1)

Next, denote

hεΠ[n] =

∫ nT0+ε

(n−1)T0+εh(τ)dτ, n= 0,1,2, . . . . (2)

Thus, (1) can be written as

r[n] = a[n]∗hεΠ[n]+∑

ma[m]

{∫ (n−m)T0+ε+τn

(n−m)T0+εh(τ)dτ

−∫ [n−(m+1)]T0+ε+τn

[n−(m+1)]T0+εh(τ)dτ

}. (3)

The first term of (3) is the jitter-free term. The second term corre-sponds to the additive, data-dependent, jitter-induced interference. Itis noted that in the above derivation, no approximation is made. Thejitter-induced interference is indeed additive.

In practice, the jitter is sufficiently small such that (3) can beapproximated as

r[n] ≈ a[n]∗hεΠ[n]+τn∑

ma[m]{h((n−m)T0 + ε)−

h((n−m−1)T0 + ε)}= a[n]∗hε

Π[n]+τna[n]∗{hεδ [n]−hε

δ[n−1]} (4)

wherehε

δ[n] = h(nT0 + ε). (5)

Hence, the channel output in Figure 3 can be modeled as a discrete-time convolution of the transmitted data samples a[n] and the chan-nel’s equivalent discrete-time impulse response hε

Π[n], plus the jitterrandom sequence τn times the convolution of the transmitted datasamples a[n] and the sequence [hε

δ[n]−hεδ[n−1]]. Notably, since h(t)

is essentially time limited in practice [11], so are hεΠ[n] and hε

δ[n].Therefore, the channel can be modeled as an FIR channel in practice.Now, we put the jitter-induced interference aside for a while. We willdiscuss it later in Section IV.

It is noted that the values of hεΠ[n] depend on ε. Therefore, a

different timing offset ε leads to different hεΠ[n]. In order to obtain

an optimum sample timing, a clock recovery subsystem is required,which is beyond the scope of this paper. When the clock recoverysubsystem is ideal, we will have ε = 0. Because h0

Π[0] = 0, we usehT0

Π [n] instead of h0Π[n] later. It is noted that hT0

Π [n] = h0Π[n+1].

In general, since the sampling rate 1/T0 and the analog filters Sm(s)and An(s) are known a priori, we can use them to construct an M-tapFIR channel equivalence hT0

Π [n], n= 0,1, . . . ,M−1, by truncating thefirst M samples of hT0

Π [n]. Next, let

hins[n] = hT0Π [M−1−n]. (6)

Thus, hT0Π [n]∗hins[n] will possess a linear phase response. This implies

that the nonlinear phase distortion introduced by Sm(s) and An(s)can be approximately compensated for by inserting hins[n] in thetransmitter or receiver.

2012 IEEE International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS 2012) November 4-7, 2012

396

III. THE FORMULATIONS OF TIME-DOMAIN ANDFREQUENCY-DOMAIN RESPONSES

Assuming a clock recovery subsystem is ideal, we denote

hD[n] = hT0Π [n]∗hins [n].

The length of hD[n] is 2M−1. From (6), hD(n) is symmetric.Let N denote the length of the SR filter. Since we employ the

matched SR filter pair, we have hr [n] = ht [N−1−n], n= 0,1, . . . ,N−1, and h[n] = ht [n]∗ht [N−1−n]. The length of h[n] is 2N−1.

The complete impulse response of the system hc[n] is equal toh[n]∗hD[n]. Since both h[n] and hD[n] are symmetric, so is hc[n]. Thelength of hc[n] equals 2N+2M−3. For the sake of convenience, welet hc[n] = hc[−n], |n| ≤ N+M−2. We next express hc[n] in matrixform. Let

h = ( h[0] h[1] · · · h[N−1] )T , (7)hD = ( hD[0] hD[1] · · · hD[2M−2] )T , (8)

and let Sn, |n| ≤N+M−2, be an (2M−1)× (2N−1) matrix whosekth-row, lth-column element is

(Sn)k,l ={

1, k+ l = n+N+M0, otherwise. (9)

Denote as Il , Jl , and 0l the l× l identity matrix, the l× l anti-diagonalmatrix, and the l×1 zero vector, respectively. Let

E=

(IN

JN−1 0N−1

)(10)

and Sn = SnE. Then hc[n] can be written as

hc[n] = hDT Snh. (11)

for |n| ≤ N+M−2.It is understood [1] that hc[0] = 1/T for Nyquist filters. Therefore,

hDT S0h=1T

. (12)

Next, we will examine the frequency domain constraints. The zero-phase frequency response of hc[n] at ω = π/T should be equal to 1/2[2]; i.e.,

N+M−2

∑n=1

2hDT SnhcosnπT

+1T

=12. (13)

The zero-phase frequency response of the symmetric h[n] canbe expressed in the following manner. Denote A(ω) = (2cos(N−1)ω 2cos(N − 2)ω · · · 2cosω 1). Then the zero-phase frequencyresponse of h[n] can be expressed as

H(ω) =A(ω)h. (14)

Notably, H(ω) ≥ 0 because h[n] = ht [n] ∗ ht [N− 1− n]. Finally, thestopband edge frequency ωs of H(ω) is specified by the over-sampling factor T and the roll-off factor ρ as

ωs =(1+ρ)πT

.

IV. THE FORMULATION OF ISI AND ROBUSTNESS AGAINSTTIMING JITTER

In this section, we express the ISI and the robustness against thereceiver timing jitter in terms of h. The formula for calculating thepeak ISI [13] is

ISI =

� N+M−2T �

∑j=1

∣∣∣hc[± jT ]∣∣∣

|hc[0]|. (15)

Figure 4. A model of interference introduced by the receiver timing jitter.

Let γISI denote the tolerable ISI. Since hc[n] is symmetric, substitut-ing hc[0] = 1/T and (11) into (15) and employing the non-negativeslack variables α j, for j = 1,2, . . . ,�(N +M− 2)/T�, to relax theabsolute values in (15) [2], we obtain

±hDT S jTh≤ α j (16)

and� N+M−2

T �

∑j=1

α j ≤γISI2T

. (17)

Next, let us consider designing hybrid analog/FIR-SR filters tocope with the receiver timing jitter. Previous papers [2], [5], [7], [8]take the widening of the eye width caused by the Nyquist pulse asa goal for desensitizing against errors that result from the receiverclock jitter. This is valid if the Figure 1(a) system is used. We find thatalthough the design methods of these publications differ, a commonproperty they share is to ‘flatten’ the waveform near the samplinginstants at the symbol rate. However, in the case of a pair of matchedSR filters, one at the transmitter and the other at the receiver as shownin Figure 1(b), the wide eye width caused by the Nyquist pulse doesnot guarantee that the waveform near the sampling instants (now thesampling rate is faster than the symbol rate) is flat. This is explainedas follows:

We assume that the phase equalizer hins[n] is inserted at thetransmitter. Hence, the model for analyzing the timing jitter is givenin Figure 4 where a[k] is the data symbol, r[n] is the sampled receivedsignal and w[n] represents the interference introduced by the receivertiming jitter. Comparing Figure 4 with (4), we obtain

a[n] = ∑ka[k]δ[n−kT ]∗ (ht [n]∗hins[n]).

Let h j[n] = hins[n]∗h0δ[n] and

v[n] = ht [n]∗h j[n]. (18)

When the jitter is small, from (4), we can write

w[n]≈ τn∑ka[k](v[n−kT ]−v[n−kT −1]). (19)

If τn is zero-mean, i.i.d., and uncorrelated with the data symbols a[k]s,and if a[k] is i.i.d., [10] shows that the power of w[n] is proportionalto the roughness cost function

R = T 20 ∑

∣∣v[n]−v[n−1]∣∣2

. (20)

We can minimize R to find an SR filter. The derivation of (20) isreviewed in the Appendix at the end of this paper.

One can observe that R is not a convex function of ht [n]. In [10],the design of symmetric SR filters is discussed. Hence, [10] hasto directly find ht [n] by looking for a local minimum of the non-convex minimization problem. Although [10] employs a sequentialquadratic programming (SQP) method, the solution SR filter ht [n]is not guaranteed the optimal one. In this paper, we design theasymmetric SR filters. Conventionally, we can design the Nyquistfilter first. Therefore, we derive the frequency-domain equivalent of R,which happens to be a linear function of the Nyquist filter coefficient

2012 IEEE International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS 2012) November 4-7, 2012

397

vector h. Let V (ω), Ht(ω), and Hj(ω) be the Fourier transforms ofv[n], ht [n], and h j[n], respectively. From Parseval’s theorem, (20) isequivalent to

R =T 2

02π

∫ π

−π

∣∣V (ω)∣∣2∣∣1−e− jω∣∣2 dω

=T 2

∫ π

−π

∣∣Ht(ω)∣∣2∣∣Hj(ω)

∣∣2(1−cos ω)dω.

From the fact that |Ht(ω)|2 = H(ω) and (14) , the roughness costfunction (20) can be written as

R =T 2

∫ π

−π

∣∣Hj(ω)∣∣2

(1−cos ω)A(ω)dωh (21)

V. THE DESIGN PROBLEM FORMULATION

Having expressed hc[0], the frequency responses, ISI, and theroughness cost function R in terms of h in (12)–(21), we can nowformulate our design problem. Let η denote the maximum stopbandgain of h[n]. The design problem becomes a linear semi-infiniteprogramming (SIP) problem :

minimizeT 2

∫ π

−π

∣∣Hj(ω)∣∣2

(1−cos ω)A(ω)dωh

subject to

A(ω)h ≤ η, ω ∈ [ωs,π]

−A(ω)h ≤ 0, ω ∈ [0,π]N+M−2

∑n=1

2hDT SnhcosnπT

=12−

1T

hDT S0h =1T

� N+M−2T �

∑j=1

α j ≤γISI2T

±hDT S jTh−α j ≤ 0,

j = 1,2, . . . ,⌊N+M−2

T

⌋. (22)

The unknowns for the above optimization problem are h andα j, j = 1,2, . . . ,�(N+M−2)/T �. In (22), there are infinite numberof constraints because ω is a continuous variable. Fortunately, sincethe zero-phase frequency response A(ω)h is sufficiently smoothin practice, we can sample A(ω)h to obtain a finite number ofconstraints. Therefore, (22) can be solved by any linear programmingpackage [14].

A. Filter Length Estimation

It is well known that there exists a simple formula for estimatingthe length of an optimal symmetric FIR filter. Taking the formulaattributed to Kaiser [15] as an example, we modify it to roughlyestimate the length N1 of a Nyquist filter as

N1 =−20log10 η−13

14.6Δ f+1

where η corresponds to the stopband gain and the transition band-width Δ f is equal to ρ/T . It is noted that we can employ Herrmann’sformula [15] as well. Since one coefficient is strictly constrained(close to zero) for every T Nyquist filter coefficients, and since theKaiser formula was not intended for Nyquist filters, we have foundthe following heuristic useful in getting the approximate length N2

TABLE ICOEFFICIENTS OF THE FIR FILTER hins[n] USED TO EQUALIZE THE PHASE

OF ANALOG PARTS IN THE DESIGN EXAMPLE

hins[0] = 1.6204×10−5 hins[9] =−4.2107×10−2

hins[1] = 4.7711×10−4 hins[10] =−7.6682×10−2

hins[2] = 6.4949×10−4 hins[11] = 1.0406×10−2

hins[3] =−6.3219×10−4 hins[12] = 2.4164×10−1

hins[4] =−3.0688×10−3 hins[13] = 4.1988×10−1

hins[5] =−2.4456×10−3 hins[14] = 3.2570×10−1

hins[6] = 6.2687×10−3 hins[15] = 9.3758×10−2

hins[7] = 1.6947×10−2 hins[16] = 3.6915×10−3

hins[8] = 5.6652×10−3

of a Nyquist filter N2 = (TN1)/(T −1).Therefore, the filter length ofthe SR filter can be estimated as

N =

⌈N2 +1

2

⌉. (23)

The above equation provides the initial guess of the SR filter lengthin the following design procedure.

B. Design ProcedureHaving the design formulations (22), we develop a design proce-

dure as follows: Given the transfer functions of two analog filtersSm(s), An(s), the initial filter length N of the SR filter, the roll-offfactor ρ, the over-sampling ratio T , the tolerable ISI γISI, and thespecification of the stopband gain constraint η of the Nyquist filterh[n].

1) Find the discrete-time equivalent hT0Π [m] of the analog parts

using (2) and find the samples of h0δ[n] = h(nT0) using (5).

2) Evaluate the M-tap phase-equalizing FIR filter hins[m].3) Calculate hD[m] = hins[m]∗ hT0

Π [m] and h j[n] = hins[n]∗h0δ [n].

4) Use formulation (22) to obtain the solution of h.5) If no feasible solution is found, N = N+1. Go to Step 4.6) Otherwise, perform a spectral factorization of h[n],n =

0,1, . . . ,2N− 2, to obtain the SR filter coefficients ht [n] andhr[n] = ht [N−1−n]. The spectral factorization method for theNyquist filter can be found in [14].

VI. DESIGN EXAMPLE

In this example, we demonstrate that designing the SR filter byconsidering the roughness of the waveform before the sampler ismore effective than by considering the wide eye width of the Nyquistpulse in improving the error performance in the presence of receiverclock jitter. The scenario of this example is listed as follows:• The smoothing filter at the transmitter and the anti-aliasing filter

at the receiver are 3rd order lowpass Butterworth filters with the3 dB frequency at 15 MHz. Hence,

Sm(s) = An(s)= 8.3717×1023

s3+1.8550×108s2+1.7765×1016s+8.3717×1023 . (24)

• The baud rate is set to 17.5 MHz and the sampling rate is setto 70 MHz. Thus, T0 = 1/(70×106) sec. and T = 4.

• ρ = 0.5, η =−41 dB, and γISI =−50 dB.First, we must construct a phase equalizer hins[n] to compensate

for the nonlinear phase response of Sm(s)An(s) using the techniqueshown in Section II. The hins[n] we found is shown in Table I, whichis a 17-tap FIR filter.

In this example, we design three FIR SR filters. The first one ht1[n],with stopband attenuation SBATT = 20.520 dB, is designed without

2012 IEEE International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS 2012) November 4-7, 2012

398

TABLE IISR FILTER COEFFICIENTS OF ht1[n], ht2[n], AND ht3[n] IN THE DESIGN

EXAMPLE

ht1[n]: Length = 11. hr1[n] = ht1[10−n]ht1[0] = 1.2615854×10−1 ht1[6] =−3.8018917×10−2

ht1[1] = 2.1089339×10−1 ht1[7] =−7.5297252×10−2

ht1[2] = 2.7974434×10−1 ht1[8] =−5.5520751×10−2

ht1[3] = 2.7255750×10−1 ht1[9] =−1.7277797×10−2

ht1[4] = 1.8559954×10−1 ht1[10] = 4.8872913×10−2

ht1[5] = 6.1769877×10−2

ht2[n]: Length = 14. hr2[n] = ht2[13−n]ht2[0] = 1.3077298×10−1 ht2[7] =−7.5417261×10−2

ht2[1] = 2.1409253×10−1 ht2[8] =−5.2432080×10−2

ht2[2] = 2.8117431×10−1 ht2[9] =−7.6346167×10−3

ht2[3] = 2.7087171×10−1 ht2[10] = 2.5964385×10−2

ht2[4] = 1.8164950×10−1 ht2[11] = 3.0864441×10−2

ht2[5] = 5.7370621×10−2 ht2[12] =−2.4849077×10−2

ht2[6] =−4.0876846×10−2 ht2[13] = 8.1849157×10−3

ht3[n]: Length = 14. hr3[n] = ht3[13−n]ht3[0] =−2.9900035×10−2 ht3[7] = 1.7975419×10−1

ht3[1] = 5.9769196×10−3 ht3[8] = 6.5675556×10−2

ht3[2] = 6.3853223×10−3 ht3[9] =−3.0678914×10−2

ht3[3] = 1.0576495×10−1 ht3[10] =−6.8703280×10−2

ht3[4] = 2.2807666×10−1 ht3[11] =−4.9626414×10−2

ht3[5] = 2.8699990×10−1 ht3[12] =−1.2475358×10−2

ht3[6] = 2.6428151×10−1 ht3[13] = 4.9283254×10−2

TABLE IIIPERFORMANCE INDICES OF SYSTEMS EMPLOYING hti[n] AND hri[n],i = 1,2,3, WITH THE PHASE EQUALIZER AND ANALOG FRONT-END IN

THE DESIGN EXAMPLE

System employing SBATT ISI EYE Rht1[n], hr1[n] 41.040 dB −50.100 dB 4.938 dB 0.0524ht2[n], hr2[n] 41.108 dB −50.100 dB 5.010 dB 0.0524ht3[n], hr3[n] 41.050 dB −50.109 dB 4.570 dB 0.0521

considering the effect of receiver timing jitter. The SR filter ht1[n] isa 11-tap FIR filter whose coefficients are given in Table II.

The second SR filter is designed by the method of [5] that employsthe conventional wide eye-width design methodology. We use themeasurement defined in [2] to examine the eye width caused by thehybrid analog/FIR-SR filters. The measurement is formulated as

EYE =hc[1]

� N+M−1T �

∑j=1

∣∣∣hc[ jT −1]∣∣∣+

N+M−2T −1

∑j=1

∣∣∣hc[ jT +1]∣∣∣.

A larger EYE value corresponds to a wider eye width. In designingthe second SR filter, we let EYE = 5 dB and obtain a 14-tap ht2[n]that can achieve 20.554-dB SBATT. The coefficients of ht2[n] areshown in Table II too.

The third SR filter ht3[n] is designed by the proposed method. Welet the filter length be the same as that of ht2[n]. The SBATT ofht3[n] is 20.525 dB and the roughness R defined in (21) we obtainis 0.0521. The coefficients of ht3[n] are also given in Table II.

All performance indices (SBATT, ISI, EYE, and R) of the systememploying hti[n] (i= 1,2,3), hins[n], DAC, Sm(s), An(s), sampler, andhri[n] = hti[N− 1− n], n = 0,1, . . . ,N− 1, are summarized in TableIII. We can see that all three systems satisfy the specifications of thisexample.

Three 256-QAM systems employing hti[n] and hri[n], i= 1,2,3, asSR filter pairs are simulated. The standard deviation of the receiver

28 29 30 31 32 33 34 35 36

10−6

10−5

10−4

10−3

10−2

Es/No (dB)

Sym

bol E

rror

Pro

babi

lity

(SER

)

Theoretical 256−QAM SERSimulated SER with ht1 and hr1Simulated SER with ht2 and hr2Simulated SER with ht3 and hr3

Figure 5. Symbol error performance comparison of three 256-QAM systemsemploying hti[n] and hri[n], i= 1,2,3. The standard deviation of the receivertiming jitter is equal to 7% of the sampling period.

timing jitter is set to 7% of the sampling period in simulations,which is equivalent to 1 ns. Their SER are compared. Figure 5 showsthe SER results. We can see that the system employing ht3[n] andhr3[n] (wherein its waveform roughness is minimized) outperformsthe system employing ht1[n] and hr1[n] (wherein the receiver timingjitter is ignored) and the system employing ht2[n] and hr2[n] (in whichits eye width is maximized) in the presence of receiver timing jitter.

A. Discussion• The SER performance of the system employing ht1[n] and hr1[n]

and the SER performance of the system employing ht2[n] andhr2[n] are similar since their roughness cost function values areclose to each other.

• In Figure 5, when Es/No is small, the SER performance isdominated by noise. Since the ISI specification is −50 dB in thisexample, when Es/No is far less than 50 dB, the ISI effect on theSER performance is not significant. From (29) in the Appendix,the power ratio of the signal to jitter-induced interference (SJIR)is

SJIR =1

(σ2τ/T 2

0 )R. (25)

In this example, στ/T0 = 0.07 and R≈ 0.052. Thus, SJIR≈ 36dB. This value can explain that, in Figure 5, the effect of jitter-induced interference becomes significant when Es/No is greaterthan 30 dB.

VII. CONCLUSIONS

We have proposed a design method for hybrid analog/asymmetricFIR-SR filters for band-limited communications. Instead of usinganalog equalizers to compensate for the analog front-end in thesystem, we employ a systematic technique to compensate for theeffects of the analog parts. We construct an FIR channel model for theDAC, the analog filters, and the ADC. Then, a time reversal of the FIRchannel model is cascaded in the system to make the overall phaseresponse approximately linear. Although this will introduce an extramagnitude distortion to the signal, this extra magnitude distortion iseasily taken care of by our systematic formulation for the SR filters.

2012 IEEE International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS 2012) November 4-7, 2012

399

Unlike prior methods, the proposed method does not target the eyewidth widening as the goal for tolerating receiver timing jitter. Rather,our method considers the time-domain ‘roughness’ of the waveformbefore the sampler as the optimization goal to reduce the effect of thereceiver timing jitter. Since the time-domain roughness cost functioncan not be formulated as a linear programming problem, we derivethe frequency-domain representation of the roughness cost function,which is a linear function of the Nyquist filter coefficients and ,thus,can be solved by any linear programming package. After we havethe Nyquist filter coefficients, we can obtain the required SR filterby applying a spectral factorization to the Nyquist filter coefficients.The design example shows that our approach is more effective thantaking the wider eye width as the goal for the Figure 2 system in thepresence of receiver timing jitter.

APPENDIX

The roughness cost function R was derived in [10] and is reviewedhere. In Figure 4,

g[n] = ∑ka[k]hc[n−kT ]+∑

mw[m]hr[n−m]. (26)

Since hc[0] = 1/T [1], at the sampling instants n= k′T , g[n] becomes

g[k′T ] =a[k′]T

+ ∑kk �=k′

a[k]hc[(k′ −k)T ]

+∑mw[m]hr[k′T −m]. (27)

The second term is the conventional ISI term that is dealt with in (15)to (17). The third term is the interference caused by the sampler’stiming jitter.

Next, consider the power of the third term of (27). Assume thatτn is zero-mean, i.i.d., and is uncorrelated with the data symbol. Letthe power spectral density of w[n] be Pww. Then

E{∣∣∣∑

mw[m]hr[k′T −m]

∣∣∣2}

= Pww∑m

∣∣∣hr[k′T −m]∣∣∣2

. (28)

If hr[n] was a square root Nyquist filter, the right-hand side of (28)would be Pww/T . In this paper, since we take the analog front-endsinto account, hr[n] is an approximation of a square root Nyquist filter.Hence,

E{∣∣∣∑

mw[m]hr[k′T −m]

∣∣∣2}≈PwwT

.

Nevertheless, to alleviate the interference caused by the receivertiming jitter, we need to reduce Pww. From (19) and the assumptionthat τn is zero-mean, i.i.d., and is uncorrelated with the data symbol,

Pww = σ2τE

{∣∣∣∣∑ka[k](v[n−kT ]−v[n−1−kT ])

∣∣∣∣2}

where σ2τ represents the variance of τn. Assume that the data symbol

a[n] is i.i.d. with an average power A. Then,

Pww = Aσ2τ ∑k

∣∣v[n−kT ]−v[n−1−kT ]∣∣2

which is equivalent to

Pww = Aσ2

τT 2

0

(T 2

0 ∑∣∣v[n]−v[n−1]

∣∣2)

. (29)

In (29), we normalize the standard deviation of τn with respect to thesampling period T0 of the ADC. Therefore, to obtain a small Pww,we should minimize the roughness cost function

R = T 20 ∑

∣∣v[n]−v[n−1]∣∣2

.

Notably, T 20 in the expression of R is due to the normalization of

the standard deviation of τn with respect to T0 in (29). WithoutT 2

0 , according to the practical situations, the value of R will beunreasonably large. should not infer that increasing the sampling ratecan reduce the

REFERENCES

[1] H. Samueli, “On the design of optimal equiripple FIR digital filtersfor data transmission applications,” IEEE Trans. Circuits. Syst., vol. 35,pp. 1542–1546, Dec. 1988.

[2] C.-Y. Yao and A. N. Willson, Jr., “The design of asymmetrical square-root pulse-shaping filters with wide eye-openings,” in Proc. 2008 IEEEInt. Symp. Circuits Syst., May 2008, pp. 2665–2668.

[3] J. O. Coleman and D. W. Lytle, “Linear-programming techniques forthe control of intersymbol interference with hybrid FIR/analog pulseshaping,” in Proc. IEEE Int. Conf. Commun., Chicago, IL, June 1992,pp. 793–798.

[4] J. O. Coleman, “Linear-programming design of data-communicationpulses tolerant of timing jitter or multipath,” in Proc. Fifth Int. Conf.Wireless Commun., Calgary, Canada, July 1993, pp. 341–352.

[5] C.-Y. Yao, “A design method of hybrid analog/asymmetrical-FIR pulse-shaping filters with an eye-opening control option against receiver timingjitter.” ETRI J., vol. 32, pp. 911–920, Dec. 2010.

[6] Lindsey WC, Simon MK. Telecommunation Systems. Engineering.Englewood Cliffs, NJ: Prentice Hall, 1973.

[7] B. Farhang-Boroujeny and G. Mathew, “Nyquist filters with robustperformance against timing jitter,” IEEE Trans. Signal Processing,vol. 46, pp. 3427–3431, Dec. 1998.

[8] P. Boonyanant and S. Tantaratana, “Design and hybrid realization of FIRNyquist filters with quantized coefficients and low sensitivity to timingjitter,” IEEE Trans. Signal Processing, vol. 53, pp. 208–221, Jan. 2005.

[9] Zamek I, Zamek S. “Crystal oscillators’ jitter measurements and itsestimation of phase noise,” in Proc. 2003 IEEE Int. Freq. Control Symp.PDA Exhib. Jointly 17th Eur. Freq. Time Forum, May 2003, pp. 547–555.

[10] C.-Y. Yao and A. N. Willson, Jr., “The design of hybrid symmetric-FIR/analog pulse-shaping filters,” IEEE Trans. Signal Processing, vol.60, pp. 2060–2065, Apr. 2012.

[11] T. N. Davidson, Z.-Q. Luo, and K. M. Wong, “Design of orthogonalpulse shapes for communications via semidefinite programming,” IEEETrans. Signal Processing, vol. 48, pp. 1433–1445, May 2000.

[12] T. N. Davidson, “Efficient design of waveforms for robust pulseamplitude modulation,” IEEE Trans. Signal Processing, vol. 49, pp.3098–3111, Dec. 2001.

[13] C.-L. Chen and A. N. Willson, Jr., “A trellis search algorithm for thedesign of FIR filters with signed-powers-of-two coefficients,” IEEETrans. Circuits Syst. II, vol. 46, pp. 29–39, Jan. 1999.

[14] Wu SP, Boyd S, Vandenberghe L. FIR filter design via spectralfactorization and convex optimization. in Applied and ComputationalControl, Signals, and Circuits. B. N. Datta ed., Boston, MA: Birkhauser,1999.

[15] J. G. Proakis and D. G. Manolakis Digital Signal Processing Principles,Algorithm, and Applications, 3rd ed. Upper Saddle River, NJ: PrenticeHall, 1996.

2012 IEEE International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS 2012) November 4-7, 2012

400