ecs 332: principles of communications 2012/1 hw 4 | due: sep 7 hw.pdf · ecs 332: principles of...

ECS 332: Principles of Communications 2012/1

HW 4 — Due: Sep 7

Lecturer: Prapun Suksompong, Ph.D.

Instructions

(a) ONE part of a question will be graded (5 pt). Of course, you do not know which partwill be selected; so you should work on all of them.

(b) It is important that you try to solve all problems. (5 pt)

(c) Late submission will be heavily penalized.

(d) Write down all the steps that you have done to obtain your answers. You may not getfull credit even when your answer is correct without showing how you get your answer.

Problem 1. The spectrum of a periodic square wave can be found using:

% specsquare.m plot the spectrum of a square waveclose allf0=10; % "(fundamental) frequency" of the square waveEndTime=2; % Will consider from time = 0 to EndTimeTs=1/1000; % sampling interval (time interval between samples)t=0:Ts:(EndTime−Ts); % create a time vectorx=sign(cos(2∗pi∗f0∗t)); % square wave = sign of cos waveplotspec(x,Ts) % call plotspec to draw spectrum

The output of specsquare.m is shown in Figure 4.1. The top plot shows the first 2seconds of a square wave with fundamental frequency f0 = 10 cycles per second. Thebottom plot shows a series of spikes that define the frequency content. In this case, thelargest spike occurs at ±10 Hz, followed by smaller spikes at all the odd-integer multiples(i.e., at ±30,±50,±70, etc.).

Modify specsquare.m to investigate the relationship between the time behavior of thesquare wave and its spectrum. Try square waves with different (fundamental) frequencies:f0 = 20, 40, 100, 300 Hz. (Keep Ts = 1/1000.) Describe the aliasing effect on each of cases.How do the time plots change? How do the spectra change?

4-1

ECS 332 HW 4 — Due: Sep 7 2012/1

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2−1

−0.5

0

0.5

1

Seconds

−500 −400 −300 −200 −100 0 100 200 300 400 5000

0.5

1

1.5

Frequency [Hz]

Mag

nitu

de

Figure 4.1: Plots from specsquare.m

Problem 2. Determine the Nyquist sampling rate and the Nyquist sampling interval forthe signals:

(a) sinc(100πt)

(b) sinc2(100πt)

(c) sinc(100πt) + sinc(50πt)

(d) sinc(100πt) + 3 sinc2(60πt)

(e) sinc(50πt) sinc(100πt)

Remark: Recall that in our class, sinc(x) = sin(x)x.

4-2

ECS 332 HW 4 — Due: Sep 7 2012/1

Problem 3. Consider a signal g(t) = sinc(πt).

(a) Sketch the Fourier transform G(f) of g(t).

(b) Find the Nyquist sampling rate.

(c) Recall that the instantaneous sampled signal gδ(t) is defined by

gδ(t) = g(t)×∞∑

n=−∞

δ(t− nTs)

where Ts is the sampling interval.

(i) Let Ts = 0.5. Sketch the Fourier transform Gδ(f) of gδ(t).

(ii) Let Ts = 4/3. Sketch the Fourier transform Gδ(f) of gδ(t).

(d) The sequence of sampled values g[n] is constructed from g(t) by

g[n] = g (t)|t=nTs .Recall the reconstruction equation:

gr(t) =∞∑

n=−∞

g[n] sinc(πfs(t− nTs)).

Note that we write gr(t) instead of g(t) to accommodate the case that the samplingrate is too low; in which case, the reconstructed signal is not the same as g(t).

(i) With Ts = 1,

i. Find g[n] for n = . . . ,−4,−3,−2,−1, 0, 1, 2, 3, 4, . . .

ii. Use the reconstruction equation to find gr(t).

(ii) Let’s test the reconstruction equation by using MATLAB to plot gr(t). Note thatthe sum in the reconstruction equation extends from −∞ to +∞. In MATLAB, wecan not add that many terms. So, we need to stop at some n. In this part, useTs = 0.5.

i. Use MATLAB to plot gr(t) when only n = 0 term is included.

ii. Use MATLAB to plot gr(t) and all of its sinc components. Include only n =−1, 0, 1.

iii. Use MATLAB to plot gr(t) and all of its sinc components. Include only n =−5,−4, . . . ,−1, 0, 1, . . . , 4, 5.

iv. Use MATLAB to plot gr(t) and all of its sinc components. Include only n =−10,−9, . . . ,−1, 0, 1, . . . , 9, 10.

In all these plots, consider t from -4 to 4. Also include the plot of sinc(πt) forcomparison.

4-3

ECS 332: Solution for Problem Set 4

Problem 1: Aliasing and periodic square wave In the time domain, the switching between the values -1 and 1 should be faster as we increase

f0. In the frequency domain, we should get impulses (spikes) at all the odd-integer multiples of

f0 Hz. The center spikes (at f0) should be the largest among them. All the plots below are

adjusted so that they show 10 periods of the “original signal” in the time domain.

From the plots, as we increase f0 from 10 to 20 Hz, the locations of spikes changes from all the

odd-integer multiples of 10 Hz to all the odd-integer multiples of 20 Hz. In particular, we see the

spikes at 20, 60, 100, 140, 180, 220, 260, 300, 340, 380, 420, 460. Note that

plotspec only plots from [-fs/2,fs/2). So, we see a spike at -500 but not 500. Of course, the

Fourier transform of the sampled waveform is periodic and hence when we replicate the

spectrum every fs, we will have a spike at 500. Note that in reality, we should also see spikes at

540, 580, 620, 660, and so on. However, because the sampling rate is 1000, these high

frequency spikes will suffer from aliasing and fold back into our viewing window [-fs/2,fs/2).

However, they fall back to the frequencies that already have spikes (for example, 540 will fold

back to 460, and 580 will fold back to 420) and therefore the aliasing effect is not easily

noticeable in the frequency domain.

When f0 = 40, we start to see the aliasing effect in the frequency domain. Instead of seeing

spikes only at 40, 120, 200, 280, 360, 440, the spikes at higher frequencies (such as

520, 600, and so on) fold back to lower frequencies (such as 480, 400, and so on). The plot

still looks OK in the time domain.

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

-1

-0.5

0

0.5

1

Seconds

-500 -400 -300 -200 -100 0 100 200 300 400 5000

0.5

1

1.5

Frequency [Hz]

Magnitude

0 0.05 0.1 0.15 0.2 0.25

-1

-0.5

0

0.5

1

Seconds

-500 -400 -300 -200 -100 0 100 200 300 400 5000

0.5

1

1.5

Frequency [Hz]

Magnitude

At high fundamental frequency f0 = 100, we see stronger effect of aliasing. In the time

domain, the waveform does not look quite “rectangular”. In the frequency domain, we only

see the spikes at 100, 300, and 500. These are at the correct locations. However, there

are too few of them to reconstruct a square waveform. The rest of the spikes are beyond

our viewing window. We can’t see them directly because they fold back to the frequencies

that are already occupied by the lower frequencies.

Our problem can be mitigated by reducing the sampling interval to TS = 1/1e4 instead of TS =

1/1e3. Now, the spikes show up again as shown by the plot on the right above.

Finally, at the highest frequency f0 = 300, if we still use T = 1/1e3, the waveform will be

heavily distorted in the time domain. This is shown in the left plot below. We have large

spikes at 300 as expected. However, the next pair which should occur at 900 is out of the

viewing window and therefore fold back to 100. Again, the aliasing effect can be mitigated

by reducing the sampling time to T = 1/1e4 instead of T = 1/1e3. Now, more spikes show up

at their expected places. Note that we can still see a lot of small spikes scattered across the

frequency domain. These are again the spikes from higher frequency which fold back to our

viewing window.

0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1

-1

-0.5

0

0.5

1

Seconds

-500 -400 -300 -200 -100 0 100 200 300 400 5000

0.5

1

1.5

Frequency [Hz]

Magnitude

0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1

-1

-0.5

0

0.5

1

Seconds

-5000 -4000 -3000 -2000 -1000 0 1000 2000 3000 4000 50000

0.5

1

1.5

Frequency [Hz]M

agnitude

0 0.005 0.01 0.015 0.02 0.025 0.03

-1

-0.5

0

0.5

1

Seconds

-500 -400 -300 -200 -100 0 100 200 300 400 5000

0.5

1

1.5

Frequency [Hz]

Magnitude

0 0.005 0.01 0.015 0.02 0.025 0.03

-1

-0.5

0

0.5

1

Seconds

-5000 -4000 -3000 -2000 -1000 0 1000 2000 3000 4000 50000

0.5

1

1.5

Frequency [Hz]

Magnitude

Q2 Nyquist sampling rate and Nyquist sampling intervalSunday, July 17, 20112:09 PM

ECS332 HW4 Sol Page 1

Q3 Sinc Reconstruction of SincThursday, August 30, 20121:51 PM


-4 -3 -2 -1 0 1 2 3 4-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

t

-4 -3 -2 -1 0 1 2 3 4-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

t

-4 -3 -2 -1 0 1 2 3 4-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

t

-4 -3 -2 -1 0 1 2 3 4-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

t



HW 5 — Due: Sep 26


Instructions





Problem 1. Consider a signal g(t) = sinc (3(t− 5)).

(a) Is g(t) time-limited?

(b) Is g(t) band-limited?

(c) Carefully sketch g(t)

(d) Carefully sketch the magnitude |G(f)| of the Fourier transform G(f).

Problem 2. State the reconstruction formula.Hint: You should be able to do this by recalling the reconstruction process.

Problem 3. State the Nyquist’s (first) criterion for zero ISI

(a) In the time domain.

(b) In the frequency domain.

Problem 4. In each part below, a pulse P (f) is defined in the frequency domain from f = 0to f = 1. Outside of [0, 1], you task is to assign value(s) to P (f) so that it becomes a Nyquistpulse. Of course, you will also need to specify the symbol interval T as well.

Hint: To avoid dealing with complex-valued P (f), you may assume that p(t) is real-valuedand even; in which case P (f) is also real-valued and even.

(a) Find a Nyquist pulse P (f) whose P (f) = 0.5 on [0, 1].

5-1

ECS 332 HW 5 — Due: Sep 26 2012/1

(b) Find a Nyquist pulse P (f) whose P (f) = 0.25 on [0, 1].

(c) Find a Nyquist pulse P (f) whose

P (f) =

{0.5, 0 ≤ f < 0.50.25, 0.5 ≤ f ≤ 1

(d) Find a Nyquist pulse P (f) whose

P (f) =

{0.5, f ∈ [0, 0.25) ∪ [0.5, 0.75)0.25, f ∈ [0.25, 0.5) ∪ [0.75, 1]

Problem 5. Consider a raised cosine pulse p(t) and its Fourier transform P (f). Assumethe rolloff factor α = 0.3 and the symbol “duration” T = 1.

(a) Carefully sketch P (f).

(b) Find p(2).

(c) Find P (0.5).

(d) Find P (0.3).

(e) *Find P (0.4).

Remark: You should be able to solve this problem without referring to the “ugly” formula.

Problem 6. Consider a raised cosine pulse p(t) with rolloff factor α and symbol “duration”T .

(a) Find p(T/2) as a function of α.

(b) Use MATLAB to plot p(T/2) as a function of α.

5-2

0 1 2 3 4 5 6 7 8-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Q1 Sinc ReviewMonday, September 17, 20129:09 PM

ECS332 HW 5 Sol Page 1

Q2 Reconstruction FormulaMonday, September 17, 20129:34 PM


Q3 Nyquist's CriterionMonday, September 17, 20129:47 PM


Q4 Nyquist PulsesMonday, September 17, 20124:20 PM


Q5 Raised Cosine PulseMonday, September 17, 20127:51 PM


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

Q6 Raised Cosine PulseMonday, September 17, 20127:38 PM



HW 6 — Due: Oct 5


Instructions





Problem 1. Consider the code {0, 01}

(a) Is it nonsingular?

(b) Is it uniquely decodable?

(c) Is it prefix-free?

Problem 2. Consider the random variable X whose support SX contains seven values:

SX = {x1, x2, . . . , x7} .

Their corresponding probabilities are given by

x x1 x2 x3 x4 x5 x6 x7

pX(x) 0.49 0.26 0.12 0.04 0.04 0.03 0.02

(a) Find the entropy H(X).

(b) Find a binary Huffman code for X.

(c) Find the expected codelength for the encoding in part (b).

Problem 3. Find the entropy and the binary Huffman code for the random variable X withpmf

pX(x) =x

21for x = 1, 2, . . . , 6.

Also calculate E [`(X)] when Huffman code is used.

6-1

ECS 332 HW 6 — Due: Oct 5 2012/1

Problem 4. Construct a random variable X (and its pmf) whose Huffman code is {0, 10, 11}.

Problem 5. These codes cannot be Huffman codes. Why?

(a) {00, 01, 10, 110}

(b) {01, 10}

Hint: Huffman code is optimal.

Problem 6. A memoryless source emits two possible message Y(es) and N(o) with proba-bility 0.9 and 0.1, respectively.

(a) Determine the entropy (per source symbol) of this source.

(b) Find the expected codeword length per symbol of the Huffman binary code for thethird-order extensions of this source.

(c) Use MATLAB to find the expected codeword length per symbol of the Huffman binarycode for the fourth-order extensions of this source.

(d) Use MATLAB to plot the expected codeword length per symbol of the Huffman binarycode for the nth-order extensions of this source for n = 1, 2, . . . , 8.

6-2

Q1 Classes of CodesSunday, August 28, 20119:33 PM


Q2 Huffman CodeSunday, August 28, 20119:38 PM


Q3 Huffman codeSunday, August 28, 201110:13 PM


Q4 Inverse Huffman ProblemSunday, August 28, 201110:07 PM


Q5 Non-Huffman CodesThursday, September 27, 20129:08 PM


Q6 Source ExtensionMonday, August 29, 20119:35 AM


1 2 3 4 5 6 7 80.4

0.6

0.8

1

1.2

1.4

1.6

1.8

n



HW Solution 7 — Due: N/ALecturer: Prapun Suksompong, Ph.D.

Problem 1. Optimal code lengths that require one bit above entropy : The sourcecoding theorem says that the Huffman code for a random variable X has an expected lengthstrictly less than H(X) + 1. Give an example of a random variable for which the expectedlength of the Huffman code is very close to H(X) + 1.

Problem 2. Consider the AWGN channel:

Y = X +N

where N ∼ N (0, σ2N). Assume that X |= N and that X takes two values, a and −a. For the

whole question, assume a > 0.

(a) In this part, use a = 5 and σN = 3.

The channel output Y is fed into a decision device (a comparator) which will comparethe value of Y to 0. The output X of the decision (thresholding) device is determinedby

X =

{a, if Y ≥ 0,−a, if Y < 0.

The whole system in shown in Figure 7.1.

Decision

device 𝑋

𝑌 𝑋

𝑁

Figure 7.1: System for Q1.a

Find

(i) P[X = −a|X = a

](ii) P

[X = a|X = a

]7-1

ECS 332 HW Solution 7 — Due: N/A 2012/1

(iii) P[X = −a|X = −a

](iv) P

[X = a|X = −a

](v) P

[X 6= X

]Hint: By the total probability theorem,

P[X 6= X

]= P

[X 6= X |X = a

]P [X = a] + P

[X 6= X |X = −a

]P [X = −a] ,

= P[X 6= a |X = a

]P [X = a] + P

[X 6= −a |X = −a

]P [X = −a] .

(b) Continue from part (a).

We can use the system from part (a) to transmit/receive binary information by addinga simple mapping device with map 0 to a and 1 to −a at the transmitter.

At the receiver, we also have another mapping device that map the a and −a back to0 and 1, respectively.

The new system is shown in Figure 7.2.

Decision

device

𝑋 𝑌 𝑋

𝑁

mapping mapping 𝑆 𝑍

Figure 7.2: System for Q1.b

Note that S and Z are binary and that the whole system (inside the dotted box)in Figure 7.2 can be reduced to a binary symmetric channel (BSC) with crossoverprobability p.

Find p.

(c) Observe that as σN increases, the value of p in part (b) will also increase.

Is it possible to find σN such that p > 0.5?

(d) Express p using a, σN , and the Q function.

Problem 3. Consider a transmission over the BSC with crossover probability p. The randominput to the BSC is denoted by S. Assume S ∼ bernoulli(p1). Let Z be the output of theBSC.

7-2


(a) Suppose, at the receiver (which observes the output of the BSC), we learned thatZ = 1. For each of the following scenarios, which event is more likely, S = 1 wastransmitted or S = 0 was transmitted? (Hint: Use Bayes’ theorem.)

(i) Assume p = 0.3 and p1 = 0.1

(ii) Assume p = 0.3 and p1 = 0.5

(iii) Assume p = 0.3 and p1 = 0.9

(iv) Assume p = 0.7 and p1 = 0.5

(b) Suppose, at the receiver (which observes the output of the BSC), we learned thatZ = 0. For each of the following scenarios, which event is more likely, S = 1 wastransmitted or S = 0 was transmitted?

(i) Assume p = 0.3 and p1 = 0.1

(ii) Assume p = 0.3 and p1 = 0.5

(iii) Assume p = 0.3 and p1 = 0.9

(iv) Assume p = 0.7 and p1 = 0.5

Remark: A MAP (maximum a posteriori) detector is a detector that takes the observedvalue Z and then calculate the most likely transmitted value. More specifically,

SMAP (z) = arg maxsP [S = s|Z = z]

In fact, in part (a), each of your answers is SMAP (1) and in part (b), each of your answersis SMAP (0).

Problem 4. Consider a repetition code with a code rate of 1/5. Assume that the code isused over a BSC with crossover probability p = 0.4.

(a) Assume that the receiver uses majority vote to decode the transmitted bit. Find theprobability of error.

(b) Assume that the source produces source bit S with

P [S = 0] = 1− P [S = 1] = 0.45.

Suppose the receiver observes 01001.

(i) What is the probability that 0 was transmitted? (Do not forget that this isa conditional probability. The answer is not 0.4 because we have some extrainformation from the observed bits at the receiver.)

7-3


(ii) What is the probability that 1 was transmitted?

(iii) Given the observed 01001, which event is more likely, S = 1 was transmitted orS = 0 was transmitted? Does your answer agree with the majority voting rulefor decoding?

(c) Assume that the source produces source bit S with

P [S = 0] = 1− P [S = 1] = p0.

Suppose the receiver observes 01001.

(i) What is the probability that 0 was transmitted?

(ii) What is the probability that 1 was transmitted?

(iii) Given the observed 01001, which event is more likely, S = 1 was transmitted orS = 0 was transmitted? Your answer may depend on the value of p0. Does youranswer agree with the majority voting rule for decoding?

Problem 5. A channel encoder map blocks of two bits to five-bit (channel) codewords. Thefour possible codewords are 00000, 01000, 10001, and 11111. A codeword is transmittedover the BSC with crossover probability p = 0.1. Suppose the receiver observes 01001 at theoutput of the BSC.

(a) Assume that all four codewords are equally likely to be transmitted. Given the observed01001 at the receiver, what is the most likely codeword that was transmitted?

(b) The Hamming distance between two binary vectors is defined as the number of positionsat which the corresponding bits are different. For example, the Hamming distancebetween 00000 and 01001 is 2.

The minimum distance decoder is defined as the decoder that compare the Hammingdistances between the observed bits at the receiver and each of the possible codewords.The output of this decoder is the codeword that give the minimum distance.

Explain why the minimum distance decoder would give the same decoded codeword asthe decoder in part (a).

(c) What is the minimum (Hamming) distance dmin among codewords?

(d) Assume that the four codewords are not equally likely. Suppose 11111 is transmittedmore frequently with probability 0.7. The other three codewords are transmitted withprobability 0.1 each.

Given the observed 01001 at the receiver, what is the most likely codeword that wastransmitted?

7-4

Q1Monday, August 29, 201110:20 AM


Q2Tuesday, September 20, 201111:05 AM


Q3Tuesday, October 09, 20128:37 AM


ecs 332: principles of communications 2012/1 hw 4 | due: sep 7 hw.pdf · ecs 332: principles of...

Documents