ECOM- 5151AUGUST 1967

t : CHANNEL CAPACITY AND CODING

ByLouis Dean Duncan

0

• ATMOSPEMC SCIENCES WAORATORY• WHITE SANDS MISSILE RANGE, NEW ;. 4EXICO

0 I Urit'

OW04 Dishtibution cf this

• • • • • • • •• •• •• •• •• •report is unlimited.

UNITED STATES ARMY ELECTR ONICS COMMAND '

Best Available Copn

CIIANNEL CAPACdrY AND) COI)INC(*

LOUIS 1). DlUNCAN

[Cnli - 55 Aurust 1967

D)A PROJECT 1L013001A91..

AT'lOrIll*R IC 'SC I ETS T.AV("RATtORYIMIITF SANDlS 'II SS IIE rA.XI , NLIW 'TXICO

Digtribution of thisreport is unlirited.

tte*D~issertatio~n for the fle'recc of D~octor o " 1Nbl*taeUniversity, Inivercity Park. N. 'I, July 1967.

Ztf1

ABSTRACr

A channel is defined to be a triple S * (X, u(.Ixl, (Y, Y'11

where X and Y are abstract sets, Y' is a o-algebra of subsets of

Y, and u(.Ix) :is a probability measure on (Y, Y'1 for each xcX.

X and Y are usually assumed to be either subsets of the real

numbers or subsets of Euclidean n-space. The channel has addi.

tive noise, w, if u(AtxI - u(A-x) for all xcX, AcY'. For a

given channel S and a given real nuaber 0 < A < 1 a X-code of

length n for S is a set of pairs (x,, D1 ) ... (Xn' D n wheren

x CX, Dicy, Di/' Di I D if i # j and w(DiLxi 1 l-X for i a I

g, ,,, n. The supreum of the nonempty set of integers N such

that S admits a X-code of length N is denoted by N(S, A).

Let Q*(Sj be the set of all probability measures defined on

X. For rcQ*(SI. Let y be the measure defined on (Y, Y' by

y (B) (Bix)d i

for all BeY'. For xcX, let f'(ylxl u du(.Ix)/dy! (the Radon-

Nikodym derivative). The capacity, C, of the channel S is

defined to be sup {C(ir)4wcQ*(S., support of w finite), where

C(O - ff log f'(ylx) u(dylx)wCdx).

The following inprovement of Fano's theorem is proved in

Chaptor II.

iii

Theorem: Let S be a channel with capacity C, and let

0 < A < 1 be given. 1hen for any n > 0, 0 < t < 1

(l-Xj log N(Sn, X) f nC * log ( J..

t

In Chapter III the connected X-code is defined and investi-

gated. This code.(xi, Dil' is defined to be a A-code where

each D. is a U-connected set; U is a topology on Y. The

supremum of the set {NIS admits a connected -code of length N}

is denoted by N(S, A, U). Most of the results of this chapter

are for a channel of type A - a channel with additive noise u

which is absolutel;' continuous with respect to Lobesiue measure,

y; X is a closed interval; Y is the real nubers; and, U is the

usual topology for the reals.

Let f u du/dy, the Radon-Nikodym derivative. f is a bell

function if there exists y such that f is increasing for all

y y y0 and f is decreasing for all y Zy. In Chapter III two

conditions for N(S, 11 u N(S. A, U) are obtained when S is a

channel of type A and f is a bell function. One of these condi-

tions is necessary; the other is sufficient.

A technique for delineating those measures which. cannot

affect the value of N(S. A, L.a is investigated in Chapter IV.

iv

Sufficient conditions for a channel to have finite capacity

are investigated in Chaptpr V. The main result is:

Theorem: Let S be a channel with additive noise U

which is absolutely continuous with respect to Lebesgus meas-

ure, y. If there exists a choice for the Radon-Nikodym deriva-

tive, f a du/dy, such that f g(y)dy < - where g(Y1

sup (f(yjx)jxcX) then the capacity of S is finite.

V

I!

TABLE OF CONTENTS

CHAPTER Pg

I. INTRODUCTION AND PRELIMINARIES 1

II* AN IMPROVEM~NT OF FANO'S THEOREM 1s

III,* CONNECTED A~-CODES 20

IV. AD1ISSABLE tEASURES 54IV. SUFFICIENT CONDITIONS FOR FINITE CAPACITY 59

B IBLIOGRAPHY 641

viiI

LIST OF FICURJES

Figure No. Page

1 A Typical Menber of F 42

2 Definition of the Interval (aIs a2) 44

3 Definition of the Interval (8, a- s/2) 45

4 A Choice for f(y, [ 1) 4S

5 The First Counter example 52

6 The Second Counter example 53

viii

QiAPTER I

INTRODUCTION AND PRELIMINARIES

Most of the terminology and notations which will be used

herein are standard; however, to remove any possible ambiguity,

much of it will be explained. For a set A, acA means that a is

a member of A; the complement of A will be written AC. It will

sometimes be convenient to refer to a member of A as a point of

A. All logarithms will be to the -se e. The convention will

be adopted throughout to define the expression 0 log 0 to be

equal to O. Integrals will always be in the sense of Lebesgue

integration. The meaning of unexplained terminology or notation

of integration theory or measure theory will be that of Halmos (4],

and the meaning of any unexplained probability theory terminology

or notation will be that of Loeve [6].

Information theory is one of the youngest branches of applied

probability theory. Its conception can, with certainty, be con-

sidered to be the appearance in 1948 of the now classical work of

Shannon [7]. From the very begining, information theory presented

to mathematicians a whole new set of problem, including some very

difficult ones. It is quite natural that early investigators,

including Shannon, whose basic goal was to obtain practical results,

2

were not able to give enough attention to these mathematical

difficulties. Consequently at many points of their investiga-

tions, they were compelled either to be satisfied with reasor-

ing of an inconclusive nature or to limit the set of objects

studied.

Investigations With the aim of setting information theory

on a solid mathematical basis have begun to appear in recent

years. However, in most of these endeavors finiteness conditions

Piave been placed on certain sets in order to establish the desired

results.

One of the most important entities considered in the mathe-

matical study of information theory is the concept of a channel.

A channel is defined to be the triple S - (X, U(. Ix), (Y, Y'))

where X is an arbitrRry set, (Y, Y') is a measurable space, and

u(. Ix) is a prbability measure on (Y, Y,) for each xcX. The

set X is usually referred to as the input or input space; the set

Y is referred to as the output or output space. If both X and Y

contain only a finite number of elements, the channel is said to

be discrete; if X has a finite number of elements but Y is infinite

(either countable or uncountable), S is called semi-continuous (the

term "semi-continuous" is of engineering origin).

i

3

In most of the literature it is assumed that the channel

is either discrete or semi-continuous. Such restrictions will

not be made in this dissertation; it will be assumed throughout

that X and Y are arbitrary sets unless there is a specific state-

mnt to the contrary.

A channel operates as follows. The existence of a sender

and receiver is assumed. The function of the sender is to choose

a mener of X and transmit it. Since it is not necessary to have

a precise definition of the term transmit, a somewhat heuristic

explanation is given. Transmission consists of hoosing a point

xcX and associating with x a point ytY which, in general, depends

on x. The points x and y will be called the transmitted syvol

(transmitted signal, symbol sent, etc.) and the recaived symbol

(received signal, received point, etc.), respectively. If a

particular transmitted syubol always results in the same received

syrbol, the transmission may be considered as a function T: X * Y.

The hiore interesting case is that in which a given transmitted

signal does not always result in the same received symbol. In

this case the function T mast be considered as a function of x

and another variable, called the noise. The received variable is

considered to be a chance variable, i.e., specific occurrences

are governed by probability. The only property of the transmission

known by the sendor and receiver is that for each x sent the

i I I I i

4

probability that the received symbol is a member of AcY' is

u(,Ax. The receiver is capable of scanning through any finite

class of sets of Y' and determining which, if any, contains the

received symbol. After making this determination, the receiver

then tries to decide, based upon some type of logical analysis,

which member of X was actualiy transmitted.

The technique which is usually employed by the receiver to

decide which point was transmitted involves a predetermined

decision scheme which is known to both the sender and the receiver.

There are, of course, many ways by which this decision sheme can

be defined. The one which has become a standard in studies of

information theory is as follows: Let 0 - A < 1 be given; let

(Xl, DI), ... , (Xn, Dn) be members of X x Y, having the properties

that DiC! D 0 if i 0 j, and such that u(Dilxi) ! 1-1. The

sender and receiver then agree to consider only those members

X18 X2 , ... , Xnex. If Xk is transmitted, the receiver scans through

the sets D, s*$ Dn and determines which, if any, contains the re-

ceived symbol. If the received symbol is in Dm, the receiver con-

cludes that x was transmitted; if none of these sets contain the

received symbol, any decision may be made. The receiver's conclusion

will be correct with probability ! 1-k.

1.1 Definition: A set of pairs ((xl, Dl), ... 8 xn, D n) hav-

ing the properties described above is called a A-code of length n.

S

A quantity which will be of considerable importance later,

in fact that subject of most of the important theorems in informa-

tion theory, is defined below.

1.2 Definition: Lot S be a channel. Given 0 ! X - 1, let

N(S, i) denote the supresmi (sup.) of the non-empty set of inte-

gers N such that S admits a )-code of length N.

Given n channels S. * { s, (. x). (Ypm' Y21)), a 1 , 2,

... , n, one can form the product channel, S(n) S x .x % in

a natural way. In fact this channel is defined by s(n) . (- n

U C. u) , yCn] yCn,))) where XCn) . XI x . x l n; (Y

Y (nl,) is the product of the measurable spaces (Y*, Y,) mP6 1, 2,

... , n, as defined by Halmos [4]; and if udX, i.e., u - (xl*....

Xn) with xmcXa). then u (n) (.Iu) denotes the product probability

measure on yCn), defined by (n) (BI x x Bn) u i(Bl1 X I) ....

u(B Ixn ) where BCY .. ,, Cy

1.3 Definition: The channel S defined above is called a

memoryless channel of length n. If S - S2 - ... S " S, one

writes S(n) * Sn and calls Sn the memoryless channel of length n

generated by S. Any channel may be regarded as a memoryless channel

of length 1.

6(n~ Ii

1.4 Remark: The definitions of N(Scn), X) and N(Sn, A)

follows immediately from definitions 1.2 and 1.3.

Suppose there are N distinct points in X which the sender

wishes to transmit in such a manner that, for a predetermined

0 ! X c 16 the probability that the receiver will wrongly deduce

which 1oint was sent is - A. The channel S caunnot necessarily

perforq this function if N > N(S, X). However, if for fixed A,

N(Sn,, X) becomes unbounded with n, the problem can be solved by

choosing an n0 such that N(S n, X) > N, establishing a one-to-one

correspondence between the N points and a properly chosen set of

nnN members of Xn° , and using the channel Sn0.

This problem and two equivalent (according to Wolfwitt [1111

versions are listed below:

Form I : Given N and X, how small an n will suffice?

Form II : Given n and A, how big an N can be achieved?

Form. III: Given n and N, how small a A can be achieved?

'A companion problem to the above problem (call it the first)

is the (second). problem of constructing a code to implement the

answer to the first problem. In fact, it might be reasonably

thought that the first problem could not be solved without a

solution of the second. This is not the case, and, at present,

existing knowledge about the first problem considerably exceeds

the existing knowledge about the second problem.

One of the main objectives of this dissertation is to define

a technique for coding (called a connected A-code) and to investi-

late what conditions must be placed on the channel so that such a

code will provide a solution to the second problem.

The solution of the first problem is usually referred to as

the coding theorem and its converse. Precise statements of these

theorems must be reserved for later, since they involve terminology

which has not yet been introduced

The other main objective of this dissertation is to obtain an

improvement of the known results for the converse of the coding

theorem for the general channel. It will be seen later that this

amounts to obtaining an improvement of the known results for an

upper bound for N(S n A) where S is a general channel (the channel

where both the input and the output are arbitrary sets).

Before continuing toward these objectives, some basic nota-

tion. definitions, and theorem will be listed for future refer-

once. Since information theory employs many of the tools of pro-

bability theory and measure and integration theory this list a

fortiori contains results from these disciplines.

A r

8

In addition to probability statements about the received

symbol, many of the important results entail probabilistic

statements about the transmitted symbol. The following develop-

mants will indicate the importance and application of such

statements as well as provide a rigorous foundation for their

formulation.

i.5 Definition: Let S' -{X, u(.Ix), (Y, Y')} be a given

channel. The set Q* - Q*(S') is defined to be the collection

of all entities

w - {XO, XO, Wi00

of the following kind: The set X .(called the support of w)0

is any fixed Ruhqet of X, X' is a a -algebra of subsets of X0 0

containing all sets of the form

( IxCXo, u(Blx)< a,BcY',a real)

i.e., such that the function u(BIx) is measurable in x for

fixed BeY'. Finally w is a probability measure on the a-

algebra X'.0

The apparent ambiguity in letting the Greek letter w

both represent and be a member of-the entity {Xo, X', w}00

will cause no confusion in usage and will allow for simpli-

city in notation.

9 "'

1.6 Definition: Let (Y, Y') be a measurable space, and

let u and y be measures on Y' . Then u is absolutely continuous

with respect to y (written u <- y) if y(A) - 0 implies v(A) - 0;

u is singular with respect to y(ujy) if there exists a set MY'

such that u(A) a 0 and y(A% . 0.

The following well-known theorem is stated for complete-

ness.

1.7 Lebesgue Decouposition Theorem: Let u and y be was-

ures defined on the measurable space (Y. Y'); then ii can be

written uniquely as the sum of two measures ul and u2 where

Ul < c y and u 2..y

1.8 Remark: Any set D such that u(D) =u (Y) is called

a support of u. This will be written D u spt u. If uyy D can

be chosen such that y(D) a 0. Whenever u- ul+V with u << Y

and u2 jy a support, D, can always be chosen for i, i1., 2

such that uj (D) - 0 jyi. hroughout this dissertation such a

choice for a support will always be implied.

1.9 Remark: Let U and y be defined as above and let

U 01 +U 2 where u1 << Y and P2iy. Then there exists a meas-

urable set D such that y(D) - 0 and u 2(Dc) 0. The set D is

called a singular set of u with respect to ,y. Let g(y) - d 1 /dy

10

(the Rion-Nikodym derivative), then g(y) is a finite-valued,

non-negative, real-valued, y-measurable function, unique up to

a set of y-masure 0, such that

1AI(BI g(y)dY, for BcY'

By definition of D, one is allowed to assume that

0 < g(y) <- if ycD,

g(y) " if ycD.

Throughout this dissertation such a determination of Radon-

Nikodym derivatives will always be chosen.

The preceding developments provide a suitable background

for the definition of a. new set of probability measures on

(Y,. Y'). Let icQ'. For BcY' define

y'(B - "(B x)1T(dx)

It is easy to see that y is a probability measure on (Y, Y')

for each wcQ*.

Let fN(ylx) w du1 /dy where ) 1A. x) is the absolutely con-

tinuous component of u(.Ix) with respect to y . Loeve (6] has

shown that f'yR x) can be chosen such that it is jointly meas-

urable in xcXo, ycY' relative to the a - algebra X, xY'. By

definition of the set D in Remark 1.9, it is clear that f?(ylx)

I

can (and will) be chosen s0 that it also satisfies this joint

measurbility condition in addition to the requirement spec-

ified in the remark.

'The measure y Wand the functions f WCytx) will now be

used to define an important characteristic of the channel.

the Capacity.

Let wcQ* be axbitrary but fixed. Define

CQw) * flog fw&Il) I(dylx) w(dx),

Hfw(yjx) log fw'CI) ylr(dy)w(dx) '

if Ia(.Ix) << for almost all [wr] xeX. Otheywise,,define

CQW) -*

If the support of wr is countable, the following equiva-

lent definition of C(!) will be convenient. If IA(. Ix) ccy W

define CQxjr) * 'log f'(ylx) i(dylx). It is observed that,

in this case, IA(.Lx) y-" for almost all [wi] x in the support

of wr. Hence,

xCSptW

1.10 Definition: Let Q a {rcQ*I support of wr is finite).

Let C m sup (CCIr) 171EQ). The quantity C is called the -capacity

of the channel.

44

12

The following important and somewhat surprising result in

due to Kemperman [5].

1.11 Theorem: Let C' - sup (C(w)JwcQ*). Then C° w C.

The coding theorem and its converse can now be stated.

The following statements of these theorems are those of

Wolfowitz (11).

1.12 Theorem: (The coding Theorem): Let 0 < A < 1 be

given. Then there exists a positive constant K such that for

any n > 0

N(Sn , A) e n C" Kn

1.13 Theorem: (The strong Converse). Let 0 < X < 1 and

c > 0 be given. Then for any n sufficiently large

N(Sn A) n(C )

1. 14 Theorem: (The weak Converse). Given 0 < A < 1.

Then for all n > 0

(l-X) log N(Sn , A) f nC + log 2.

Theorem 1.12 was conjecturad for the. discrete channel by

Shannon (7] in 1948. The first proof was given by Feinstein

(1] in 1954. Essentially different proofs were given in 1957

by Shannon [8] and Wolfowitz (9]. Shannon also conjectured

13

theorem 1.14 for the discrete channel. The strong convere is

due to Wolfowitz [9].

Wolfowitz [10] has shown that theorem 1.12 is true for a

semicontinuous channel by approximating the semicontinuous

channel by a discrete channel. The proof of theorem 1.13 for

the semicontinuous channel is also due to Wolfowitz (10]. The

following stronger version is due to Kemperman (S].

1.15 Theorem: Let 0 < A ( 1 be given. Then for any

semicontinuous memoryless channel S there exists a constant

K > 0 such that for any n ) 0

N(Sn, X) •nC + K n

Theorem 1.14 is due to Fano (2] who proved it for the

general channel. An essentially different proof of this

theorem has been given by Kenperman [S]. A different expres-

sion for the right-hand side of the formula given in theorem

1.14 will be *otained in Chapter II. For small A this will

give a much better result (in the since of a smaller upper

bound) than the one given in 1.14. The following well known

theorem which will be needed for the proof is listed for

reference.

.... w~ N m

14

1.16 Theorem: Let S, m I 1.... n be arbitrary channels i

of capacity Cm . Then the capacity of the product channel S( n )

is C1 ... C .

I

.I,

is

CHAPTER II

MN IMPROKVEM~ENT OF FANO'S THEOREM

It is not unt'sual in the field of mathematics for one to

conjecture an extension of a known result to a more general

setting without being able to obtain a proof. Sometimes this

conjecture remains an open problem for many years. This io

the current statu~s of both the coding theorem and the strong

converse for the general channel.

In the formulation~ of a theorem the primuary objective of

which is to obtain an upper bound for ms quuntity, one

usually attempts to establish a **all an upper bound as pose-

ib le. The author is unaware of any theorem which gives a better

upper bound for N(S, A) for a general channel than theorem 1'.14,

Farto's theorem, The theorem presented belowe gives an improve-

ment, for small A," of Fano's result. The proof of theorem 2.1

is a modification of the proof of 1.14 given by Kemperuan (5).

2.1 Theorem: Let S *(X, u(.Ix),(Y. Y)) be a given

channel. Then given 0 'A < 1, 0 t < I, one has, for each

positive n

n I t(1-A) log N($~ A nC *log ( 0 1)

t

where C is the capacity of S.

16

Proof:

It follows from theorem 1.16 that the capacity of the

channel Sn is nC; therefore, it suffices to prove that

(1-A) log N(S, X) . C + log t

t

Let N be any positive integer such that N I N(: A ). 7hen

there exists a X-code of length N; call it {(xi, Dijj I. Let

A a {xilCz&, Dij is a member of this X-code). (This notation

could be simplified by asuming that xi's were distinct members

Nof X). For each xcA let N('x) - 1, and let w(x) a N(x)/N.

jul uxX

It is easy to see that i is a probability measure on X with

finite support. (More precisely itcQ(S).)

For each BcY' define

Y' (B) - j (gX fxcA

As has been pointed out yT is a prcbability measure on (Y, Y')

and u(.Ix. << Y' for all xzcA. For ff(x) > a let

f'(ylx) - duC.Ix)/dyw.

Now, by definition,

(it(X) f fW ulx) log f"(ylx)'y T'(dy) " CC,) I C.xcA

17

Obrerve that

INN- I a~i N' 1 N(x) f'(ylx)

1-1 xcA

-c WC(X) f'Cy'x) " 1.xcA

Similary,

N- N" I I fwCylxi .og f"Cylxi)YWCdyI

jul:

- '(xj f f"(ylxl log f (ylx)y'W("y) . C.xcA

A new set of fumctions is defined on Y as follo.s. For

each 1 i I N. define

N if ycDi

hiCYI -

t if ycD5

NThen I hi(y) .< N(l+tj for each y.

i-i

The desired results will now be obtained by analyzing the

terms of the following equation:

N1 N h i. L(

N'l f e lx, log y , dyIi-~~ ( wy Xii

A

n in n. .........

i-11

*N-1 I f fCjx) log fW(ylxj)y'(dyj

NN -I I f'Cyxil log hiCy)yCdy).

Jul

For reference this will be called equation (,

For each 0 .1 i1 N

f fy(yIx 1) log hi(yh 'O(dy f log hi(y ' d 1'' '4,j ,

UI(Di xi log N * u(Dclxi1 log t.

. (1 - 1) log N * A log t.

N

Therefore, N 1 e f f(ylxi) log hi(y)-Cfdy)1.1 1

.>(.1 - 1) log N * A log t,

The function log Z is concave; hence,

Il N lo icy~) -~N

N f,(Ylxil log log N- I hiLy. log (1 *t1~.

I.

I

19

Therefore. N 1 f'(Ylxi) log h() Y(d1- log (1I )

Inserting these inequalitites into equation C*one cbtains

log (1 + t) + C (1I A log N *A log t.

which is eq~uivalent to

(I - Al log N .~C *log(1 tt

Sine N N(S, A) was arbitrary, the theorem is proved.

2.2 Remark:, Given 0 < A <1/2 let f(t) (Il t)/tA

for t c(0, 1) f (t) + - if t 0(O* 1) . Then f h as a minimum at

t - A/(l X) ,) The funmction g(A) A" (I X)~A is an increas-

ing function of A and g(y) < 2.

2.3 Corollary: Let S be a given channel. Then for

0 A < 1 one has, for each positive n, (1 - A) log N(S~ & A)

tiC -A log A. log( I +), If 0 < A -c 1/2 then

(1-A) I.,gN(S,A)nC A log A-(I-A) log (1-AX),

Jk

20

C1HPTER III

CONNECTED A-CODES

Although for a given channel, S, the value of N(S, A)

may be known for each OcX< 1, the actual construction of a

A-code of length N(S, A) may be quite difficult. In addi-

tion, for practical reasons, it may be desirable to place

various restrictions upon the entities of the code. Such

restrictions may, of course, preclude the possibility of

attaining a code of maximal length. In this chapter, a

A-code with a specified restriction is defined and analyzed.

In the operation of a channel, one of the functions of

the receiver is to scan through the sets Di of a given A-N

code {xi, Dli.l and determine which of-these, if any, contains

the received signal. This determination may be quite difficult,

the degree of the difficulty depending on the nature of the

st ( (Recall that the only restriction placed on the

D i's is that they may be members of Y'.)

3.1 Definition: Let U be a topology on Y. A 1-cennnpert. '

A isa A-code (xi, D1), ..., (xn, D n) where Di is a U-

connected set for each il, 2, ..., n.

3.2 Definition: For a given channel and a fixed real

number O<X <1, let N(S, A, U) denote the supremum of the

21

non-empty set of integers N such that S admits a U-connected

X-code of length N.

It is clear that N(S, A, U) . N(S, 1) for each 0 - A < 1

and any topology U. It is also clear that equality need not

hold unless some restrictions are imposed upon the channel.

It appears intuitively clear that these restrictions should

be placed on the set of probability measures {u(.Ix)lxX) and

must include restrictions upon the supports of these Mes"ures.

Although it would be desirable to let X and Y be arbitrary

sets and U be any topology on Y,. such generality leads to com-

plicated and unwiedly analysis and yields very few results. In

this chapter some restrictions will be placed upon the input

space and the output space as well as the set of probability

measures, These restrictions will allow for almost all of the

practical physical situations one might expect to encounter.

The following example indicates some of the restrictions

which must be placed upon this set of probability measures.

3.3 Example: Let X (0, 1); let Y = (O, 13; let Y' be

the Borel sets on (0, 1); and let y be Lebesgue measure. The

measures 1(.Il) and u(.10) are defined as follows: For BeY'

define

(Bo) - C-4x+2)dy + I (4x-2)dy.B1 B2

1II 2

22

u (BI1) " f 4xdy * f (-4x.4)dy,B B1 2

where B ( 10, 1/21 and B2 1 B/' (12, 13.

IAt U be the usual topology for [0, 11. Since X contains

only 2 elements N(S. A) - 2 for all 0 < A g 1/2. Observe that

,([0, 1/4] U (3/4, 1o10) -3/4 tj([l/4, 1/4]11]. ence

N(SO 1/4) - 2. It is easy to see that if I is any interval0

contained in Y such that IA(I 10) ! 3/4 then y(Io) ' 1/2; also if

1 is my interval contained in Y such that U(I1 11) . 3/4 then

¥(I ) -! 1/2. It follows that N,(S, 1/4, U) -LI

In the example above, one is able to obtain a longer non-

connected code becase of the nature of the Radon-Nikodym

derivative of u(.I01 with respect to Lebesgue Measure. It will

be observed that if one chooses 0 < X < 1 and censiders any con-

nected set A such that u(AlO) a 1-), then there exists a set B

such that u(B10) - 1-X and y(B) < y(A).

Although it was not difficult to show that N(S, 1/4) - 2 while

N(S6 1/4, U) - I, it is easy to see that this problem could rapidly

become difficult as the number of points in X is increased. Given

xcX, let f(ylx) - du(.Ix)/dy. The relationship between N(S, X) and

N(S, A, U) for arbitrary x is very difficult, in fact, almost

impossible, to determine when X is infinite, umless one requires

23

some type of uniformity among these functions. Such a restric-

tion, to be rigorously defined later, will be placed upon the

channels investigated in this chapter.

The following auxiliary results will be needed later.

Throughout the remainder of this chapter R will denote the real

numbers. 8 will be the Borel sets, y will be Lebesgue masure

and U will be the usual topolog for the reals. If Y is a

subset of R then the a-algebra Y' will be the Borel sets on Y.

3.4 Theoren: Let u be a totally finite measure of (R, 8).

Let * be any measure defined an (R, 8). Let a a u(R), let

0 < A be given, and let A - (AM: u(A) > -I). ThenA con-

tains a member of minimal *-measure.

Proof:

Let u - U1 in2 where u <<* and u2i*. Let

Ao - spt U26 If U(Ao ) ! a-A the theorem is trivially true.

Suppose not. Let 8 - a-A-(A). Let f(y) - du 1/do. Let01

z " {y: f(y) ! z}. Observe z1 C z2 implies EzI. E Z. Note

that Ez is a *-measurable sot for each zcR. Let I(z) a f f(y)d*.E E

Then I(z) is non-negative, monotone non-increasing and left-

conti uous. Let z0 inf (z: I(z) ) B. Cbserve that

I(z 0 ) > B z > 0.

24

implies *ChCn ' *(By.d(I ycC * yc,

Now if BC E and CC EC then fBy)d* f(yld*

implies *03 :, *01. [If ycC then f(y) -c zo0 while for ycB,

f(y 1 zo]. Hence if I(z 0) then E1 U Ao is clearly a set

of minimal *-measure such that p(EzoL Ad! . aiA. Suppose0

I(z ) % 0. Lot E' - {y: fy) > zo ) and E" - (y: f(y) a Z00 0

If u(E') a B then AoU El is the required set. If u(E') < B then

p(El") V 0 - uCE t l > o. Lot El" be a subset of E" of t-measure

(B . (E')1/z. 7hen u(E'U El') - 0 and E'IUE" has minimal

0-masure. Hence A a A 0oUE'L)E' is a set such that, u(A) Z a-A

and such that *CB) Z *(A) for any set B such ti(B) !-A.

3.5 Remark: Lt A be any set of minimal #-measure such

that i(Al - a-A. Let B - spt u2 and let yo - inf (yb4,C- , y)

. a-A - 2 (B)}. Let f(y) a dui/d*. If f(yl) f(Y 2 for almost

all 1] yl < Yo < Y2 then A can be chosen to be.A - BU(--, yo).

The proof of the following two Corollaries cones immediately

from the proof of theorem 3.4.

3.6 Corollary,: Let f be a non-negative real-valued Lebesgue

measurable function. For BeS and 0 .5 < f f(ydy there exists aB

set ACB of minimal y-measure such that I fdy ! a.A

3.7 Corollary: Let u be a totally finite measure defined

on (R. B). Let a = u(R). If i. << y, then there exists AMB such

that uCA) a c-A and y(B) . y(A) for any BeB such that y(B) Z a-A.

2S

In several of the theorem developed in this chapter, it

will be hypothesized that u(.Ix) 4 y for all xcX. The follo-

ing theorem gives a partial justification for such requiremnts.

3.8 Theorem: Lot v be a totally finite measure on CR, 81.

Suppose that fc: each 0 ,1 A . p1CR) there exists A eS of minimal

-mas.ure such that u(AX) ! 1(RI-X and AX is connected. Let

* W 1V 2 wiere 11 y and jr. Then there exists a support

of 1A2 which contains at most one point (lspt 121 . 1).

Proof:

Suppose Ispt U2I . 2. Lot A . (AA - apt u. and

yCA) a o). Then given AMA, A is not connected. Let X - p(R) -

12 CR1 and let AX be any set of minimal y-measure such that

u(.Ak a p(R) - . (bserve that, given AA, u(A1 a p(R) - A.

Honco y(A.) = 0. Therefore u(A.) a U2(Aj). It follows that

AxcAs hence is not connected.

3,9 Loma: Let f: R - R be a.e. continuou and non-

negative such that 0 ( . fdy - a -, Suppose there existsR

X X 2 x3 such that f is continuous at xi i-la 2, 3. and

f(x1) > f((2 4 f(x 3),. Then there exists A > o such that, if

A is any set of minimal y-masure such that I fdy Z .- A A IsA

not connected.

" • ..u un ! I I II •

26

Proof.:

Let E ux: f(xI : 1/2 * m+ minMxzl. fCx3D). Let

B * f(xldY8E

Le t E EE (x: x z1

E E /(x: x }

and

B - *J f(dy 1 1. 2.

Since f is continuous at x1 , x2 and x3 , it is easy to see

that B1 0 o j0 B2 and E is not connected.

Let . > obe chosen such that 1 B 2 . a-). >max (B1, B2).

Now XcE inplies f(x) ! 1/2 f (x 2 ., min (f(x,. f(xj).) and

x;Ei imlies that f(x) < 1/2(fCx 2) min (f(x 1 ., f(x 3 )11. Thus,

if A is any sot of mir-mal y-measure such that I f(xldy %-A'A

then y(AAEcl 0. moreover, 1 o, 2> O, and e-A > max

(B1 821 implies AflE 1 # 0, AAE 2 0 0. Hence A is not connected.

With the aid of Lemma 3.9, one can characterize those a.e.

continuous sumable functions f: R - R such that given

o < ) I fdy thore exists a set A~, of minimal y-msasure forR

which f fdy . f fdy - A. which is connected.A), R

3.10 Definition: A bell function is any function f: R * R

such that there exists xo such that f is monotone nondecreasing

for x • xo and f is monotone nonincreasing for x > xo.

0I I

27

3.11 Theorem: Let f be a non-negative aee. continuo s

real-valued function defined an R such that o < I fdy * a -.R

Given o < X a let B be any set of minimal y-measure such that

• fdy .-. Then there exists for every A a connected set A

such that f fdy w J fdy and y(A) + Y(B) iff f is a bell functionA R

Proof:

Cbserve that if f is not a bell function a.e. then

there exists points of continuity x1 < x2 < x 3 such that f(xl)

> f(x 2 ) < f(x 3). Thus, the necessary part follows immediately

from Lemma 3.9. The sufficient part will be proved by con-

structing the set A. The construction is similiar to that used

in 3.4.

Assume f is a bell fmction. Let E = {xlf(xl ! y}. LetY

I(yj * f f(x)dy. Let yo a inf (y: I(y) . c-X). Then I(y o) 0

ya-X. Let E' - (x: f(xj > y} and E" - {x: f(xj a y0). NewYO YO

it is clear that if A1 and A2 are y-measurable sets such that

AICE~ and A2CEc and o < y(A 2 ) . y(Al) then f fdy > f fdy.YO YO AI A 2

Thus if y(E") o the proof. is completed. Suppose y(E" o.

Then. E" alr b 1 Ua b and E' - (bl, a whereYo Laz' La 2 2 1 a2)

0 0.

b I a 2' Clearly one can choose a c [a,, bl], b £ (a2 . b21

28

such that l-a * b-a 2 ]y o o 0-1 fdy. No A. * [a, b] is they'o

required interval.

The preceding results will be used in the analysis of an

important special channel which will now be introduced.

Let S be a given channel. If anytime a value x is trans-

itted, the receiver i* able to determine from the received

symbol that x was the point set, then the .hannel is called

noiseless. Such channels rarely occur in practice and are cf

little mathematical interest. On the other hand, if there is a

definite positive probability that the receiver's decision will

be wrong, then the channel is called noisy. An important type

of noisey channel is the channel with additive noise.

3.12 Definition: Let S a {X, u(.Ix). (Y, Y'11 be a given

channel. S has additive noise if there exists a probability

measure u on (Y, Y') such that U(AIx) - u(A-x) for each xcX,

AcY'. The measure u is called the noise.

3.13 Remark: In order to assure that the operations indi-

cated in 3.12 are well defined, it will be assumed throughout

the remainder of this dissertation that if S is a channel with

additive noise then (Y, +) is a group.

3.14 Theorem: Let S be a channel with additive noise u.

If u < y, then u(. Ix) << y for each xcX and f(ylx) = f(y-x)

29

where f(y) - du/dy and f(ylx) du(.Ix)/dy.

Proof:

By the Radon-Nikoclym theorem

u(AIx) - •(Arx) f -(Y)dY fCy-x) d.

A-x A

Hence, by the absolute continuity of the integral, (AIx) << y.

Also, by the Radon-Nikodym theorem

u(Aix) I " f(yfx)dy.A

Therefore, f(ylx) a f(y-x) for almost all [y] ycY.

Now it is clear that to determine a necessary and sufficient

condition for N(S, A, UJ a N(S, A), where S is a channel with

additive noise, one need only investigate a single measure, the

noise u. The following concept will play an important role in

this investigation.

3.1S Definition: An interval (a, b] is left adjusted in

an interval (C, d) iff a a c and b . d. A sequence of disjoint

intervals ((ai, bi]1 i is left adjusted in an interval Cc, dl

iff (a,, bl] is left adjusted in Cc, d) and (a., bI] is left ad-

justed in (bil, d) for i - 2, 3, .... n.

3.16 Definition: Let u be a meaure defined on (R, 8).

Let 0 1 A < 1. An interval (a, b] is Co, A)-minimal left ad-

juted in (c, d) iff (a, b] is minimal left adjusted in (c, d),

U(a, b) . and if m(c, b) .I A theon b I . b. Lt fuini, b e

.30

measures defined on (R, 8). A sequence (Cat. bil)i is

((sAi)'n.i. x)-minimal left adjusted in (c, d) I1ff (&I# b13 is

Cia1, XI-mininal left adjusted in (c.* d) and (ai, bi) is Clpp X)-

minimal left adjusted in (bi_1 , dl for 1 2, 3, .... n.

3.17 Leum&: Lot Y a Cc, d). If 1A is a totally finite

inasure an CYO Y%) then for 0 c X 1 isCY) there exists an inter-

va1 (a, bl which is (ui, A).-miniuaal loft adjusted in Y.

Lot A a (b: 1a(c, bl ! A). A jI 0 since X !~ iCY). Let

b n-lb bA.Lo b b a sequence-contained in A

which converges to b 0 It =ay be assumed with no loss of goeri-

ality that (b~ )n,18 is a decreasing sequence. Thn(c5b~nl

is a decreasing sequence of intervals such that li~c. bn < for

all a and Cc, b0 3 (Cc. bile Therefore js(c. bo]

11. X(cOb] 1,A Now it is clear by the nature of bthat

Cc, b 3is (u X)-.inimal left adjusted.0

3.18 Theorem: Let S *(X, i(.IxJ, (Y, Y')Jb. a channel

with Y connected. Lot 0 ! A < 1 be given, Let (xl, D1)* ,

(xni D ) be a connected A-cod. of length N(S, A, U). Then there

exists a CC. Ixln, l-XI-inimal left adjusted sequence.

Proof.

It may be assumd that the sequence (D. is ordered;IL 3,01

31

i.e. if one denotes (a . bil then i implies a - a

By lema 3.17, there exists a (C.(x I l-a -minimal left ad-

justed interval in Y. Call it D1. Since D i 1t.xj), 1-A)-.

minimal left adjusted, it is clear D2 CY-D ; hence,

'A(Y-D] 1 x2 ) ! -A. Thus, again by 1emma 3.17, ther exists and

interval D2 which is C(C. x2), -A)-minimal left adjusted in

Y-Dj. It is clear that proceeding thusly one obtains the desizd

sequnce {D) . l .

3.19 Remark: In lemma 3.17 if m -c< then *(c. b. I A;

hence in theorem 3.18, if u(.Ix) c" Y for each axs (Di[L x)-

1-A for i a 1, 2, ._8 n.

3.20 Definition: Let S - {X, i(,.x)a (Y, Y'T} be a channel

with additive noise m which is absolutely continuus with respect

to y. If Y is the real numbers, Y' the Borel sets, and X a

closed interval contained in Y, then S is called a channel of

tye A.

Throughout the remainder of the Chapter the emphasis will

be on channels of type A. In most of the analysis it will also

be assumed that f - du/dy is a bell function. Tehniques for

actually constructing connected A-codes of length (S, A, U) will

now be presented. The first such construction is for an arbi-

trary channel of type A. Since w((y)) * 0 the sets will be

written as open intervals.

i i I I I Ii P

32

3.21 Construction: Lt S be a channel of Type A. LOt

0 1 Obe given. Lot X a [a, b]. Lt dI be such that

,Cco d1J is ({(.is)I, I-) minimal left adjusted. The remainder

of the code is constructed inductively as follows.

Suppose mebers xi s DiiN. have been obtained. Lot

CdN.I & d.. . If ((Cd. d) [b) < l-X the construction is com-

plated; otherwise$ dN ! is obtained as follows. Lot

. inf (d* - dNL there exists x c(a, b] such that ,CdK. d*) Lx)

- 1-).) Clearly ((dN. dN * L- c)!x) 1 -). for all c 2 o and

all xcX. It will be shown that there exists xcX such that

u((dN. dN * £tx . 1- . The n~lst element of the code will be

chosen to be (xCdN, * 1. If-there is more than one x

such that u((d , dN * !x) 1-k then any such x mW be chosen.

Lot F(y be the distribution function of v. (( y iqPlies

F(y) is continuous. Cbsorve that for any xcX F(y) F(y-x)

whore F(y) is the distribution function of P(.Ixl. Also if

xcX# k 0 then u(C(d. dN * k)Ix) - F(d - x+k - -x).

Lot tn t L then for each n there exists xn such that

boundN xnd + e )a - lidN -poiJ ! -X n l C X hence isbounded and therefore has a limit point& say x'. Lot

~33

(n)nk- x'. Mhen F( dN -t +' t) R;- d x'l

HEm (FCdN x% , Fd N - This provesn k k kk

the induction step. It is clcar that the code constructed above

has length N(S. X, U).

In the preceding construction the specific nature of the

code is not readily appaent. In the case where f is a bell

function one can construct the code so that it is more trans-

parent, and, hence easier to manipulate.

3.22 Construction: Let S be a channel of type A. Let

f - du/dY be a bell function and let 0.5 A < 1 be given. Let

X - [a, b] andY a Cc. d) where both cn-- and d s - are

allowed.

Since f is a bell fumction, there exists, by theorem 3.11,

a connected set (tl, t2 of minimal y-measure such that

U(t 1 , t 2 ) 1-4. The nusbers t 1 and t will be used to con-

struct a A-code of length N(S, A, U).

Let (c, d1 be the (u(.1a), 1-AX-minimal left adjusted

interval in (c, d). This interval exists by lema 3.17. The

remainder of the X-code is constructed as follows. Let the ithj

pair be denoted by (xi, (di.1 di). The i~lst pair, if it exists,

I_ _ _ _ _ _

34

is constructed by

Case..I: If di • tI let (di, di I) be the (W{. 14. 1-A)-

minimal left adjusted interval in (d.8 d) and let xi+ 1 - a.

Case 2: If a + tl di b + t1I lot di+ 1 and x i+1 •d i - l*.

By definition cf t1 and t2, it is clear that (di, di1,) is

CU. l-)d-minimal left adjusted in (di . dl.

Case.3? If di -b + t1 and v(di, d)[b) < 14- the construction

is completed; otherwise (di Z b * t1 and u((di, d)Jb) ! 1-X)

let Cdi, di) be the (u(.bJ, 1-)-minimal left adjusted inter-

val in (d,, d) and let xi ,.- b.

The code constructed above will be labeled by ((xi, D ) 1 "

N N

It is clear that (D. ) is (NU(.1x 1-XI-minimal left

adjusted. in Y and that if B, i N is any set of pairs

such that (B,~ is ( &il'N 1-XI-minimal left adjusted

in Y then bN.! dN where Bi a (ai, bi]. Thus it follows from

theorem 3.18 that N - N(S, A, U1. Any member of the code con-

structed above of the form (a, Di) will be called an a-pair;

any member of the form (b, Di) will be called a b-pair.

It is clear that the code constructed above is not, in

general, the only connected code of length N(S, I, U1. For

reference later this code will be referred to as a connected

code of tye 1.

35

Fra the cons tructions outlined above it is clear that ai ~ore precise notation ffor the code is lxi(Ajo Di(Xj) , )

However when there is no possibility of confusion the A will be

supressed.

3.23 Definition: let S ( (X, u(.lxl, (Y, Y'} be a

Nchannel of type A. Let 0 : X - 1 and let (xi ,D DI bea1 1Jal

connected X-code of type 1. This code will called full if

Nu((Y - () Di) * 0 for xcX. The term full code will always

refer to a connected code of type 1 which is full.

It is easy to see that givenany N > N(S, 0, )) there exists

XN such that N(S, AN. U) NO and the A -ode is full iff given

X -XN then N(SX NO U) > N(So AO ().

3.24 Remark: Let S - {[a, b], u(. x), CY, Y')) be a

channel of type A with additive noise u. Let 0 .1 A < 1 be

given. Then there exists a channel S1, more precisely an input

alphabet [a, b1 ]. such that the A-code for S is full. In order

to verify this statement it suffices to demonstrate such an

alphabet. ro remove any possibility of ambiguity let the family

of measures (u(. Ix) IxcR} be defined by u(Alx) a u(A-x. for all

xcR, AtY'. Since u '< Y there exists an interval I a (tlO t2)

such that I(I) u 1-A and given any interval J withK(J) < t2 - 1

. ..~ ..... .... H

36

then, u(J) < I-A. It is easy to see that there exists y > a

such that U(AJxJ < 1-A for all x Z yls AC(- -, a + t2)

similarly there exists Y2 < b such that o(Aix) < 1-4 for all:

x .1 y 2 6 AC(b + tl, I. Let (a', b'1 be defined by a' * a,

b' - yl + (b - y2 ) + 2(t2 - tl. and let S' be the channel

Nwith alphabet Ea', b1. Let (xi, D i I. be the type 1 A-code

for S'. It is easy to see that if DiC(a + tl, b + t2l then

Let W, DiN) 1 be the code constructed(Di - t2 - t .l i iN

analogously to (xis Di}Ni- where one uses right adjusted se-

quences. Let a - sup (ylycDi# Di1 C ( - , a + tl) 00); let

a - inf (yJycD, D 0 (b + t2,-) #0}, Given . b"let B(Q)

be the point, considering the channel with input (a, [], ana-

logous to B. Then given 2 b' then (

C2 - &16 Thus one can clearly choose b* .> b' such that

0(b) - a is an integral multiple of (t 2 - tl). The type 1

1-code for the channel with input alphabet [a, b*J is full.

In 3,21 it is clear that for a given 0 f A < l and

1 - i _ N(S1 A, U1 there may exist several xcX such that

u(Dilx) Z 1-1, 'We define X (xlxcX u(DiLx) ! 1-A).

3.2S Theorem: Let S be a channel of type A. Suppose

N(S, A, U1 * N(S, A) for all 0 f X < 1. Let 0 ! X < 1 be given, N(S, A, U sful Te

such that the A-code (xi(AJ. Di(A)1N( Ai l is full. Then

given 1 I # j . N(S, A, U) and A.C Di(A), A.C D.(Al such

II37

that there exists x? X for which m(Aijxrj - m(AIxt. 0 0

then u(A.Jx'). ! (Afxfl for all x A " x

Proof:

Suppose there exists A such that the X-code is full

and there exists AiC D.(X) ' A CDC(k) for some i j such

that there. exists x? cx * x X such that iA(Ajxt).

,a(AItx),3 0 and c(Ajxj, - u(Ajlx,). ,'hn u((Di(XA - Ail

UI A. lix?) a 1-X and i&(D.(XI - A.) U A.) jx > 1-A. Hence

choosing D!(X) a D.(XA) -Ail 1)Aj and D'j1AC (D(XAilA.

A such that ji(Dixt) 1-A one can otain a -code

{ ., B IN(S, As U) such that there exists at least one xgX

N(S4 XI La)such that K(Y - U B.x) > 0. , It follows by the abso-

lute continuity of A, (More precisely by the fact that the and-

point of each Bi is a continuous function of X) that there

exists X < A such that N(S, Xo)>N(S IA o. But thsis a00 0

a contradiction of thehypothesis that N(S, A) = N(5, A, LU) for

all 0 1 Ac 1.

3.26 Corollary: If there exists a full A-code

. . .. , 6 U.

{xi Dii1adfo oe1 i j fNS w1)teei

38

AicDi. AjCDj such that p(AIlx?) 0 0 U(A x?) and

f(yilxj)/fCyi l x) > f(yjIx)/f(yjlxP) for almost all [ry

YitAib yCA. and some xtc 1 * xiXCX then there exists 0 'A

such that N(S, I1) N(S, X1, U).

Proof:

Lot BiC A, B CA such that uCBilxll -

14(B1xp 0. 7hen

J f Tylx Tyt4 l dy u(BlIx)

3.27 Corollary: Suppose f - du/dy is a bell function

and is unbounded. Then there exists a channel S of type A with

additive noise is and 0 < A < 1 such that N(S, Al > N(S, A& U).

Proof:

It follows from the definition rf a bell function

that there exists at most one point. yo. such that T1. f(y) = ".yCYh

{ke may m well assum that ao 0.

39

Observe that it suffices to show that there exists a

channel S a ((a, b]. 1A(.Ix, (Y, Y')) and a full A-code

(z D N, l such that either (1) b1 < a < x2 or (2) xN. ,- b <

where Di ( &ai& bij. In the first case f(yi)/f(yIx2) is

unbounded on D2 while this function is bounded on a sbset of

DI of positive 4(.Ix2 ) measure. Similarly, in the second case,

f(ylb)/f(ylxN. I ) is unbounded on DN.I while this function is

bounded on a subset of DN of positive u(. IxN u

If either u(--, 01 - 0 or V(0, -) a 0 then, for a given

channel S, more precisely for a given input alphabet (a, bJ. it

is clear that one can choose 0 < A < 1/2 such that the A-code

is full. Thus either (1) or (2) must hold.

Suppose (-i-, 0) I 0 u(O, -), One may assume with no

loss of generality that p(- , 0) > ,Z(O, O). bserve that as

1-A increases tI(A) decreases. Thus there exists I-A iC-", 0)

such that tl(A. c a1. By 3,24 there exists a charuel S such

that the A-code is full. Since tY(A) < a1 < a (1) must hold.

3.28 Remark: Sufficient conditions for N(S, A) = N(S, A ) L

will now be shown. Throujgout the remainder of this chapter it

will be assumed that f is bounded and f(O) > f(y for all ycY.

The importance of the behavior of the ratio f(y./f(y-d) was

partially shown in 3.26 and 3.27 in the form of necessaxy

w

lilll I I III I |

40

conditions. The following theorem, which continues this

investigation, will be used to derive a sufficient condition

for N(S, X) a N(S, , U).

3.29 Definition: Given sets B1 and B2 then B1 is less

than B2 , B1 ' B2, if given any ylcB1 , y2cB2 then yl < Y2'

3.30 Theorem: Let xl, x2cX with x2 X*.do d > 0. Let

0 < A < 1. If f y)/f(y-d) is a decreasing function of y then

given any A,, A2 with A, M A2 * 0 and 1(Aix ) . 1-A, i - 1, 2

the:, exists a, B2C A1 UA 2 with B1 < B2 and u(Bix.) > 1-,

i * 1, 2.

Proof:

Observe that fCy)/f(y-d) * dIA(.x I/di(.lxp. Let

- inf (ybl((A1 U A2 ) C) (--,Y)jx 1 ) Z 1-A}. Let

B 1 (AU A2 ) f(--, 9). Then, since P < y, u(B 1 x1 • I- A.

By 3.S 8I is a set of minimal U(. Ix2I measure suh that

uBlix 1 -I 1-A. Lot B2 n (A1U A2). U(Bl1 x2) + u((B2 1x2)= 1A(AIx 2) + w(A2Ix2), Hence, since u(BlIX 2) .< (A lX 2),

ucB 21x 2) !> U(A2lx 2) ->-.

3,31 Lemma: Let 0 -5 A < 1. Suppose that given any

x1, x1,X with x < x2 such that there exists Al. A2 with

A2 0 and u(A I x1) ? I-A, i-1, 2 then there exists B1D

B2C A1 U A2 with B1 < B2 , Bl) B2 u 0, and U.BBlxil I 1-A.

i-i, 2. Then N(S. A) N(S, AD U1.

I.j

41

Proof:

Let (Xij Ai) N - be a A-code of length- N(S. ). It

will be assumed that the indexing is such that i -J implies

x.i xi. By hypothesis there exists a code (xi, A[}I with

Al U C A, LJAN and All < A. By the same argumnt there is

a code Ux. At)hilwit All 1, end A" < A; It is easy to

see that repetition of this logic proves the existance of a

-code (Xi, N with B < S <.B for 3 :, i 1 N. It is now

clear that there exists a A-code (xitl N 1 C. when-

ever i < J.

Let ai - inf (ylycCI; b. * sup {ylycCi}. Let D, - (a 1 , bi).

Then u,(D it xi u(Cilxi) 1 -X and Di.OD = 0 if i. J. Thus,

(xi Di}N is a connected A-code of length N(S, Al.

The following theorem is an immediate consequence of the

two lemmas.

3.32 Theorem: Let S be a channel of type A with

f = du/dy a bell fumction. If f is log-concave then N(S, ) A

N(S, A6, U) for all 0 $A c 1. In particular, N(S, A) = N(S A, U),

for all 0 S A < I if S has additive Gaussian noise.

42 IIt will no be shown that there eists daaa of type A

with f a du/dy a bell function for which there eats a 0 c Isuch that N(S, Al > N(S, A, U).,

3.33 Definition; Let F be the set of all AmnCtons ofthe following kind, Given 0 (< l I ( t l/90 define

fCY) • t 'if y tC-s/2, s/21

u~)ai if y cC-1/26 -s/2)U Lis/2j 1/2).

a0 elsewhere.

U

Figure I, A Typical Menber of F

3t34 Reaark. Given feF then £ induces a probailityWmasurea Ifo on the Borel sets in a natural way, i.e.

wf(A).a ft ).dyA

for all AE6,

3,35 Definition Given t > 0 and icF let S(fa t) denote

' 43

any channel ((a, b]. u(. jx) (Y. Y'1) with aditive noise i a

and b-a a

It will now be shown that for certain members of the

fuily (S(f. L) fF. 0) N(S, X) a N(S, 1, (1) for all

0 5 X < I while for other mabers of the family equality does

not always hold.

3.36 Remark: Let S be a channel of type A. Given ycY

define f*(yj " (sup f(ylx)ixcX). Let F(y) - f(yldy. ThenY

Ngiven 0 - l c I and any X-code (x. Ai 1 then

I4(ailxi) - F(Y.

Loma: Given t > 0, fcF, N(S(f, t), 1) N(SCf , tl,

A, U) f .. k I-st.

Proof:

Let X1 " 1-st such that X I-code is full. Let

N

{Nip Di} H be a full code. Then a(Dijxi - F(Y).

Hence N(S(f, t), 1 1) .- N(S(f, t), A1, LI.. Suppose there exists

X2 < A 1 such that N(S(f, ti, A2) N(S(f, t), Xl) a N. Let

1i

44

NN

(cis.Ili 1, 1 be a A 2 code. ThM u I(&LC1) j N(.1-A1) N(I.l..1)i-I ,

a F(Y). Which is a contradiction. Uence A c A implies

N(S(f, t), A) < N(S(f; t) , At) This completes the proof.

N3.38 Remark; Let (xif Al.)NI be a full A-code with

A c-st. Let k be the number of a-pairs. Lot A£ ia (.ja b18

Cbserve that b.- a. s, Suppose xk 1 0 h. Then X A is

an interval, say [a,' 02 ]. In fact, see Figure 2, a1 u h7 * s/2

1-sand a2 - a, a (l-A-stl/uo If bk - La - s/21< there exists

an interval (8, a-s/2) such that f(y aL) u for all

y c(8, a-s/2., see Figure 3, Thus under these conditions it is

Aeasy to see, Figure 4, that one can choose xkilskl such that

there exists y1Ak, Y2rAk., such that fCyla) - u - fO'l1 qx, 1 )

while f(Y2 a) a t and f(Y2 !XkQl' a uo It follows from 3,25

that, uider the conditions specified above, there exists

o X. <. I such that N(S(f, t) , A) > N(S(f, j IA' A, U1.

b -(a- t' fy!a)

I LI

k '

a a - s a b Q I2 bk+l2 2 k

Figure 2. Definition of the Interval (ak It'2

4S

f (y a) f

Fg*,: D ,

S I

f o. (ya, ( j 11

r, - _,_l~ ~ ------1t.. . . . ..---

II *I

a bSII ,I

kI ," 'k l 12 k+I

Figure 3: Definition of dhe Interval (B. a-s/2)

f(yf!a) )' . -

* II

.. . I '/B b k d! Xk 1 bkl

Figure 4: A Choice for fyx l

I _ _I _ _I _ I I I

46

3.39 Lama: If st' - 1/3 there, exists 0 4 l- such that

*N(S(f, t),a Al > N(S(fe t), X8 U).K

Proof:

If b-a < s let A be such that N(S(f, t). A, to - 3

and the -code (xi, Ai)N. 1 is full. Let A. (a , bij. Observe

that bt a # S/2. One can assume that x2 (a * b)/2. Nows

b, -(a-s/2) < s <-- Thus, for this case, the conclusion

follows from 3.38.

Suppose b - a Z s. Let A0 max {AlA < 1-st and the X-code

is full). Let {xi, Ai}N*1 be a full A.ocode where Ai w (ai. bi)

and let k be the nunber of a-pairs. Then either (1) bk < a * s/2,

or (2) bk > a s/2, or (3) bk a s/2. Observe bk a + s/2

implies k Z 2 since bk a + s/2, k 1, and s < 1/3 implies

1-sU2 Z a- s/2 +s. +.- *s > a- s/2 + 3s; but, this indicates

that b2 - (a - s/2) > 3s which says that there exists

X < A I - st such that N(S. A , U) ! N(S. Ao, U) + I which

contradicts the definition of Xo"

1-sIf (1) is true then bk - (a - s/2) < s < and the con-

clusion follows from 3.38. If (2) is true then there exists

c(a, b) such that CcX]k. Now, replacing (xk, Ak) by (F, Ak),

47

one obtains a code with k-i a-pairs and bk. I a - s/Z. Again

the conclusion follows by 3.38. If (3) is true let

" " (11A 4 A and the A-code is full). Leot (ti BL) lbe

a f tll Yc-¢ode and lot n be the number of a-pairs. Let

All (ai, bl. Then either (1') b' 4 a + s/2 or (2')

bn a + s/2. The conclusion follows by the sam arguments

used above.

3o40 Theorem: NCS(t, t), A) a N(S(f, t), A, U) for all

0 1 A c 1 ff given A < I - st such that the A-code is full thent 1

either N(S(i- t), A, U) v 2 or A 6 1 + st -t-

.Proof:t 1

The condition A -< 1 * st - is equivalent to

1 A Z t +~ I s* -st, 1-s

To shou that the condition is n( -essary assume N(S(f, t).

A, U). N(S(f, t), A) for all 0 A c 1. By 3.39 s t 1/3.

Let A. vx (XIA A 1 -st and the X-code is full). Leot

{x. DN} *1 be a full A -code and let D (a2 , bi1 . Suppose

N(S(f, , ) '0' U) > 2. Since 1 - Ao st Z 1/3 Z

WS(- a - sf2)1a) I a - s/2. Ifb, < a s/2 then

i-A0 ~ b I I~ - " -t - (b 1 -5 lt which, by

3,38 is greater then or equal ~ t t. If bf &s/

MUM_

4$

it is easy to see that bi ai 1 s for 1 $ i S N. Thus

a+ hence b (a - s12) ; 2s. It

follows by the- Yximality of A 0 that < s. Hence,

u; -s u(( -. D~ > - y, a + s/211&) " st >+

( t. Thus the condition is necessary.

To show that the condition is sufficient lot 0 < A S 1

t 1st- T- Let xI <x 2 and let At A2 cY', A,/) A2 . If

> )s and x2 - x < -r then u(AXI) * i(A2x 2) < 1- st

1-5* st < 2(1-Aj. Thus if u(AI xi - 1-A, i - 1. 2 then

) 1-s1-Seither (1) x2 - x,.-~ or (2) x-

Case 1: x- x < s It is easy to see that

x + s/2 > x s/2. Lt f*(y') max (fCyx 1). f(yx 2 )) and

lot u*(A) * f f*(yldy for all acY'. Observe that givenA

X2 - s/2 < y < xI + s/2 then ",*(A1 (JA 2 ) - u((AUA2)/(-", y)1x 1 )

m u((AIUA 2)1(y, -)jx2j.. Let - inf (yIu((AlUA2)m--, Y) [xi)

1-A. Let B1 - (A1UA 2))(--, &). Then u(Blxll a 1-X. Let

B2 ' (A1UA2)nI(C, -n). Observe that u*C(AUA2)1t(--. x 2 - s/2))

< 1-A, and u*((AIUA2)nxl1 + s/2)1 < 1-k. If follows, since

M'(AI U A21 !- 2(1-A), that x2 - s/2 < , < x1 + s2 . Hence,

2(I-A) - m*(A UA21 u ( 1IxI) " u(B2 k2 ). Therefore,

49

Case- 2 x x - Then f(ylx 1)/f(YlX 21 is a decreasing

function of y. Thusa by 3.30, there exists B1 c B C A, U A2

such that u(Bi'x i ) : 1-A, i - 1, 2.

If follows from 3'31 that N(S(fj L), Z) U) = N(S(f, t), )

t 1for all A -t 1 + '

Suppose N(S~f, 1), A, L 2 and the X-code (xi. Ai)?j1

Suppose NSlb. t . $

'is full then xi a a, x2 -b and A1 a (a- -1, T - A

then (Alla) + m(A2 1bl *I (b-a)t a F(y). If follows from

3.36 that N(S(f, L), ) - N(S(f, t), I, U). Ifb - a s s then

b~a > a s/2. Benco if A is any set of minimal y-measure such

Ithat i(A) 1-X then u(A) - (1-A-st)/u - .b+a+ll. Hence

N(Sf, A), A) " 2.

By lema 3.37 N(S(f, 1., X) - N(S(f, Z-,As U) for all

A 1-st. This completes the proof.

3.41 Corollary: Given Y > 0 there exists fcF such that

N(S(f, £), A) > N(S(f, LZ, X, U) for some 0 c X c I.

Proof:

Choose feF such that st < 1/3.

3.42 Corollary: Given fcF with st > 1/3 there exists

> 0 such that N(S(f, t), A) - N(S(f, A) A, U) for all 0 1 A < I.

" "" "" " " " ' " " ' ", " , ,_ _ _ _ _ I { { [ [

Proof:

Chooset- s- LZ38 p At a aas bt. - b - str

Then W((A a) I, ~a) a IA((O , b+ j' 1b S t.. B-a - so Hence,

given A < 1 -st such that the A-code is full then

N(S(f, t), A, U) *2.t+l

3.43 Corollary: Given fcF with 1/3 < s < p there exists

t > 0 such that N(S(f, l Al), • N(S(f, t), A, U) for some

0 l A < I and therc exis" Z > 0 such that N(SCf, Z21. A) IN(S(f, t2), A, U) for all 0 < A < 1.

Proof:

The second conclusion is an immediate consequence of

Corollary 3.42. To show the first conclusion observe thattl t 1

s '-- implies that 1 - st I + st -. o . N givent 1

st A, A 1. st - t- one can choose t > 0 such that the

A-code is full and N(S(f, t), A, U) > 2.

3.44 Corollary: N(S(f, t), A) - N(S(f, C) A, U) for

all t > 0 and all 0 < A < 1 iff S ! t l4t

Proof:

The necessity of the condicion is proved in 3.43.

Suppose s. s . Then 1- st - 1 *st-- - t4t~ 2 2'

$1

3.4S Conjecture: It has been shown that if f is a bell

function one may still have N(S, 1) > N(S, X, U) for some

0 5 A 1. The author has been unable to formulate a proof that

if N(S, A) a N(S, X, U1 for all 0 1 A < 1 then f must be a bell

function. However, with the aid of theorem 3.25 many non-bell

functions have been investigated and in all cases it has been

possible to find a X such that N(S, A) > N(S, X, U1. This, along

with intuitive feeling, has led to the conjecture that N(S, A) =

N(S, X, U) for &1: 0 - A -c I implies f is a bell function.

Examples are listed below of channels of type A where f is not

a bell function. It will be observed that in each example a

family of functions (hence, a family of channelsl is defined,

and, in each example, given c > o there exists a meber f of the 4

family and a boll function g such that If(yI - g(y). < c for all

ycY. Moreover, N(S(g, A), A) * N(S(,g, t-, A, U1 for all

0, X <1.

3.46 Example: Given 0 < B < 1, 0 < 6 < 1/2. Let

f(y) - 1-8 for y c(- 6/2, 6/2)

. 1 for y L(- 1/2, - 6/2) U (6/2, 1/2)1-6

= 0 otherwise

Let X n [0# 1/21le t A - I- uf(- 1/2, 1/4). Then

N(S, X, U) • 2 and tne A-code is full. Observe that f(yj1/2)

> f(ylo) for all y c(0, 6/2) and f(yjo) > f(yJl/2) for all

l I I I I I I I I I I I IM I I I "1j

y c(1/2 - 6/2, 1/2). Hence, by 3.2S there exists 0 . A I

such that N(S, A) > N(5, X, U).

Define g(y) w I for y .(- 1/2, 1/2)

0 0 otherwise

1hen, for 0 8 B c , f(y) - g(y)[ < c. It is clear that

N(SCg, ) ), .X, U) for all 0 !-. 1.

6 0 6

7 T

Figure S. The First Counter Example.

3.47 Example: Given fcF with s > (p1-). Define Mf to

be the farnily of a!1 functien defined by: Given 0 < m < u( ]

define g(y) T-y + u- Tr(--) for y cC- 1/2 , s2)

1+S*y + u- m(---) for y c(s/2,fy. otherw/2i.

=f(y). otherwise

S3

Given - such that there exists a full X-code

(xi, DiN then there exists I j N such that

a - /2 < y < a * s/2 for all ycD.. Hence there exists

x c). -(a) such that ja -;(! < lence there exists33 2'

0 such that f(y~lx!) > f(ylxl). N c f(ylx 1) (y a x )

for all yD.. Thus by 3.25 N(S, A) > N(S, Al ') for some

0 A (1.

Given c > 0, then for 0 < m < r, If(y) - g(y)j I -. for

all ycY. N(S(f, i) , A) -N(S(f, t) A, U) for all 0 < X I

by 3.44.

I U ,

II

'I U

-0 I

Figure 6. The Second Counter Example.

i j)

I u

MiAPTER I'

ADMISSIBLE WEASUPE

It is reasonably clear that in formulating necessary and

sufficient conditions for N(S, A) -a N(S, X, U) one need not

worry about every measure in the family - {U(.x): xcX}. In

fact for a fixed value of , say X if it is known that there

exists a connected X 0-code, { (xi I Di) of length N(S, A) ,

then N(S. Ao U) = N(S, X0) regardless of tha nature of the meas-

ures in the set ({(,Ix): xE(X - {x}N=)}. In this chapter a1il

method will be defined for delineating those measures which can-

not affect the value of N(S, A,

Throughaut this chapter both the input and the output will

be subsets of the reals; y will be Lebesgue measure; and U will

be the usual topology for the reals.

4.1 Definition: A measure u(. Ix) is admissible if and

only if there exists a connected set AcY" such that u(AIx) 0 ,

uCAlx') I u(AIx) for all x'cX, and there exists x"cX such that

,,(Ar ' ' ) > 0.

4.2 Definition, Given 0 5 A < I a measure u(.Ix) is

A-admissible if and only if there exists a connected set AEY'

4 •

5$

such that (Ajx) 1A., u(Aly') -5 u(Alx) for all x'cX, and there

exists x"cX such that w(AC IX ) > 0.

4.3 Theorem: Let X" a {xcX: i(.lx) is admissible):. Let

X (xcX: u(.Ix) is X-admissible). Let {An} be a sequence

such that 0 )X< 1 which converges to one. Then X"

n1 n

Proof:

Any A-admissible measure is clearly admissible. Hence

1U X C X*. Let xcX*, then there exists a connected set AcY'n-l n

such that w(Aix') - u(Alx) for all x'cX, u(ACIx") > 0 for some

x"sX, and u(Alx) > 0. Thus, there exists n such that

u(Alx) > I- An hence xcX.n

4.4 Remark: If the channel is semi-continuous, then for

any 0 A < I and any connected A-code (x1, DI), ... , (n, D)

there exists connected a A-code (xi, D1). ... , (X', Dn) where1 ~ n

xjtXA for i 2 2, -..., n uless X, is empty which implies

NCS, X, U) a 1. However, for the general channel this need not

hold since maximums may not be attained. The following theorem

and corollary show that this is true for ce:tain interesting

special cases.

.1 I

56

4.3 Thcreir: Let S ; X, X) (Y, Y')) be a given

channel; let S'* X, u(.'x), (Y, Y')). Then if X is compact

and w(.Ix) is continuous in x, i.e.*u(Ajxl is a continuous

function of x for each AcY', then N(S', A, 1) - N(S, X, U)

whenever N(S, ., U) ' 1.

Proof:

Suppose N(S. A, U1. 2. Let {xi, D.}N(S, A, (4 be a

connected A-code of length N(S, A, U). Let 1 ! i -< N(S, A, U).

Since ;j(Dijx) is a continuous function of x and X is compact

uA(D. lx) has a maximum at, say, x'cX. D.i CDC for all j j i

hence, there exists x"cX such that u(Dclx") > 0. Therefore,

x.sX , If follows that N(S', A, U1 z N(S, A, U).

4.6 Corml]v! '" t b- t!e "anncl dcfinadin.4.S. If

(. IA) << y for all xcX then N(S', X, U) = N(S, A, U) for all

0 < A <1.

Proof:

Suppose there exists 0 < A < 1 such that N(S, A, U) = 1.

Given x 0X then, rince W << Y, there exists a connected ,et DcY'

such that j(DLx ) - I-A. Ilence w(DC[x) = A > 0. Let x'EX be0 0

such that i4(U)x) has a maximum at x'. Then x'cXA.

S7

The following example demonstrates a channel with an

uncountable input alphabet, in fact a closed interval, where

the set of admissible measures is countable.

4,7 Example: Let X a [o, 1] and lt (Y, Y'1 be the real

nmbers and the Borel sets respectively. Define measures UI

and u2 by

(A) e •-y2/2 dy,

2(Ai 1 e'Y2 / 8 dy2/F5 A

for each AY'. Let X2 (xlxcX, x is irrational); let X- X-X2.

The family of measures (u.Ix)IxcX} is defined by

-,,(Aix),- t.A-") if xCX W 1)

o(Aix) a u2(A-x) if xcX 2 U (0 U()

for each AMY'.

It is easy to see that X cX*. In fact, if x w 0 let

.A - C--, o); if x - 1 let A ( (o, -); if xcXlt'(o, 1) lot

A - Cx-l. x.l). Then w(A!xl 0, uI(Acx) > 0 and

u(Ajxj -Z u(AIx') for all x'cX.

Consider x2cX2. Let AY', A connectedwith uLjAtxj 0

md u(ACIx') > 0 for some x'cX. Then there exists a, 8cR

, m ! I I I I I ! ir

with Cx o (z 0 *aX' and eitlior !x-ciI < or .xfi if

either x-a c (o, 1) or x+6 c (o, 11' there exists x 1 cXf'j(c. 11

such that w(i iu(Ajx.,). If x+6 -S 0 then iA(A1O) > waAIxjIand if x-m I then u(Al1I) > iu(Ajx 2 . It followts that x 2 0X.

59

CIAPTER V

SUFFICIENT CONDITIONS FOR FINITL kAp.rTTY

Since many of the studies in information theory involve the

channel capacity, it is highly desirable to know when the capac-

ity is finite. In this €hapter, sufficient conditions are

obtained for a channel with additive noise to have finite capac-

ity. It will be assumed that the input and output are subsets

of the realnumbers.

5.1 Lemia: Let M - u(.[x~lxcX be a family of probability

measures defined on a measure space (Y, Y'). Suppose there

exists a probability measure y such that w(.jxj - y for all

xcX and {Ju(.Lx)/dyjxcX) is uniformly bounded. Given n > o let

Pn (.1u) and Yn represent the product measures an Lyn, (ynl,).

Then giv-.€ > o there exists 6 > o such that given ucXn and

Ac(Y ) ' then 1n(Aju) ! L irplies Yn(A) 6n

Proof:

Let f(ylx) - du(.Ix)/dy. Suppose {f(yjxflxcX) is

unifornly bounded by M . 1. Let c > o be given. Let 6 E c/M.

Given ucXn let n(yu) = dUn(. u)/d n. Then fn(ylu) S

nn,, r n.Hence if u (ALu; r, then 6 > ' . (c.t) =6

I-

60

The preceding lerin a enables one to det,-rine N(Sn . 0 for

any fixed -alues of n and A > 0 provided that there exists a

probability measure y with respect to which the family

{du(.Ix)/dvl is uniformly bounded.

S.2 TIecrem: Let S w (X, i(.Ix), (Y, Y'IJbe a channel.

Suppose there exists a probability measured y defined on (I,. Y')

such that t.e family is uniformly bounded.

Then, given 0 c I c 1, there exists 6, > 0 such that N(S n , A

--. for any n 0.6

Proof:

Let 0 < < 1 be given. By the lemma, there exists a

6 such that for any n - 0 and any uEXn 4n A(n)1u) Z l-) inp'ies

yn(An n . Since is a pro iability measure, it is clear that

q nthere are at most disjoint subsets of yn of y' measure 6x.n

The conclusic is now clear.

The following result, due to Kemperman L5]. will be used

tn show that the channel defined in thzorem 5.2 hs finite

capacity.

5,3 T4,ouiem. Let S be a channel with capacity C. For

0 < < 1 define

-- 1 (n, )C (XI T'I- log n N(Sn A)

n-

Then for each 0 < X I

C f ()

S, 4 -CorolIlary: Lot C be the capacity of the chinel

described in theorem 5..2. Then C <,

Proof:

Lot 0 < X < 1 be given. B/ theorem 5.*2, there

exists 6 > G 5uch that N(S n, A) f -~ Hence,A n6A

<1C .i C(X) 5 li.2 log C-) log 6n*n 6nA

5.5 Theorem~: Let S - (X, iu(.Ix), (Y. Y')} be a channel

with additive noise w which is absolutely continuois withI

respect to Lebesgue measure y. If there exists a chuice for

the Radcn-Nikodyn derivative f - d;4/dy such that f g(y)dy

where g(y) a sup {f(yjx) IxcXl. then the capacity of S is finite.

Proof:

Suppuse there exists a choice for dw/dyf, say f,

such that f g(y)dy < -, A totally finite measure. Y1 0

is defined on (Y, Y') by yl(A) I g(y)dy for all MY'.A

Let h(ytx) *d~adjx)/dy'. Then, if dyl/dy 0 0,

h~l)- & .) dy . yIX 1. Hen~ce fh(ylx)lxcX) ish~~) dy 8(y

uniformly bounded almost everywhere lyv]..

62

Define i(A) y y'(A)/y''Y) f.r all AY'. Then i is a

probability weasure and it is clear that (d,.(.Lx)/d1ix[X1 is

uniformly bounded a.e. [ ]. Thus by 5.4, the capacity of S

is finite.

S.6 Corollary: If S is a channel of type A and f is

a bounded bell function then the capacity of S is finite.

In particular, if S has additive Gaussian Noise the

capacity of S is finite.

The hypothesis of theorem 5.5 requires that there must

exist a choice for du/dy which is bounded, The foll wing

example provides a partial justification for this restriction,

.7 Example: Let S be the channel of type A defined

by. X u 10,1] i Y is the real numbers, Y' is the Borel sets,

and, gx,'!n AY u(A) - where A Al(0,A. 2x(-ln x)3/2 A1 1.

It is clear that u << y. In fact f(x) 3/2 almost2x(- In x)

everywhere [y] where f(x) is any choice for du/dy. It is easy

to see that lim f(x) = Let F(x) be the distribution func-X o

function of w. Then

F(x) 0 if x < 0

=i'-- -Xif 0 < x < e"1 xe

1 if 0x~ e

63

Hence, liven 0 X < 1Y y(A) = (T -7 for ay set A of

minimal Y-masure such that u(A) a 1-A. It follows& since

12

X [0, l3. that N(S, A) t e . Therefore lii (1-1) log

NS, X) ! hIi It follows fra Fano's theorem that

the capacity of S is infinite.

S.8 Remark: Let S {X, u(.Ix), (Y, Y')} be a channel.

nIf X can be written in the form X I Xi X. (' XJ * 0 for

i 0 j, such that the hypothesis of theorem S.S is true for

each of the subchannels Si a (Xi, u(.Ix). (Y, Y'1}, then the

capacity of C is finite. In theorem 5.5 it is shown that

for' each i there exists a probability measure and

0 < m. < such that du(.Ix)/d.,i ! m. for all xcX.. Letn

n Then 1 is a probability measure andjii

du(.Ix)/dp - n max {mill 1 i ! n) for all xcX.

Ii

64

BIBLIOGRAPHY

1. Feinstein, A., "A New Basic Theorem of Information IlTheory," Trans IRE. PGIT, 2, (1954).

2. Fano, R. M., "Statistical Theory of Communication,"Notes of a Course given at Mass. Inst. Techn.,(1952, 1954).

3. Gnednko, B. V. and Kolomogorov, A. N., Limit Distri-b'utions of Independent Random Variables, Cambridge,Mass., Addison - Wesley Publishing Co., Inc., (1954).

4. Halmos, P. R.,.Measurq Theory. New York, D. VanNostrand Co., Inc., 1950.

5. Kemperman, J. H. B., "Studies in Coding Theory I,"Unpublished Monograph, Univ. of Rochester.

6. Loeve, M., Probability Theory, 2nd, ed., New York,D. Van Nostrand Co., Inc., 1960j

7. Shannon, C. E., "A Mathematical Theory of Communication,"Bell Systems Tech J., 21, 379-423, (1948).

8. , "Certain Results in coding Theory forNoisy Channels," Inform. and Control, 1, 6-25,(1957).

9. Wolfowitz, J., "The Coding of Messages Subject to CanceErrors," Illinois J. Math, 1, No. 4, 591-606,(1957).

10. , "Strong Converse of the Coding Theoremfor Semi-Continuous Channels," Illinois J. Math.,3 No. 4, 477-489, (lq59).

11. , Coding Theoiems of Information Theory,Berlin, Springer-Verlag, 1961.

ATMOSPHERIC SCIENCES RESEARCH PAPERS

1. Webb, W.L., "Development of Droplet Size Distributions in the Atmosphere," June1954.

2. Hansen,. F. V., and If. Rachele. "Wind Structure Analysis and Forecasting Methodsfor Rockets." June 1954.

3. Webb, W. L, "Net Electrification of Water Droplets at the Earth's Surface." J. Me-teorol, December 1954.

4. Mitchell, R., "The Determination of Non-Ballistic Projectile Trajectories," March1955.5. Webb, W. L.. and. A. McPike, "Sound Ranging Technique f'r Determining the Tra-

jectory of Supersonic Missiles," =I, March 1955.6. Mitchell, R., and W. L Webb, "Electromagnetic Radiation through the Atmo-

. sphere," -1, April 1955.7. Webb, W. L., A. MlcPike, and H. Thompson. "Sound Rangirg'Technique for Deter-

mining the Tr.4jectory of Supersonic Missiles," -2, July 1955.8. Barichivich, A., "Meteorological Effects on the Refractive Index and Curvature of

Microwaves in the Atmosphere," August 1955.9. Webb, W. L, A. McPike and H. Thompson, "Sound Ranging Technique for Deter-

mining the Trajectory of Supersonic Missiles," =3, September 1955.10. Mitchell, R., "Notes on the Theory of Longitudinal Wave Motion in the Atmo-

sphere," February 1956.11. Webb, W. L., "Particulate Counts in Natural Clouds," J. MeteoroL, April 1956.12. Webb, W. L., "Wind Effect on the Aerobee," =-1, May 1956.13. Rachele, H, and L. Anderson, "Wind Effect on the Aerobee," #2, August 1956.14. Beyers, N.,"Electromagnetic Radiation through the Atmosphere, :2. January 1957.15. Hansen, F. V., "Wind Effect on the Aerobee," :3, January 1957.16. Kershner, J., and H. Bear, "Wind Effect on the Aerobee," --4, January 1957.17. Hoidale, G., "Electromagnetic Radiation through the Atniosphere," =63, February1957.18. Querfeld, C. W.. "The Index of Refraction of the Atmosphere for 2.2 Micron Radi-

ation," March 1957.19. White, Lloyd, "Wind Effect on the Aerobee," t:5, March 1957.20. Kershner, J. G.. "Development of a Method for Forecasting Component Ballistic

Wind," August 1957.21. Layton, Ivan, "Atmospheric Particle Size Distribution," December 1957.22. Rachele, Henry and W. H. Hatch, "Wi6d Effect on the Aerobee," _6, February

1958.23. Beyers, N. J., "Electromagnetic Radiation through the Atmosphere," #4, March

1958.24. Prosser, Shirley J., "Electromagnetic Radiation through the Atmosphere," a5,

April 1958.25. Armendariz, M., and P. H. Taft, "Double Theodolite Ballistic Wind Computations,"

June 1958.26. Jenkins, K. R. and W. L. Webb, "Rocket Wind Measurements," June 1958.27. Jenkins, K. R.. "Measurement of High Altitude Winds with LOi," July 1958.28. Hoidale, G., "Electromagnetic Propagation through the Atmosphere," #6, Febru-

ary 1959.29. McLardie, M., R. Helvey, and L. Traylor, "Low-Level Wind Profile Prediction Tech-

niques," :1, June 1959.30. Lamberth, Roy, "Gustiness at White Sands Missile Range," 1, May 1959.31. Beyers, N. J., B. Hinds, and G. Hoidale, "Electromagnetic Propagation through the

Atmosphere," tt7, June 195"32. Beyers, N. J., "Radar Refraction at Low Elevation Angles (U)," Proceedings of the

Army Science Conference, June 1959.33. White, L., 0. W. Thiele and P. H. Taft, "Summary of Ballistic and Meteorological

Support During IGY Operations at Fort Churchill,. Canada," August1959.34. Hainline, D. A., "Drag Cord-Aerovane Equation Analysis for Computer Application,"

August 1959.35. Hoidale. G. B., "Slope-Valley Wind at WSMR," October 1959.36. Webb, W. L, and K. R. Jenkins, "High Altitude Wind Measurements," J. Meteor-

oL, 16, 5, October 1959.

i i , Id __ j

37. White, Lloyd, "Wind Effect on the Aerobee," #9, October 1959.38. Webb, W. L, J. W. Coffman, and G. Q. Clark, "A High Altitude Acoustic Sensing

System," December 1959.39. Webb, W. L. and K. R. Jenkins, "Ap lication of Meteorological Rocket Systems,"

I. Geophys. Re&,, 64, 11, November 1959.40. Duncan, Louis, "Wind Effect on the Aerobee," #10, February 1960.41. Helvey, R. A., "Low-Level Wind Profile Prediction Techniques," :t2, February 1960.42. Webb, W. L, and K. R. Jenkins, "Rocket Sounding of High-Altitude Parameters,"

Proc. GM ReL Symp., Dept. of Defense, February 1960.43. Armendariz, M., and H. H. Monahan, "A Comparison Between the Double Theodo-

lite and Single-Theodolite Wind Measuring Systems," April 1960.44. Jenkins, K. R., and P. H. Taft, "Weather Elements in the Tul, rosa Bp-in," July 1960.46. Beyes, N. J., "Preliminary Radar Performance Data on Passive Rocket-Borne Wind

Sensors," IRE TRANS, MIL ELECT, MIL-4, 2-3, April-July 1960.46. Webb, W. L, and R. R. Jenkins, "Speed of Sound in the Stratosphere," June 1960.47. Webb, W. L, K. R. Jenkins. and G. Q. Clark, "Rocket Sounding of High Atmo-

sphere Meteorological Parameters," IRE Trans. MiL Elect., MIL-4, 2-3,April-July 1 960.

48. Hehvey, R A., "Low-tLevel Wind Profile Prediction Techniques," #3, Sintenber1960.49. Beyers, N. J., and 0. W. Thiele, "Meteorologicil Wind Sensors," August 1960.50. Armijo, Larry, "Determination of Trajectories Using Range Data from Three Non-

colinear Radar Stations," September 1960.51. Carnes, Patsy Sue, "Temperature Variations in the First 200 Feet of the Atmo-

sphere in an Arid Region," July 1961.52. Springer, H. 8, and R. 0. Olsen, "Launch Noise Distribution of Nike-Zeus Mis-

iles," July 1961.53. Thiele, 0. W., "Density and Pressure Profiles Derived from Meteorological Rocket

Measurements," September 1961.54. Diamond, M. and A. B. Gray, "Accuracy of Missile Sound Ranging," November

1961.55. Lhmberth, R. L. and D. R. Veith, "Variability of Surface Wind in Short Distances,"

I1, October 1961.56. Swanson, R. N., "Low-Level Wind Measurements for Ballistic Missile Application,"

January 1962.57. Lamberth, R. I. and J. H. Grace, "Gustiness at White Sands Missile Range," #2,

January 1962.58. Swanson, R. N. and M. M. Hoidale, "Low-Level Wind Profile Prediction Tech-

niques," #4, January 1962.59. Rachele, Henry, "Surface Wind Model for Unguided Rockets Using Spectrum and

Cross Spectrum Techniques," January 1962.60. Rachele, Henry, "Sound Propagation through a Windy Atmosphere," #2, Febru-

ary 1962.61. Webb, W. L, and K. R. Jenkins, "Sonic Structu.e of the Mesosphere," J. Acous.

Soc. Amer., 34, 2, February 1962.62. Tourin, M. H. and M. M. Hoidale, "Low-Level Turbulencm Characteristics at White

Sands Missile Range," April 1962.63. Miers, Bruce T., "Mesospheric Wind Reversal over White Sands Missile Range,"

March 1962.64. Fisher, E., R. Lee and H. Rachele, "Meteorological Effects on an Acoustic Wave

within a Sound Ranging Array," May 1962.65. Walter, E. L, "Six Variable Ballistic Model for a Rocket," June 1962.66. Webb, W. L, "Detailed Acoustic Structure Above the Tropopause," J. Applied Me-

teoroL, 1. 2, June 1962.67. Jenkins, K. R., "Empirical Comparisons of Meteorological Rocket Wind Sensors," J.

Appl. Meteor., June 1962.. Larnberth, Roy, "Wind Variability Estimates as a Function of Sampling Interval,"

July "962.69. Rachele, Henry, "Surface Wind Sampling Periods for Unguided Rocket Impact Pre-diction," July 1962.

70. 'raylor, Larry, "Coriolis Effects on the Aerobee-Hi Sounding Rocket," August 1962.71. McCoy, J., and G. Q. Clark, "Meteorological Rocket Thermometry,' August 1962.72. Rachele, Henry, "Real-Time Prelaunch Impact Prediction System," August 1962.

73. Beyers, N. J., 0. W. Thiele. and N. K. Wagner, "Performance Characteristics of7. Meteorlogical Rocket Wind and Temperature Sensors," October 1962.74. Coffman, J., and R. Price, "Some Errors Associated with Acoustical Wind Measure.

ments through a Layer." October 1962.75. Arnendariz, M., E. Fisher. and J. Serna, "Wind Shear in the Jet Stream at WS.

MR," November 1962.76. Armendariz, M., F. Hansen, and S. Carnes, "Wind Variability and its Effect on Roe-

ket Impact Prediction," January 1963.77. Querfeld, C., and Wayne Yunker, "Pure Rotational Spectrum of Water Vapor, I:

Table of Line Parameters." Febmary 1963.78. Webb, W. L. "Acoustic Component of Turbulence," J. Applied Mteorol., 2, 2,

April 1963.79. Beyers, N. and L Engberg, "Seasonal Variability in the Upper Atmosphere," May

1963.80. Williamson, L E., "Atmospheric Acoustic Structure of the Sub-polar Fall," May 1963.81. Lamberth, Roy and D. Veith, "Upper Wind Correlations in Southwestern United

States," June 1963.82. Sandlin, E., "An analysis of Wind Shear Differences as Measured by AN/FPS-16

Radar and AN, GMD-IB Rawinsonde," August 1963.83. Diamond, M. and R. P. Lee, "Statistical Data on Atmospheric Design Properties

Above 30 km," August 1963.84. Thiele, 0. W., "Mesospheric Density Variability Based on Recent Meteorological

Rocket Measurements," J. Applied Meteorol., 2, 5, October 1963.85. Diamond, M., and 0. Essenwanger, "Statistical Data on Atmospheric Design Prop-

erties to 30 km," Astro. Acro. Engr., December 1963.86. Hansen, F. V., "Turbulence Characteristics of the First 62 Meters of the Atmo-

sphere." December 1963.87. Morris, J. E., and B. T. Miers, "Circulation Disturbances Between 25 and 70 kilo-

meters Associated with the Sudden Warming of 1963," J. of Geophys.Res., January 1964.

88. Thiele, 0. W., "Some Observed Short Term and Diurnal Variations of Stratospher-ic Density Above 30 km." January 1964.

89. Sandlin, R. E., Jr. and E. Armijo, "An Analysis of AN/FPS-16 Radar and AN/GMD-IB Rawinsonde Data Differences," January 1964.

90. Miers, B. T., and N. J. Beyers, "Rocketsonde Wind and Temperature Measure-ments Between 30 and 70 km for Selected Stations,' J. Applied Mete-orol., February 1964.

91. Webb, W. L, "The Dynamic Stratosphere," Astronautics and Aerospace Engineer-ing, March 1964.

92. Low, R. D. H., "Acoustic Measurements of Wind through a Layer," March 1964.93. Diamond. M., "Cross Wind Effect on Sound Propagation," J. Applied MeteoroL,

April 1964.94. Lee, R. P., "Acoustic Ray Tracng." April 1964.95. Reynolds, R. D., "Investigation of the Effect of Lapse Rate on Balloon Ascent Rate,"

May 1964.96. Webb, W. L., "Scale of Stratospheric Detail Structure," Space Research V, May

1964.97. Barber, T. L., "Proposed X-Ray-Infrared Method for Identification of Atmospher-

ic Mineral Dust." June 1964.98. Thiele, 0. W., "Ballistic Procedures for Unguided Rocket Studies of Nuclear Environ-

ments (U)," Proceedings of t.,,- Army Science Conference, June 1964.99. Horn, J. D., and E. J. Trawle, "Orographic Effects on Wind Variability," July 1964.

100. Hoidale, G., C. Querfeld, T. Hall, and R. Mireles, "Spectral Transmisivity of theEarth's Atmosphere in the 250 to 500 Wave Number Interval," #1,September 1964.

101. Dunce.n, L D., R. Ensey, and B. Engebos, "Athena Launch Angle Determination,"September 1964.

102. Thiele, 0. W.. "Feasibility Experiment for Measuring Atmospheric Density Throughthe Altitude Range of 60 to 100 KM Over White Sands Missile Range,"October 1964.

103. Duncan, L. D., and R. Ensey, "Six-Degree-of-Freedom Digital Simulation Model forUnguided, Fin-Stabilized Rockets," November 1964.

104. Hoidale, G., C. Querfeld. T. Hall, and R. Mireles, "Spectral Transmisaivity of theEarth's Atmosphere in the 250 to 500 Wave Number Interval," #2,November 1964.

105. Webb, W. L, "Stratospheric Solar Response," J. Atmos. Sci., November 1964.106. McCoy, J. and G. Clark. "Rocketsonde Measurement of Stratospheric Temperature,"

December 1964.107. Farone. W. A., "Electromagnetic Scattering from Radially Inhomogeneous Spherm

as Applied to the Problem of Clear Atmosphere Radar Echoes," Decem-ber 1964.

108. Farone, W. A., "The Effect of the Solid Angle of Illumination or Observation on theColor Spectra of 'White Light' Scattered by Cylinders," January 1965.

109. Williamson, L E.. "Seasonal and Regional Characteristics of Acoustic Atmospheres,"J. Geophys. Res.. January 1965.110. Armendariz, M., "Ballistic Wind Variability at Green River, Utah," January 1965.111. Low, R. D. H., "Sound Speed Variability Due to Atmoopheric Coar-, -ition," Janu-

ary 1965.112. 4uerieid, C. Wv.. Mie Atrnc.;phcrc uptics," .0. u.pt. 1-o, Jio.n-. tar., ;.e

113. Coffman, J., "A Measurement of the Effect of Atmospheric Turbulence on the Co-herent Properties of a Sound Wave," January lV15.

114. Rachele, H., and D. Veith. "Surface Wind Sampling for V .guided Rocket ImpactPrediction." January 1965.

115. Ballard. H., and M. Izquierdo, "Reduction of Microphone Wind Noise by the Gen-eration of a Proper Turbulent Flow," February-1965.

116. Mireles, R., "An Algorithm for Computing Half Widths of Overlapping Lines on Ex-perimental Spectra." February 1965.

117. Richart, H., "Inaccuracies of the Single-Theodolite Wind Measuring System in Bal-listic Application." February 1965.

118. D'Arcy, M., "Theoretical and Practical Study of Ae-obee-150 Ballistics," March1965.

119. McCoy, J., "Improved Method for the Reduction of "?0cketsonde Temperature Da-ta," March 1965.

120. Mireles, R., "Uniqueness Theorem in Inverse Electromagnetic Cylindrical Scatter-ing." April 1965.

,21. Coffman, J., "The Focusing of Sound Propagating Ve-tically in a Horizontally Stra-tified Medium." April 1965.

122. Farone, W. A., and C. Querfeld. "Electromagnetic Scattering from an Infinite Cir-cular Cylinder at Oblique Incidence." April 1965.

123. Rachele, H., "Sound Propagation through a Windy Atmosphere," April 1965.124. Miers. B., "Upper Stratospheric Circulation over Ascension Island," April 1965.125. Rider, L., and M. Armendariz, " A Comparison of Pibal and Tower Wind Measure-

ments," April 1965.126. Hoidale, G. B., "Meteorological Conditions Allowing a Rare Observation of 24 Mi-

cron Solar Radiation Near Sea Level," Meteorol. Magazine, May 1965.127. Beyers, N. J., and B. T. Miers. "Diurnal Temperature Change in the Atmosphere

Between 30 end 60 km over White Sands Missile Range," J. Atmos.Sci., May 1965.

128. Querfeld. C., and W. A. 7arone, "Tables of the Mie Forward Lobe," May 1965.129. Farone, W. A., Generalizition of Rayleigh-Gans Scattering from Radially Inhomo-

geneous Spheres," J. Opt. Soc. Amer.. June 1965.130. Diamond, M., "Note on Mesospheric Winds Above White Sonds Missile Range," J.

Applied Metrorol.. June 1965.131. Clark, G. Q., and J. G. McCoy, "Measurement of Stratospheric Temperature," J.

Applied Meteoroi., June 1965.132. Hall, T., G. Hoidale, R. Mireles, and C. Querfeld, "Spectral Transmis 'vity of the

Earth's Atmosphere in the 250 to 500 Wave Number Interval," #3,July 1965.

133. McCoy, J.. ar'd C. Tate. "The Delta-T Meteorological Rocket Payload," June 1964.134. Horn, J. D.. "Obstaclt Influenice in a Wind Tunnel," July 1965.

.135. McCoy, J., "An AC Probe for the Measurement of Electron Density and CollisionFrequency in the Lower Ionosphere," July 1965.

136. Miers, B. T., M. D. Kays, 0. W. Thiele and E. M. Newby, "Investigation of ShortTerm Variations of Several Atmospheric Parameters Above 30 KM,"July 1965.

137. Serna, J., "An Acoustic Ray Tracing Method for Digital Computation," September1965.

138. Webb, W. L., "Morphology of Noctilucent Clouds," J. Geophys. Res., 70, 18, 4463-4475, _ptemer 196o.

139. Kays, M., and R. A. Craig, "On the Order of Magnitude of Large.Scale Vertical Mo-tions in the Upper Stratosphere," J. Geophys. Res., 70, 18, 4453-4462,September 1965.

140. Rider, L., "Low-Level Jet at White Sands Missile Range," September 1965.141. Lamberth, R. L.. R. Reynolds, and Morton Wurtele, "The Mountain Lee Wave at

White Sands Missile Range," BuU. Amer. Meteorol. Soc., 46, 10, Octo-ber 1965.

142. Reynolds, R. and R. L. Lamberth, "Ambient Temperature Measurements from Ra-diosondes Flown on Constant-Level Balloons," October 1965.

143. McCluney, E., "Theoretical Trajectory Performance of the Five-Inch Gun ProbeSvstem," October 1965.

144, Pena, R. and N1. Diamond, "Atmospheric Sound Propagation near the Earth's Sur-fa.e," October 1965.

145. Mason, J. B., "A Study of the Feasibility of Using Radar Chaff For StratosphericTemperature Measurements," Novemober 1965.

146. Diamond, M., and R. P. Lee, "Long-Range Atmospheric Sound Propagation," J., , po, "n. 29. November 1965.

147. Lamberth. R. L. "On the Measurement of Dust Devil Paraeters," N'wp.-t"r 1965.148. Hansen, F. V., and P. S. Hansen. "Formation of an Internal Boundary over Heter-

ogeneous Terrain." November 1965.149. Webb, W. L., "Mechanics of Strtospheric Seasonal Reversals," November 1965.150. U. S. Army Electronics R & D Activity, "U. S. Army Participation in the Meteoro-

logical Rocket Network," January-1966.151. Rider, L J., and M. Armendariz, "Low-Level Jet Winds at Green River, Utah," Feb-

ruary 1966.152. Webb, W. L., "Diurnal Variations in the Stratospheric Circulation:" February 1966.153. Beyers, N. J., B. T. Miers, and R, J. Reed. "Diurnal Tidal Motions near the Strato-

pause During 48 Hours at WSMR," February 1966.154. Webb, W. L., "The Stratospheric Tidal Jet," February 1966.155. Hall, J. T., "Focal Properties of a Plane Grating in a Convergent Beam," February

1966.156. Duncan, L. D., and Henry Rachele, "Real-Time Meteorological System for Firing of

Unguided Rockets," February 1966.157. Kays, M. D., "A Note on the Comparison of Rocket and Estimated Geostrophic Winds

at the 10-mb Level," J. Appl. Meteor., Febriary 1966.158. Rider, L., and M. Am 'ndariz, " A Comparison of Pibal and Tower Wind Measure-

mentq," J. Appl. Meteor., 5. February 1966.159. Duncan, L D.. "Coordinate Transformations in Trajectory Simulations," February

1966.160. Williamson, L. E., "Gun-Launched Vertical Probes at White Sands Missile Range,"

February 1966.161. Randhawa, J. S., Ozone Measurements with Rocket-Borne Ozonesondes," March

1966.162. Armendariz, Manuel, and Laurence J. Rider, "Wind Shear for Small Thickness Lay-

ers," March 1966.163. Low, R D. H., "Continuous Determination of the Average Sound Velocity over an

Arbitrary Path." March 1966.164. Hansen, Frank V., "Richardson Number Tables for the Surface Boundary Layer,"

March 1966.165. Cochran, V. C., E. M. D'Arcy, and Florencio Ramirez, "Digital Computer Program

for Five-Degree-of-Freedom Trajectory," March 1966.166. Thiele, 0. W., and N. J. Beyers, "Comparison of Rocketsonde and Radiosonde Temp-

eratures and a Verification of Computed Rocketsonde Pressure and Den-sity," April 1966.

167. Thiele, 0. W., "Observed Diurnal Oscillations of Pressure and Density in the UpperStratosphere and Lower Mesosphere." April 1966.

168. Kays, M. D., and R. A. Craig, "On the Order of Magnitude of Large-Scale VerticalMotions in the Upper Stratosphere," J. Geophy. Res., April 1966.

169. Hansen, F. V., "The Richardson Number in the Planetary Boundary Layer," May1966.

,4

' ' ' ' ' , l l I I I I I I ii

170. Ballard, H. N., "The Measurement of Temperature in tLe Stratosphere and Meso-sphere," June 1966.

171. Hansen, Frank V.. "The Ratio of the Exchange Coefficients for Heat and Momentumin a Homogeneous, Thermally Stratified Atmosphere," June 1966.

172. Hansen, Frank V., "Comparison of Nine Profile Models for the Diabatic BoundaryLayer," June 1966.

173. Rachele, Henry, "A Sound-Ranging Technique for Locating Supersonic Missiles,"May 1966.

174. Farone, W. A., and C. W. Querfeld. "Electromagnetic Scattering from InhomogeneousInfinite Cylinders at Oblique Incidence," J. Opt. Soc. Amer. 56, 4, 476-480, April 1966.

175. Mireles, Ramon, "Determination of Parameters in Absorption Spectra by NumericalMinimization Techniques," J. Opt. Soc. Amer. 56, 5, 644-647, May 1966.

176. Reynolds, R., and R. L. Lamberth. "Ambient Temperature Measurements from Ra-diosondes Flown on Constant-Level Balloons," J. Appi. Meteorol., 5, 3,304-307, June 1966.

177. Hall, James T., "Focal Properties of a Plane Grating in a Convergent Beam," Appl.Opt., 5, 1051. June 1966

178. Rider, Laurence J., "Low-Level Jet at White Sands Missile Range," J. Appl. Mete-orol., 5, 3, 283-297, June 1966.

17q. McCluney, Eugene, "Projectile Dispersion as Caused by Barrel Displacement in the5-Lncn 4, - Pr ' qyqtpr." July 1966.

180. Armendariz, Manuel, and Laurence J. Rider, "Wind Shear Ca.,:uatLons for SralShear Layers." June 1966.

181. Lamberth, Roy L. and Manuel Armendariz. "Upper Wind Correlations in the Cen-tral Rocky Mountains," June 1966.

182. Hansen, Frank V., and Virgil D. Lang. "The Wind Regime in the First 62 Meters ofthe Atmosphere," June 1966.

183. Randhawa, Jagir S.. "Rocket-Borne Ozonesonde," July 1966.184. Rachele, Henry, and L. D. Duncan. "The Desirability of Using a Fast Sampling Rate

for Computing Wind Velocity from Pilot-Balloon Data," July 1966.185. Hinds, B. D., and R. G. Pappas, "A Comparison of Three Methods for the Cor-

rection of Radar Elevation Angle Refraction Errors," August 19cS186. Piedmuller, G. F., and T. L. Barber, "A Mineral Transition in Atmospheric Dust

Transport," August 1966.187. Hall, J. T., C. W. Querfeld, and G. B. Hoidale. "Spectral Transmissivity of the

Earth's Atmnspher- in the 250 to 500 Wave Number Interva!," PartIV (Final). July 1966.

188. Duncan, L D. and B. F. Engebos, "Techniques for Computing Launcher Settingsfor Unguided Rockets," September 1966.

189. Duncan, L D., "Basic Considerations in the Development of an Unguided RocketTrajectory Simul:ition Molel." September 1966.

190. Miller, Walter B., "Considerstion of S)me Problems in Curve Fitting," September1966.

191. Cermak, J. E., and J. D. Horn. "The Tower Shadow Effect," August 1966.192. Webb, W. L, "Stratospheric Circulation Response to a Solar Eclipse," October 1966.193. Kennedy, Bruce. "Muzzle Velocity Measurement," October 1966.194. Traylor, Larry E., "A Refinement Technique for Unguided Rocket Drag Coeffic-

ients," October 1966195. Nusbaum, Henry, "A Reagent for the Simultaneous Microscope Determination of

Quartz and Halides," October 1966.196. Kays, Marvin and R. 0. Olsen, "Improved Rocket-onde Parachute-derived Wind

Profiles," October 1966.197. Engebos, Bernard F. and Duncan, Louis D., "A Nomogram for Field Determina-

tion of Launcher Angles for Unguided Rockets," October 1966.198. Webb, W. L, "Midlatitude Clouds in the Upper Atmosphere," November 1966.199. Hansen, Frank V., "The Lateral Intensity of Turbulence as a Function of Stability,"

November 1966.200. Rider, L J. and M. Armendariz, "Differences of Tower and Pibal Wind Profiles,"

November 1966.201. Lee, Robert P., "A Comparison of Eight Mathematical Models for Atmospheric

Acoustical Ray Tracing," November 1966.202. Low, R. D. H., et al., "Acoustical and Meteorological Data Report SOTRAN I and

II," November 1966.

203. Hunt, J. A. and J. D. Horn, "Drag Plate Balance," December 1968.204. Armeadaris, M., and H. Rachele. "Determination of a Representative Wind Profile

from Balloon Data," December 1966.205. Hans n, Frank V., "The Aerodynamic Roughness of the Complex Terrain of White

Sands Missile Range," Jaiuary 1967.206. Morris, James E., "Wind Measurements in the Subpolar Mesopause Region," Jan.

uary 1967.207. Hall, James T., "Attenuation of Millimeter Wavelength Radiation by Gasou

Water," January 1967.2G8. Thiele, 0. W., and N. J. Beyers, "Upper Atmosphere Pressure Measunreents With

Thermal Conductivity Gauges," January 1967.209. Armendariz, M., and H. Rachele, "Determination of a Representative Wind Profile

from Balloon Data," January 1967.210. Hansen, F. V "The Aerodynamic Roughnes. oi the Complex Terrain of White SandsMissile Range, New Mexico," January 1967.211. D'Arcy, Edward M., "Some Applications of Wind to Unguided Rocket Impact Prg-

diction," March 1967. I212. Kennedy, Bruce, "Operation Manual for Stratosphere Temperature Sonde," March

1967.213. Hoidale, G. B., S. M. Smith, A. J. Blanco, and T. L. Barber, "A Study of Atmophe.

tic Dust," March 1967.214. Longyear, J. ., *... .... ..... " 'or ObtaininF "' tinns tn TAPLp:P. Titad Z"-

tions," March 1967.215. Rider, L. J., "A Comparison of Pibal with Raob and Rawin Wind Measuremnnts,"

April 1967.216. Breeland, A. H., and R. S. Bonner, "Results of Tests Involving Hzniapherical Wind

Screens in the Reduction of Wind Noise," April 1967.217. Webb, Willis L, and Max C. Bolen, "The D-region Fair-Weather Electric Field,"

April 1967.218. Kubinaki, Stanley F., "A Comparative Evaluation of the Automatic Tracking Pilot-

Balloon Wind Measuring System," April 1967.219. Miller, Walter B., and Henry Rachele. "On Nonpararnetric Testing of the Nature of

Certain Time Series," April 1967.220. Hansen, Frank V., "Spacial and Temporal Distribution of the Gradient Richardson

Number in the Surface and Planetary Layers," May 1967.221. Randhawa, Jagir S., "Diurnal Variation of Ozone at High Altitudes," May 1967.222. Ballard, Harold N., "A Review of Seven Papers Concerning the Measurement of

Temperature in the Stratosphere and Mesosphere," flay 1967.223. Williams, Ben H., "Synoptic Analyses of the Upper Stratospheric Circulation Dur-

ing the Late Winter Storm Period of 1966," May 1967.224. Horn, J. D., and J. A. Hunt, "System Design for the Atmospheric Sciences Office

Wind Research Facility," May 1967.225. Miller, Walter B., and Henry Rachele, "Dynamic Evaluation of Radar and Photo

Tracking Systems, " May 1967.226. Bonner, Robert S., and Ralph H. Rohwer, "Acoustical and Meteorological Data Re-

port - SOTRAN III and IV," May 1967.227. Rider, L. J., "On Time Variability of Wind at White Sands Missile Range, New Mex-

ico," June 1967.228. Randhawa, Jagir S., "Mesospheric Ozone Measurements During a Solar Eclipse,"

June 1967.229. Beyers, N. J., and B. T. Miers, "A Tidal Experiment in the Equatorial Stratosphere

over Ascension Island (8S)", June 1967.230. Miller, W. B., and H. Rachele, "On the Behavior of Derivative Processes," June 1967231. Walters, Randall K., "Numerical Integration Methods for Ballistic Rocket Trajec.

tory Simulation Programs," June 1967.232. Hansen, Frank V., "A Diabatic Surface Boundary Layer Model," July 1967.233. Butler, Ralph L, and James K. Hall, "Comparison of Two Wind Measuring Sys-

tema with the Contraves Photo-Theodolite," July 1967.234. Webb, Willis L., "The Source of Atmospheric Electrification," June 1967.

235. Hinds, B. D.. "Radar Tracking Anomalies over an Arid Interior Basin," August 1967.236. Christian, Larry 0., "Radar Cross Sections for Totally Reflecting Spheres," August

1967.237. D'Arcy, Edward M., "Theoretical Dispersion Analysis of the Aerobee 350," August

1967.238. Anon., "Technical Data Pa(kigu for Rocket-Borne Temperature Sensor," August

1967.239. Glass, Roy I., Roy L. Lamberth. and Ralph D. Reynolds, "A High Resolution Con-

tinuous Pressure Sensor Modification for Radiosondes," August 1967.240. Low, Richard D. H.. "Acoustic Measuternent of Supersaturation in a Warm Cloud,"

August 1967.241. Rubio. Roberto, and Harold N. Ballard. "Time Response and Aerodynamic Heating

of Atmospheric Temt, rature Sensing Elements." August 1967.242. Seagraves, Mary Ann B.. "Theretiral Performance Characteristics and Wind Effects

for the Aerobee 150." August 1967.243. Duncan, Louis Dean. "Channel Capacity and Coding," August 1967.

DOCUMENT CONTROL :.' A A, 0

Fort "onnouthi, N'ew Jersey 07703 '

CH!ANNEL CAPACIT'Y AND CrDING.r

4 oec"IN'is Nova 4Tv OfhD dpe Sf KI .St.)

I £W 1.04181 (0, o~. . 4" lE* e t~. to At )

Louis 0. Duncan

4S55505? O.Cre I& ,TTAL f OF PAGUS 776. w oou Far@

August 1967 64 1

l.PeeN L01300l.\91A LC(Ykl SISI

Distributior of this report is unlie.itcd

if. OUPOLIIsG.Tafv eve$ TB*PaCl s ILT 0" WLTA*V Ac ra-V~

Dissertation for Degree of Doctor of Atmospheric Sciences LaboratoryPhilosophy, Ne' !exico State University U. S. Arm~y Electronics Comrand

-.hite Sands "issiie Rnpe, 'ew "exico

- This report presents original results in information. An irprove-ment of Faflo's theorem is proved. A sufficient condition for finitecapacity for certain channels with additive noise is proved. Animmediate corollary shows that the c~iannel with additive tiussiarinoise has finite capacity.

A new techniqu^ for coding -the connected code -is defined andinvestigated.

DDf Nwl o 47 :4=CE D0 "0." .1 " A" 'c4 'aCw.

Security CIesAcet$.

ik

uj'CLAr5tI E

LINK LON. 8 1."

PLE GPLEt wt POLE v

1. Information Theory2. rinite Caj,,ac ity3. Connected CDe4. rano's TVieerc".

UNC LA SS TFl EDSecwtty CIlu.*flcation