Sequence Transformations and their ApplicationsJet Wimp DEPARTMENT OF MATHEMATICAL SCIENCES DREXEL UNIVERSITY PHILADELPHIA, PENNSYLVANIA
@ 1981
New York London Toronto Sydney San Francisco
COPYRIGHT © 1981, BY ACADEMIC PRESS, INC. ALL RIGHTS RESERVED. NO PART OF THIS PUBLICATION MAY BE REPRODUCED OR TRANSMITTED IN ANY FORM OR BY ANY MEANS, ELECTRONIC OR MECHANICAL, INCLUDING PHOTOCOPY, RECORDING, OR ANY INFORMATION STORAGE AND RETRIEVAL SYSTEM, WITHOUT PERMISSION IN WRITING FROM THE PUBLISHER.
ACADEMIC PRESS, INC. III Fifth Avenue. New York, New York 10003
United Kingdom Edition published by ACADEMIC PRESS, INC. (LONDON) LTD. 24 28 Oval Road, London NWI 7DX
Library of Congress Cataloging in Publication Data
Wimp, Jet. sequence transformations and their applications.
(Mathematics in science and engineering) Bibliography: p. Includes index. 1. Sequences (Mathematics) 2. Transformations
(Mathematics) 3. Numerical analysis. I. Title. II. Series. QA292.W54 515'.24 80-68564 ISBN 0-12-757940-0
PRINTED IN THE UNITED STATES OF AMERICA
81 82 83 84 987654321
Preface
In this book we shall be concerned with the practical aspects of sequence transformations. In particular, we shall discuss transformations T mapping sequences in a Banach space 81 (often, but not always, the complex field) into sequences in 81. Certain practical requirements are ordinarily made of T: that its domain f» contain an abundance of" interesting" sequences and for S E f» also as + e E ~, e being any constant sequence; further, we shall usually require that T satisfy the following requirements:
(i) T is homogeneous: T(as) = aT(s) for any scalar a; (ii) T is translative: T(s + e) = T(s) + T(e) for any constant se-
quence e; (iii) T is regular for s: if s converges, then T(s) converges to the same
limit.
Often more than (iii) is required, namely,
(iii') T is accelerative for s: T(s) converges more rapidly than s.
This requirement sometimes takes the form that
lim II{T(s)}n - sll = f3 < I n~OCJ [s, - sliP
for some indexp ~ I, where {T(s)}n and Sn are the nth components of T(s) and s, respectively, and s is the limit of s.
Historically, most of the work done in this area up to 1950 focused on transformations that are also linear: T(s + t) = T(s) + T(t). Such trans- formations have a very simple structure, namely, the components of T(s)
ix
x Preface
can be characterized by weighted scalar means of the components of s (at least when :!4 is separable), and such transformations have beautiful theor- etical properties. [The classical work in this area is the book "Divergent Series" (Hardy, 1956), and more modern developments are discussed in Cooke (1950), Zeller (1958), Petersen (1966), and Peyerimhoff (1969).] However, linear methods are distinctly limited in their usefulness primarily because the class of sequences for which the methods are regular is too large. In defense of this somewhat paradoxical statement, I only remark that experience indicates the size of the domain of regularity of a transformation and its efficiency(i.e., the sup of p values in the foregoing equation) seem to be inversely related. Furthermore, linear transformations whose domains of regularity are all convergent sequences (called regular transformations) generally accelerate convergence at most linearly, i.e., p = 1, 0 < f3 < 1. Obviously, for safety's sake, when one uses a nonregular method, one wants a criterion for deciding when s belongs to its domain of regularity. This, however, is not the problem it might seem to be.
Linear regular transformations are discussed (at length, in fact) in this book, but primarily those transformations whose application can be effected through a certain simple computational procedure called a lozenge method.
As the reader will find, the subject touches virtually every area of analysis, including interpolation and approximation, Pade approximation, special functions, continued fractions, and optimization methods, to name a few; and the proofs of the theorems draw their techniques from all these dis- ciplines. Incidentally, I have included a proof only if it is either short or conceptually important for the discussion at hand. It was simply not feasible to include very detailed and computational proofs, e.g., estimates for the Lebesgue constants for various transformations (Section 2.4), or inequalities satisfied by the iterates in the e-algorithm, or long proofs whose flavor was totally that of another discipline-results on Pade theory, for instance, or results requiring the theory of Hilbert subspaces. In such cases, I have always indicated where the proof can be found.
The techniques given will, I hope, be useful in any practical problem that requires the evaluation of the limit of a sequence: the summation of series, numerical quadrature, the solution of systems of equations. Particu- larly welcome should be the discussion of methods to accelerate the con- vergence of sequences arising from Monte Carlo statistical experiments. Since the convergence of Monte Carlo computations is so poor, O(n -1/2), n being the number of trials, techniques for enhancing convergence are highly desirable.
A closely related subject is the iterative solution of (operator) equations. In fact, any sequence transformation can be used to define such an iterative method (cf. Chapter 5). However, this is not the subject proper of this book,
Preface xi
there being available already several excellent works in this area. I have, in fact, restricted myself mostly to material which has not appeared in book form in English. Some of the material is available in French [any numerical analyst will have on his shelf C. Brezinski's two important volumes (Brezinski, 1977, 1978)], but much of the material has never appeared in book form, some has not appeared in published papers [the thesis work of Higgins (1976) and Germain-Bonne (1978) for instance], and much is new altogether.
I have not usually opted for abstraction. In most instances the trans- formations can be generalized from complex sequences to Banach-space- valued sequences, and often I have indicated how this can be done and have established appropriate convergence results. But where abstraction can confuse rather than elucidate, I have left well enough alone. For instance, I believe that the theory of Pade approximants, at least for my purposes, is most firmly at home in classical function theory.
My notation may at times seem idiosyncratic, but it is one I have found necessary to diminish clutter and bring some focus to the development. Before the reader gets into the book, I strongly advise him to read the section on notation. Otherwise, certain unfamiliar conventions-for instance, xnR: Yn, which I have found most useful-may well render the material completely opaque. The notation for special functions is, by and large, as in the Bateman manuscript volumes. Ad hoc notation is explained in Notation or as needed.
I have provided many numerical examples, but these are illustrative only, not exhaustive. The reader interested in further numerical examples and applications should consult C. Brezinski's (1978) book, and, for a compari- son of methods, the survey of Smith and Ford (1979).
The problem of rounding is always an annoying one in a book dealing with numerical methods. Generally speaking, all numbers free from decimal points or occurring in definitions may be considered exact. Others, particu- larly those occurring in tables, have been rounded to the number of places given. However, I should be surprised if I have been consistent.
Acknowledgments
Several people have contributed to this book. John Quigg has read and commented on some of the material. Bob Higgins, my former student, has provided most of the theory in Chapters 12 and 13. Steve Yankovich and Stanley Dunn have contributed their programming and analytical skills for the preparation of numerical examples. Drexel University has been generous in its support and encouragement. I am grateful to Alison Chandler, whose combined typing and mathematical skills led to such a beautifully prepared manuscript, and to Don Johnson and Harold Schwalm, Jr., who assisted in the proofreading.
Finally, I consider myself fortunate to be working in a field where friends are so easily made. My colleagues have proved to be warm and enthusiastic. I have enjoyed thoroughly meeting and exchanging ideas with Bernard Germain-Bonne and Florent Cordellier. I am particularly indebted to correspondence and discussions with Claude Brezinski. He has generously provided me with unpublished results (Chapter 10). Some of the ideas in the book originated in a lengthy afternoon discussion with Claude and other colleagues. That meeting demonstrated to me the delights of the mutual, as opposed to solitary, quest.
XIII
Notation
Spaces
fJI Banach space
-* dual space
B(81, fJI') space of all bounded linear mappings of one Banach space into another
IITII = sUPllxll:511I T(x) ll, TEB,xEfJI
n cone in fff (ncontains a nonzero vector and if x E n,A.X E n, A. > 0).
for any matrix A = [aiJ, 1 :s; i :s; n, 1 :s; j :s; m, first subscript of aij denotes row position, the second column position, of the element
Real and Complex Numbers
complex numbers space of ordered real p-tuples, p > 1
xv
m, n, k, r, i, j generally denote integers
d(A, B) = infxEA,YEB [x - yl D(A, B) = SUPXEA,YEB [x - yl Np(a) = {z[lz - al < p}
oNp(a) = {z[lz - al = p}
Np(a) = {z[lz - al ::::;; p}
NiO) = N p
Sequences
boldface letters denote sequences, s, t, etc,
for any space d, d s denotes the space of sequences with elements in d; s = {s.} E ds, Sn E sf
de space of convergent sequences
d N space of null sequences, e.g., d a metrizable t.v.s.
e., «.. «; fJIlTM, fJIlTQ' 'CE=(r) special real and complex sequence spaces (see Sections 1.4, 1.5, 2.2)
related sequences (the space d must be such that the definitions make sense)
a:
r:
.1ks : {.1ks }. = .1k s. , k ~ 1
L' indicates first term of sum is to be halved
L" indicates first and last term of (finite) sum are to be halved
1 •T,,(f) = - L" f(k/n), /1 k=O
n ~ 1 (trapezoidal sum)
sequence relationships: let R be a binary relationship between members of two sequences x, y
x.R :Y. means x.Rv; holds for an infinite number of values of n
x.Ry. means x.Ry. holds for alln sufficiently large
this notation is used only when the sequence variable is n
Example
IA.kl s. 1 means: for some no, IA.kl ~ 1, 0 s k s /1, n > no
Functions
'I' class of real nondecreasing bounded functions on [0, 00) having infinitely many points of increase
'1'* subclass of 'I' such that LOOt' dt/J < CfJ, n ~ 0; t/J E '1'*
support of t/J is the set of points of increase
(Pochhammer's symbol)(rt)k = rt(rt + 1)··· (rt + k - 1)
I a l a~-I
1 a 2 a~ - I
k- 1 k
(van der Monde determinant)
k~1 (Hankel determinant)
XVlll Notation
R~a,p)(x) = p~a'P)(2x - 1), Jacobi polynomial shifted to [0, 1]
All other special functions are as defined in the Erdelyi volumes (1953),
Special Sequences
8 = 0,604898643421630
ao = 1, ak = k~ 1 + In(k ~ 1), k> 0,
8 = Y = 0.577215664901533
(EX 2)n: ak = (0,8t/(k + 1), 8 = 1.25 In 5
(EX 3)n: ak = (k + 1)(1.2)\ s divergent
(FAC)n: ak = (-ltk!, s divergent, but generated by
f'X)~ = 0.5963473611Jo t + 1
(IT 1)n: generated by 8n+ 1 = 20[s,; + 2sn + IOr 1 ;
8 = 1.368808107
(IT 2)n: generated by Sn+1 = (20 - 28'; - 8~)/1O;
s divergent
Numerics
Generally, in tables n SF representing a number is a rounded value; for instance,
n=3 n = 3.1 tt = 3.14 n = 3.142, '" .
For rational numbers, it may occasionally be important to know that the given value is exact. If that is the case, we write
~ = 1.5 (exact).
In definitions, all numbers are exact, e.g., s, = (1.18t, or it is indicated by ... that the number has been truncated.
Chapter 1 Sequences and Series
1.1. Order Symbols and Asymptotic Scales, Continuous Variables
Let 1/ and 1/' (see Notation) be equipped with pseudometrics d and d', respectively; let n be a cone in 1/ and ¢, IjJ E /T(1/, 1/').
¢ = O(IjJ) in n (1)
means for some M > 0 there is an R(M) > 0 such that
Further,
(3)
means for any e > 0 there is an R(e) > 0 such that
d'(¢, O)jd'(IjJ, 0) < e, XEn, d(x,O) > R. (4)
If ¢, IjJ depend parametrically on a E .sf and (2) holds for all a E .sf, then we shall write "¢ = O(IjJ) in n uniformly in .sf," and similarly for (3).
F or the foregoing definitions to apply, the implicit assumption is made that denominators are never zero; for example, there must be some R such that d'(l/J, O) oF 0, X E n, d(x, 0) > R. Thus anytime an order symbol is used, an implicit statement is being made about the zeros of d'(IjJ, 0).
The concept of asymptotic equivalence is often useful. This is written
in n and means both ¢ - IjJ = o(ljJ) and IjJ - ¢ = o(¢) in n.
(5)
2 I. Sequences and Series
Now let cj» E :YsC'f/, y'), where Y and 'Or' are linear spaces with pseudo- metrics d and d', cj» is called an asymptotic scale in Q if, for every k ;;::: 0,
in Q, (6)
and if this holds uniformly in k or uniformly in some parameter space, we speak of a uniform asymptotic scale (properly qualified). See Erdelyi (1956) for many examples.
Letf E :Y(Y, y'), A E res and cj» be an asymptotic scale in Q. The state- ment
in Q (7)
is to be read "f has the right-hand side as an asymptotic expansion in Q with respect to the scale cj»" and means, for every k ;;::: 0,
k
in Q. (8)
Often cj» is understood from context, so "with respect to the scale cj»" may be deleted from the definition.
Note that 0, 0, and -- are transitive and ~ is symmetric. Clearly no asymptotic scale can contain the zero vector or two identical vectors. If d' is a metric induced by a norm II ·11, the asymptotic expansion (7) is unique (but not otherwise). This is a simple consequence of the fact that 4> = 0(1]), l/J = 0(1]) then imply 4> + l/J = 0(1]). Thus assume another expansion (7) with coefficients A' holds. Setting k = °in (8) and its analog and subtracting the two gives
(A o - A~)4>o = 0(4)0),
or lAo - A~ I < s for all e, so that A o = A~, similarly, A j = Aj,j > 0.
1.2. Integer Variables
(9)
In discussions of sequences, the relevant variable x in 4> or l/J takes values in JO. We write 4>n or l/Jn for 4> or l/J, respectively, or, when there is a possibility of confusion with the index of an asymptotic scale, 4>(n) or l/J(n). 1.1(1) is then written
(1)
and means that for some M > 0, there is an N > °such that
d'(4)n, O)jd'Cl/Jn' 0) < M for n> N.
1.3. Sequences and Transformations in Abstract Spaces 3
A similar modification is made of 1.1(3). An additional complexity occurs when ¢ and t/J depend on a p-tuple with elements in r: say, n = (ml' m2' ... , mp) . It is usually important to know exactly how the elements mj become infinite, and it is hardly ever sufficient to say, for instance, that ml + m2 + ... + mp > N. In fact, the concept of a path in n-space becomes important (see Section 1.3).
1.3. Sequences and Transformations in Abstract Spaces
In this book we shall be concerned with two kinds of sequence transforma- tions. The first is the transformation ofagiven sequences E d sinto a sequence S E.r4's with, generally, a formula given to compute sn in terms of elements of s. (In some situations there is no explicit formula.)
The other case is where the given sequence s is mapped into a countable set of sequences S(k), k ~ 0, with a formula given (called a lozenge algorithm) for filling out the array {S~k)}, n, k ~ 0.
The whole point is to compare the convergence of the transformed sequence(s) with that of the original sequence. The most useful concepts are formulated in the definitions that follow.
Definition 1. Let s, tEAte, a metric space.
(i) t converges as s means d(sn, s) = O(d(tn, t)) and d(tn, t) = O(d(sn's)), (ii) t converges more rapidly than s means d(tn' t) = o(d(sn, s)).
(iii) The convergence of sis pth order if, for some p Er, d(sn+ I' s) = O(d(sn, s)")
and (I)
d(sn' s)" = O(d(sn+ I' s)).
It is easy to show that p, if it exists, is unique.
Definition 2. Let T E !!T(d, JIts) where d c JIte and T(s) = s. (i) Tis regular for d ifs Ed=> S E Ate and s = s.
(ii) T is accelerative for d (or accelerates d) if T is regular for d and S converges more rapidly than s, s e ss,
Definition 3. Let T E 5"(d, Jlts)whered c Jlt s . T sumsdifT(s) E JItc- SEd.
P = {(im,Jm)lim, JmEjO} IS called a path if io <l« = 0, im+ 1 ~ im, Jm+ 1 ~ i.; and either im+1 = im + lor Jm+ 1 = i; + I (or both). Increasing m assigns a direction along P. Obviously, i; + i; ........ XJ. Paths where i; is
4 I. Sequences and Series
n/k
(0,0)
(0,1)
(1,0)
(2,0)
1,\ )
( 3)
(4,3)
(5,3)
(6,3)
(3,4)
(44)
(54)
(2,5)
(3,5)
(4,5)
ultimately constant are called vertical paths, paths with t; ultimately constant diagonal paths. Figure 1 shows how the (n, k) position on P is labeled for illustrative purposes. Generally the (n, k) position of the diagram itself will be occupied by the (n + l)th component of the (k + l)th member of the set S(k), i.e., S~k). S~k) may converge as n + k --> 00 along certain paths but not along others. The following definitions contain the key ideas.
Let P be a path and <jJ(n, k), t/!(n, k) E §'(P, 1/') where 1/ is equipped with a pseudometric d.
<jJ = O(t/!) in P (2)
means for some M > 0 there is an N > 0 such that d(<jJ, O)/d(t/!, 0) < M for (n, k) E P, n + k > N. A similar interpretation is made of o.
104. Properties of Complex Sequences 5
Definition 4. Let vIt be a metric space, T E :Ysed, vIts) where d c vItc and let 7k(s) = S(k), k ~ 0.
(i) T is called regularfor d on P if sEd = d(s~kl, s) = 0(1) in P. (ii) T is called accelerative for d on P if T is regular for d on P and if
d(s~k), s)/d(sn' s) = 0(1) in P, s e se. (3)
If, in the foregoing definitions, d == vIt c- we shall omit the wordsror d and say simply that .r is regular, etc.
We now discuss certain computational aspects of the foregoing definitions. Usually To = I, the identity transformation, so s(O) = S and an algorithm that is computationally feasible for filling out the array {S~k)} will start with the values s~O) = s; and assign one and only one value to each (n, k) position in the array. There seems to be no easy characterization of those algorithms that are feasible in this sense. However, several important ones have been discovered recently. Among these are formulas of the kind
called a deltoid; and
s~O) = Sn, n, k ~ 0, (4)
S~k+ I) = H(S~k;/), S~k~ I' S~k», n, k ~ 0, S~-I) = 0, s~O) = Sn, n ~ 0; (5)
S~k+l) = H'(S~kL, S~k-l), S~k), n, k ~ 0, S~-lI = 0, s~O) = Sn' n ~ 0, (6)
called rhomboids. There is as yet no general theory for constructing such algorithms.
Those that are known have been derived using ad hoc arguments from diverse areas of analysis: Lagrangian interpolation, the theory of orthogonal polynomials, and the transformation theory of continued fractions. Much work remains to be done in this area.
For transformations in vector spaces, there are several important con- cepts that involve the linearity of the underlying space.
Definition 5. Let T E :Y(d, 11s) where d c; 11s- T is linear if, for all x, y E d and c l , C2 E '??, T(c1x + e2Y) = C1T(x) + C2 T(y); otherwise, Tis nonlinear. T is homogeneous if T(cx) = cT(x) for xEd, c E '?? T is trans- lative if T(d + x) = d + T(x), where d is a constant sequence (dn == d) whenever d + x, XEd.
1.4. Properties of Complex Sequences
When the metric space of the previous sections is the complex field, its sequence space possesses elegant properties. Some of these have been long
6 1. Sequences and Series
known, and others are surprisingly recent. This section contains a discussion of some of these results.
Definition 1. Let S E ~c and
rn+drn = (sn+ I - s)/(sn - s) = p + 0(1).
(i) If 0 < Ipi < I, s converges linearly and we write s E ~l'
(ii) If p = 1, s converges logarithmically and we write S E ~l"
Theorem 1. Let Ip I i= 0, 1. Then
(1)
n-e co Sn - S iff I' an + I1 IHl -~ = p.
n-e oo an (2)
Remark. For the divergent case Ipi> 1, S can be any number.
Proof The validity of either limit implies an i=. O. Assume, without loss of generality, an =f 0 for any n; otherwise delete the finite number of ans that are zero and relabel the members of a and s.
=: We have
an+1 ~ (p - 1)(sn - s), an = (p - 1)(sn_1 - s). (4)
Dividing the former by the latter shows
(4')
Note that for this part of the theorem p can be zero. =: We do only the convergent case 0 < Ipl < 1. The other is similar.
Since I an converges,
(5)
Let
Then g E ~N' Taking products in (6) gives n-I
an = aopn n(1 + G), j=O
empty products interpreted as 1. Define
Thus
(8)
(9)
(10)
. I r kl Ipl,+1 11m F" ~ L p - 1 _ I I n-e co k=O P
1 (1 I)> -lpr+ 1 + . - II - pi 11 - pi I - Ipl
For r sufficiently large, the right-hand side is >0. Thus lim F; > 0 and IIFn is bounded.
Now S -s au /00 00 /00n+ 1 '=. L ak+ 1 L a, = L akP(1 + Gk) L ak
Sn - S k=n+1 k=n+1 k=n+1 k=n+1
=. p + Un' (11)
(12)
(The foregoing operations are valid since it will turn out that s, of. s.) Thus 00
jUnl~· L lakGkl/lan+IJFn+1 k=n+1 00
~.Cgn+1 L Iplk(1 + gn+d k=O
Cg n+ 1
l-p(l+gn+I)'
which actually shows a bit more, namely,
Sn+1 - S = P + o(suplak+1 _ II).• Sn - S k>n pak
(13)
(14)
Corollary. Cfla :=; Cfl/.
limlanl 1
/ n slim lan+1/anl .• (15)
Another useful result has to do with the order of growth of partial products.
Theorem 2. If
n ~ 1, Vo = 1 n-l
», = Il (1 + G), j=O
for some t E CflN, Gj i= -1,j ~ 0, then there is an t* E CflN such that
Proof We have
j=no (18)
(19)
but the quantity in square brackets is the Cesaro means of a null sequence, and hence the nth term of a null sequence, say, <5n • So
(20)
and this may be extended to all n ~ 0. (s", because of the multiple valuedness of log, is not unique.) •
1.5. Further Properties of Complex Sequences
Some unusual convergence properties have recently been demonstrated for complex sequences. These properties are a help in determining whether important sequence transformations are regular or accelerative. Sources for this material are Tucker (1967, 1969).
In what follows let s, a E Cfls and be related in the usual way. For all an i= °and n ~ 0, define Pn by
and if s is convergent, r, by
Tn = (s - sn)/an· Otherwise Pn' r, are undefined.
(1)
(2)
1.5. Further Properties of Complex Sequences 9
Since we shall in general be concerned only with members of a sequence with large index, the notation "xnR.Yn" (see Notation) will be employed constantly.
Lemma 1. Let S E C(/C and an#. O. Let c E C(/, and define
then Cn = C + (sn - s), n ~ 0; (3)
Proof
( I - Pn) c, Cn+ 1 I - Pn (1 + C -- + - - -- =.-- S - Sn)' an+ 1 an an+ 1 an+ 1
1 ( 1 - Pn) Cn Cn+ 1+C -- +---- an+ 1 an an+ 1
=.1 + c(_I__~) + C + Sn - S _ C + Sn+l - S
an+ 1 an an an+ 1
=. 1 + S - Sn + 1 _ S - Sn =. S - Sn _ S - Sn
an+ 1 an an + 1 an
(4)
• (5)
Then S diverges.
Proof Assume s converges. Since (1 - Pn)/an+ 1 is bounded, there is an 8> 0 such that 18(1 - Pn)/an+ 11 <.l
Let C be any complex number satisfying Ic I = 8, so that
- Re[c(1 - Pn)/an+ 1] <·l (7)
Set Cn = C + (s, - s). From the previous lemma
Re[1 + c(1 - Pn) + Cn _ Cn+l] =. Re[1 - Pn(s - Sn)], (8) an+ 1 an an+ 1 an + 1
so
(9)
10 1. Sequences and Series
Using (7) and (9) shows
~ + Re c" <. Re C,,+ 1 _ Re[C(l - P,,)] _ ~ <. Re C,,+ 1 (10) 2 a" a"+1 a" +1 4 a"+1
from which it follows that Re c.fa; -+ 00 and so Re c"/a,, >. O. Since C" -+ c,
a,,¢. {z larg c + 3n/4 ~ argz ~ argc + 5n/4}. (11)
Choosing arg c to be successively 0, n/2, n; 3n/2 shows that a cannot be a complex sequence, a contradiction. •
[This beautiful proof is due to Tucker (1967).] We state without proof a similar result for infinite products.
Theorem 2. Let
Lemma 2. Let P" be defined ultimately and
Ip,,1 ~.p < t· Then
Proof Note that r" is, ultimately, defined and that
r" =. P" + P"P,,+ 1 + P"P,,+ 1P"+2 + "', since r,,+1 + 1 =. r,./p" and the above series converges. We have
(15)
Ir,,1 ~·lp,,1 + Ip"P"+11 + ... ~·lp"I/(1 - p) ~. p/(1 - p) < 1. (16)
Thus I»J», I s. 1/(1 - p) and
1iJ», I =.11 + r"+11;::::.ll -lr"+lll =.1 -lr"+11 ;::::.1 - p/(1 - p) = (1 - 2p)/(1 - p) > 0.. (17)
Theorem 3. Let s, s* be two sequences such that a:/a" = 0(1) and Ip,,1 ~.p < t, Ip:1 ~.p* < 1 for some numbers p, p*. Then s* converges more rapidly than s.
Proof An implication of the hypothesis is a" #-.0, a: #-.O. The pre- vious lemma shows
0<(1 - 2p)/(1 - p) ~·lr,,/p,,1 (18)
1.5. Further Properties of Complex Sequences I I
and (19)
One concludes that
IS: - S* I=.1 a:+ 1 II ':/P: I Sn - S an+ 1 'n/Pn
~.la:+ll[(1 -p*)(l - 2p)(1 - p)r 1 = 0(1). • (20) an + 1
Tucker gives an example (1967, p. 358) to show that 1- cannot be replaced by a larger number.
Lemma 3. Let b, S, s* E rrls with
(21)
Then s* converges more rapidly than s to the same limit if and only if
bn + 1 ~ S - s, = 0(1). (22) Proof Either hypothesis implies the convergence of sand S - s, #. O. In
either case, therefore,
and from this the lemma follows. •
Theorem 4. Let t, s E rrlc and
(23)
(24)
and suppose t converges more rapidly than s to the same limit. Then u converges more rapidly than s to the same limit if and only if {In ~ (J.n'
Proof From the previous theorem, an+l(J.n+ 1 ~ S - Sn = 0(1). Also u converges more rapidly than s to the same limit if and only if an+ l{Jn+ 1 ~
S - Sn' Since S - Sn #.0, we have an+l(J.n+l #.0, an+1{Jn+l #.0, and so an' (J.n' {In #. O. By transitivity of ~ we conclude an+ 1 (J.n+ 1 ~ an+l{Jn+ 1 or (J.n ~ {In' This step is reversible, so the theorem follows. •
Theorem 5. Let S be convergent and
(25)
Then the three conditions below are all equivalent:
(i) s* converges more rapidly than s to the same limit; (ii) (J.n+ 1 ~ 'n/Pn;
(iii) (J.n ~ 1 + Tn'
I 2 1. Sequences and Series
Proof From Lemma 3, s* converges more rapidly than s iff an + 1(Xn+ 1 ~
<r« -+ 0; this is equivalent to (Xn+1 ~ -rnlan+1 = ,jPn' Moreover, (Xn+1 ~
'nlPn is equivalent to (Xn ~ 1 + 'n since 'nlPn = 1 + 'n+ l' •
1.6. Totally Monotone and Totally Oscillatory Sequences
Definition. s is totally monotone (written s E ~TM) if
(-I)kNsn~O, «i :« (I)
s is totally oscillatory (written s E 9fT O) if {(-I)nsn} E 9fTM •
Here
(2)
so s converges since it is monotone decreasing and bounded. On the other hand, if s E 9fTQ, S is alternating and so converges to O.
Examples. The sequences S(k) E ~TM' where
s~1) = I/(n + 2), (X > 0, (3)
since
(4)
(see Theorem 3).
~TM is an important regularity space for certain nonlinear transformations in ff(~s, ~s).
Theorem 1. Let s E ~TM' Then the sequences whose nth elements are given below are also E ,'3iPT M • (Empty products are interpreted as 1.)
(so < 1);
(ii) nj:6 (1 - s) (so:s: 1);
(iii) A.(-I)k+l~ksn (0 < A.:s: 1, k > 0, a, k fixed);
(iv) A.J:~lSj (0 :s: A.:s: 1).
Proof The proofs are straightforward. We prove here only (i); the reader is referred to Wynn's paper (1972) for the others. Write
1/(1 - sn) = tn· (6)
Multiplying both sides by 1 - s; and using the difference formula 1.6(40) gives
(-~)ktn = (1 - sn)-I ±(~)(-~)k-jtn+i-~)jSn. (7) j= 1 )
Now s, - Sn+ 1 ?: 0, so °:S: s, < 1 for all n, and thus Eq. (7) provides an immediate induction argument on k. •
Theorem 2. Let S, t E BfTM • Then {sntn}, {asn + btn} E BfTM , a, b > 0.
Proof Obvious. •
(8)n ?: 0.
Theorem 3. S is totally monotone if and only if there is a function t/J(t) bounded and nondecreasing on [0, 1] that satisfies
s; = ftndt/J(t),
( -1)k~ksn = f(1 - t)ktn dljJ(t) ?: 0.
=: For all k and °:S: n :S: k, (-It-n~k-nsn ?: 0,
(9)
(10)
or
°:S: n :S: k, (11)
°:S: n :S: k,
where r E Bf~. It is easily seen that this system of equations has the solution
_ k (k - n) _ k m(m - 1)··· (m - n + 1) sn - L r; - L L m,
m=n m - n m=n k(k - 1)··· (k - n + 1) (12)
14 I. Sequences and Series
where
(13)
. 1(1 - llk)(1 - 2Ik)··· [1 - (n - l)jk]
= t" + O(k- 1) (15)
uniformly in t, and l/Jk(t) is the step function defined by
t ::; 0, °< t ::; 11k,
(17)°s t ::; 1.
Lo + L 1 + + Lk - 1,
Lo + L 1 + + Lk ,
So = l/Jk(1) :2: l/Jk(t) :2: l/Jk(O) = 0,
But any sequence of bounded nondecreasing functions on [0, 1] contains a subsequence converging to a bounded nondecreasing function [see Wall (1948, p. 246)]. It is easy to justify taking the limit over this subsequence inside the integral sign (Wall, 1948, p. 245), so for some bounded nondecreas- ing l/J(t),
s, = fr dl/J(t), n :2: 0. • (18)
For S E fJils , the determinants
sn Sn+ 1 Sn +k- 1
H~k)(s) = Sn+ 1 Sn+ 2 Sn+k n, k :2: 0, (19)
Sn+k-l Sn+k Sn+ 2k - 2
are called Hankel determinants.
1.7. Birkhoff-Poincare Logarithmic Scales 15
Theorem 4. If s E 3lTM, H~k)(S) 2: O. If s E ~TO, (_l)nkH~k)(s) 2: O.
Proof Let
k-1 I1 Q~k) = L sn+i+A¢j = tn(~o + ~lt + ... + ¢k_ltk-1)Z dt/J(t)
i.j~O 0
2: O. (20)
But this means by a known result on quadratic forms (Bellman, 1970, p. 75) that the determinant of the coefficients of Qin) must be nonnegative, which gives the first part of the theorem. The second is similar. •
Theorem 5 (Brezinski). Let f(x) = Lk~O CkXk be a power series with nonnegative coefficients and radius of convergence p > O. Let s E 3lTM and So < p. Then {f(sn)} E 3lTM •
Proof Obvious, since the sequence S(k) E ~TM when
S~k) = Co + C1Sn + '" + CkS~
by Theorem 2, and any limit of totally monotone sequences is totally monotone. [This also provides a proof of Theorem l(i).] •
Theorem 6. If S E ~TM' then
H~k)(~2rs) 2: 0 and
If S E ~TO' then ( - 1)knH~k)(LlZrs) 2: 0
Proof Obvious. •
1.7. Birkhoff-Poincare Logarithmic Scales
Let p E J+, Ill' Ilz, ... , II p, ebe complex constants, Iloan integral multiple (positive or negative) of lip. Define
Q(w) = lloW In w + 1l1W + IlZW(p-1)/P + ... + IlpW1/p, WE 3l+. (1)
Consider the sequence of functions
t/J;jw) = eQ(W)w6 - j/P(ln wy, i,j = 0, 1,2, ... , co E ~+, (2)
and let F = {t/Ji,j}' It is easily verified that F is strictly (nonreflexively) well ordered under the operation written" </J = o(t/J) in «:: Even so, one may not be able to rearrange the elements of F into a scale. However, if i is
16 1. Sequences and Series
bounded, i = 0, 1, 2, ... , p, then one may define a unique asymptotic scale on F, say, {<Pn}, with
<Pn = I/Ii", k"' (3)
This scale is called the Birkhoff-Poincare logarithm scale (B-P log scale); p is called the index of the scale. If p = 0, it is called simply the Birkhoff- Poincare scale. The ~pecial case J1; = () = 0, p = 1 is called the Poincare scale.
Any function satisfying a fairly general difference equation or differential equation is known to possess an asymptotic expansion in a B-P log scale, or, more precisely, the function can be written as a linear combination of such expansions, once it is decided how to interpret sums of asymptotic ex- pansions. [In fact, this can easily be done; see Wimp (1974b).] For difference equations, this is called the Birkhoff-Trjitzinsky theory (1930, 1932),and for differential equations, the theory of subnormal forms. [See Wasow (1965) and the references given there.]
Theorem 1 (Birkhoff-Trjitzinsky). Consider the difference equation Ao(w)y(w) + Al(w)y(w + 1) + '" + Am(w)y(w + m) = 0, (4)
where Ai is defined for w E ~o and Am(w) i:- 0. Let Ai have an asymptotic expansion with respect to some B-P scale F
with Q = J1j = () = °in ~+. Then there is a B-P log scale G and a basis of solutions Yl' Y2'" ., Ym of the equation such that Yj has an asymptotic expansion gi with respect to G in ~ +.
The general form of these solutions is
y(w) = eQ(W)w8[(aoo + aOlw- l/p + ...) + (a lO+ allw- l/P + .. ·)lnw + '" + (amo + amlw- l/ P + .. -)(In co)"], (5)
Proof The proof is the subject of two papers. The first (Birkhoff, 1930) treats the formal (constructive) theory of the question; the second (Birkhoff and Trjitzinsky, 1932) treats the analytic theory. •
While the theorem is simple to state, in the construction ofthe asymptotic expansions there are many complexities. For instance, p for G and F need not be the same. Further, once certain expansions gi are obtained others may be found from these by formal manipulations, thus vastly simplifying the work involved.
For example, the difference equation
() (w + 1)[(2w + b + c + 1) + A] ( 1) yw - yw+
(w + b)(w + c)
(w + 1)(w + 2) + (w + b)(w + c) y(w + 2) = 0, b, c > ° (6)
1.7. Birkhoff-Poincare Logarithmic Scales 17
has solutions r(b + w)r(C + w)
h1(w) = r(w + 1) 'P(b + w, b + 1 - c; A),
r(w + b) hz(w) = f(w + 1) <J)(b + os, b + 1 - c; A)
(see Wimp, 1974a). There is a formal basis of solutions
(7)
00
gj(w) = exp[( -IY+ lAl/2Wl/2]W9 L Ek( _lykw - k /2, k=O
and
h1(w) '" fiA(e-b)/2-1/4e~/2g1(W) in ~+. (9)
Here the A i are rational; p = 1 for F; for G, P = 2. Let us see how the construction proceeds for the important case m = 1.
For the formal computations, assume all series are convergent for w > R, say. Then Eq. (4) can be written
yew + 1) = w/i/P(aoe/i/P + a1w- 1/P+ .. -)y(w), f.1 E J, ao # O. (10)
Let
z(w + 1) = b(w)z(w), b(w) = 1 + b1w- 1 /P+ .... (12)
Finally, let u(w) = In z(w). Then u satisfies w- 2 /P
u(w + 1) - u(w) = In b(w) = b1w- 1/P+ (2b2 - bi) 2 + ... = C1W- 1/P + C2W-2/p + .... (13)
In this equation write
+ de In w + d1w- 1/p + d2w- 2/P+ "', (14)
and it is immediately found that d1 - P ' d2 - P " ' " do are uniquely determined, with
(15)
18 I. Sequences and Series
by comparison of terms w- I /P, w-z/p, ... ,w- I • On comparison of terms w- k /p , (k > p), one gets equations of the form
k :::: p + I, (16)
in which IY.k is a known polynomial in d ,_ p, dz-p, ... ,dk-I-P' Thus all the d, are determined in succession and uniquely. Then, writing
(17)
and exponentiating the series for u(w) gives the desired formal series. This construction, of course, shows that a unique formal asymptotic series always exists, and also that, for the first-order case, p for F = P for G and the index of G is zero.
The next result shows how the partial sums of a series grow when the general term of the series has an asymptotic expansion of the form eQs.
Theorem 2. Let
an'" eQ(W)wlJp(w), co = n + ( In J+ (19)
where ~ is arbitrary and complex and where
pew) = IY.0 + IY.IW- I/P + IY.zW- z/ P + .... Then
s, - S '" eQ(W)w 8' p*((I))
where ()*, (;(6 are as follows:
Case I. Q"¥= O. Denote the first nonzero fl j in the seq uence
by u., Then
r=O T = I;
(22)
(23)
Case II. Q == O. ()* = 8 + I, IY.6 =':1.0/(8 + I). (24)
1.7. Birkhoff-Poincare Logarithmic Scales 19
Proof. A straightforward application of Theorem 1; see Wimp (1974b) (whose r = 0 values are incorrect). •
Also of interest is a related result for the partial sums of divergent series.
Theorem 3. Let s ~ rcc and let an' p, to be as in (19), for some constant c. Then
in r: (25)
where lJ*, O(~ are as in (22) and (23) except in the following cases:
Case I. llo;6 O. Then O(~ = 0(0' lJ* = lJ.
Case I I. Q == 0 and p(w) contains a term w - I. Then for some c, d,
S,,- C + din w -- W6+ lp*(W)
Corollary. Let
l An+Ino ( /31 /32 )-- IXo+-+-+"',(A-I) n n2
Sn - S --
-n 6
and
(26)
(27)
(28)
Sn -- iXo In n + lJdn + lJ 2/n 2 + ... , A = I, lJ = -I. (29)
As examples of the use of these formulas, consider the computation of e' and (s) from their defining series. Let
For Sn'
k = 1
IXj = 0, .i> 0, (32)
20 I. Sequences and Series
and (for both may be taken to be O. We have
(33)
(34)
Higher coefficients are easily determined by formal series manipulations [see Smith (1978)]:
PI = 1, Pl = X, P3 = Xl - 2x, (35)
For the Riemann zeta function,
~ 1 I "( (Xl (Xl )((a) - L. a '" n - (Xo + - + 2 + ... , k= I k n n
(36)
where
k '2. 1,(ahk-l (Xlk = (2k)! Blk>
the higher coefficients being obtained by the formula in Wimp (1974b, 2.42) or from the integral representation for ((a). Equation (36) is known to hold for all complex a =I- 1 [see Olver (1974, p. 292)].
In what follows let
(39)
be a formal asymptotic series. In future sections we shall need to know the effect of certain difference operators on this series.
Lemma 1
Proof See Milne-Thomson (1960, p. 35). •
(40)
Lemma 2
APy(n)= (-OM-l)Pno-P(lJ(o+Ydn+ ... ), A=l, 0#0,1,2, ... ,p-1, (41)
for some Pi' Yi'
Theorem 4 iLdijA)AP+ry(n) = An(A - 1)P( -l)i( - O)ino-i(lJ(o + lJ('dn + ."),
r=O
where IJ('J' IJ(~, .••• depend on j and p and
di.rCA) = C)A-i(1 - A)i- r.
A-nAPy(n) = (A - l) PnO(lJ(o + Pdn + " -).
(42)
(43)
(44)
Then letting v(n) = A-n and u(n) = APy(n)and using the two previous lemmas gives the theorem. •
The paper by Wimp (1974b) includes a number of applications of the previous results, particularly to the problem of finding asymptotic formulas for the remainder terms in expansions in orthogonal polynomials.
Brezinski has shown that something like Theorems 2 and 3 is true for series of arbitrary real terms provided the terms are ultimately positive and that their differences are ultimately of one sign.
First, note that Un ~ Vn is equivalent to the statement that u; #.0, Vn #.0, and
Lemma 3. Let an >.0, s diverge to + 00, and b; = 0(1). Then n
L akbk = o(sn)' k=O
Proof n
(45)
(46)
(47)
where
(48)
by the Toeplitz limit theorem, Theorem 2.1(3). Note b may be a complex sequence. •
Theorem 5. Let S E ~s, an >.0 and h; = an/l1anwith I1hn = 0(1).
Case I. Sa; <. O. Then s converges and
(49)
Proof
Case I.
n-I (n-I ) h; - ho = k~O I1hk = 0 k~O 1 = o(n),
by the lemma (with s = {n}). This means
[n(an+ dan - 1)r 1 = 0(1),
(50)
(51)
(52)
or, since an+ dan <. 1, the sequence {n(an+ dan - I)} is definitely divergent to - 00. By Raabe's test s is convergent. Thus
Now
00 00 00
L ak = L hkl1ak = -hnan - L akl1hk-t· k=n k=n k=n+1
(53)
I f akl1hk- l ! S sup Il1hkl(s - Sn) s.sup/l1hkl(s - Sn-I)' (54) k=n+1 k~n k~n
Thus 00L akl1hk_1 =(s - Sn-I)¢n, (55)
k=n+1 where ~ E ~N' so (53) may be written
S - Sn-I = -hnan - (s - Sn-I)~n,
or, since S - Sn =1=.0, -hnan/(s - Sn-I) =. 1 + ~n
and letting n -> CJJ gives the result.
(56)
(57)
Case II. Sincean+1 >.an'Sn--' +00. Also, n n
Sn = L hk~ak = hnan+ I - hoao - L ak~hk-I' (58) k=O k=1
or n
ao + Lak(l + ~hk - I) = h; an+ I - hoao . k=1
But, by the lemma, n
L ak(l + ~hk-I) = sn(l + ~n) k=1
where; E fYtN . Since s; =1-.0,
hnan+ Js, =. 1 + ao(l + ho)/sn + ~n,
(59)
(60)
(61)
and letting n --. 00 gives the result. •
Sometimes the variable appearing in a Poincare series is n + f3 rather than n. This is immaterial, however, as the following result indicates.
Theorem 6. Let 00
Then, for any f3 E 'fl,
e, «e «. (62)
(63)
Proof
k k (a _ f3)O-' L c,(n + at-' =. L c,(n + f3)O-' 1 + --f3
,=0 ,=0 n + k 0 00 (f3 - ay-'(r - (})s-,
=. LcrCn + f3) L (- )'( f3y ,=0 s=, sr. n +
=. i>,(n + f3t ±(f3 - aY-,'(r - (}):-, + O(nO- k- l ) ,=0 s e r (s - r).(n + f3)
k
and from this the theorem follows immediately. •
(64)
2.1. Toeplitz's Theorem in a Banach Space
The most famous result dealing with the regularity of linear transforma- tions is the Toeplitz limit theorem. In its classical guise, this concerns the convergence of transformations of (6's where the (n + 1)th member of the transformed sequence is a weighted mean of the first n + 1 members of the original sequence:
n
(1)
The theory of this transformation is covered quite adequately in the existing literature (Knopp, 1947, Hardy, 1956; Petersen, 1966, Peyerimhoff, 1969).
For what follows, we shall need an abstract version of the theorem. This, in a way, is fortunate, since the proof is cleaner than the proof of <i&'s , which is rather computational. However, we shall have to begin with two lemmas concerning linear operators in Banach spaces.
Let f18j, j = 1,2,3, be Banach spaces and Pnk' (nk be sequences of linear operators,
(2)
lim lim (nk(Y)
exists for every Y E &13 , then the linear operators
(n(Y) = lim (niy) k--+ 00
satisfy
(4)
n z o. (5)
Lemma 2. If II (nkII :::;; M, n, k z 0, and
lim lim (nk(Y) = ((y) n-oo k-oo
(6)
exists for all Y in some dense subset W c &13 , then ((y) exists for all YE&l3 and (E B(&l3,&l2 ) , with 110 :::;; M.
We omit the proofs of these lemmas, which are straightforward double applications of standard functional analysis results [see Banach (1932, Theorems 3 and 5); Zeller (1952)]. (All limits above, of course, are in the norm topology.)
Now let s be a convergent sequence in fJd\ i.e., s E &I~ and let S E 88~ with 00
s, = 2: J.lnk(Sk) k=O
(7)
subject to certain convergence considerations, which we shall examine presently.
T(s) = s is called a generalized Toeplitz transformation.
Theorem 1 (Toeplitz Limit Theorem). The sum (7) converges, n z 0, and SE &12 iff
(i) 112:~=o J.lnis) II :::;; M for Ilsjll :::;; 1,j z 0, and n, k z 0; (ii) 2:;;'= 0 J.lnk(Y), n z 0, and limn --+ 00 2:;;'= 0 J.lnk(Y) exist for Y E f1l1 ;
(iii) limn --+ 00 J.lnk(Y) exists for Y E &11, k z O.
If (i)-(iii) are satisfied, 00 00
s = lim 2: J.lnk(S) + 2: lim J.lnk(Sk - s). n-co k=O k=O n-e co
Proof To apply the lemmas, put f!J3 = f!J~ with the usual norm
[x] = sup[x.], n
(8)
(9)
(10)
26 2. Linear Transformations
For W, the dense subset of ~3, we pick all finite linear combinations of sequences of the form
{c, c, c, ...} and {O, 0, 0, ... ,0, b, 0, ...}. (11)
This works since for any x E f!J~,
{x, x, x, ...} + {xo - x, Xl - X, X2 - X, ... , Xk - x,O, 0, ... }~ x, k -> 00. (12)
=: The necessity of (ii) and (iii) is immediate. For (i) note that each sequence of the form
(so, Sl, S2' ... , Sk, 0,0, ...) (13)
belongs to f!J~ and has norm [s] :::;; 1. =: Consider C E W, Cj = c. Statement (i) shows that condition 1 of
Lemma 2 is satisfied and (ii) guarantees the second. For (0,0, ... , 0, b, 0, ...) E W, condition 1 is clearly satisfied and (iii) guarantees the second. Thus s converges and T(s) = s defines a continuous linear transformation from f!J~ tof!Jl:. (8) is obvious. Equation (7) shows
II TIl = sup supll f Jlnk(Sk)ll· • (14) Ilsll=1 n k=O
Theorem 2 (Toeplitz Regularity). T(s) = s is regular for f!Jc iff
(i) III'=o Jlnj(S)II :::;; M for Ilsjll :::;; 1,j 2: 0, n, k 2: 0; (ii) Ik'=o JlniY) = Y + 0(1), n 2: 0, Y E f!J;
(iii) Jlnk(Y) = 0(1) in n, k 2: 0, Y E~.
Proof
=: Obvious. =: We determine the limits ii, vof the two sequences
u = (u, u, u, u, ...), v = (0,0, ... , 0, v, 0, ...) (15)
where v in the second is in the (K + l)th position. Applying Eq. (8) yields 00
ii = lim I Jlnk(U), n-e co k=O
v = lim JlnK(v).
Requiring ii = u and v= °finishes the proof. •
Any transformation (7) defined by a matrix {Jlnd of linear operators is called a generalized Toeplitz summation method.
2.2. Complex Toeplitz Methods 27
2.2. Complex Toeplitz Methods
flnk E~. (1)
[
°flll fl21
then U, or the transformation defined by U, T(s) = s,
(3)
n
n = 0, 1,2, ... (4)
is called a triangle. We now restate the Toeplitz limit theorem in a form suitable for U in Eq. (2).
Theorem 1. U is regular iff
(i) Li:=o Iflnkl ~ M; (ii) LI:=o flnk = 1 + 0(1);
(iii) flnk = 0(1), k fixed.
Proof (ii) and (iii) are obvious. Condition (i) of Theorem 2.1(2) produces ID=o flnjsjl ~ M for Isjl ~ 1, but for n fixed, there is an s for which this maximum is attained, namely, Sj = sgn flnj' The smallest M that will do is, in fact, the norm of T, and
IITII = sup M n , n
n
(5)
Of course, if U is a triangle, condition (ii) can be deleted. A method U satisfying (i)-(iii) is called a Toeplitz method. If flnk ?: 0, U is called positive. Complex Toeplitz methods are very useful
and, when applied to the right sequences, can greatly enhance convergence. Because of their numerical stability, positive methods are the most frequently used ones.
28 2. Linear Transformations
Real positive Toeplitz triangles (even triangles that are "nearly" positive) have an important limit-preserving property; i.e., negative elements appear in only a finite number of columns of V iff
(6)
jar all real bounded sequences s [see Cooke (1955, p. 160)]. One cannot expect too much from any linear summability method. The
improvement in convergence is, in general, no greater than exponential; in other words, (s, - s)j(sn - s) = O(t), 0 < y < 1, and one cannot find a method, at least a positive triangle, that is accelerative for all convergent sequences. To see this, let V be such a method and s a monotone decreasing null sequence.
Then
(7)
Pennacchi (1968) has shown that no method of the form p
s, = L JijSn-p+j, j=O
(8)
(9)n ~ 0,
where the Jij are independent of n, can be accelerative for all sequences. (The foregoing is a band Toeplitz process with constant diagonals.) A minor modification of his proof permits the generalization that no band Toeplitz process can be accelerative for all Cf/c. Whether any Toeplitz method can be accelerative for all Cf/c is an important open question.
There are many triangles that sum divergent bounded sequences, but it is a consequence of the Banach-Steinhaus theorem that no regular triangle can sum all bounded sequences (Schur, 1921).
The polynomial
A ~ k On (A - Ank) Pn( ) = L- Jink A = 1 _ A '
k=O k=l nk
is called the nth characteristic polynomial of the triangle U. The regularity and accelerative properties of V are intimately connected with the location of the complex zeros Ank of Pn(A).
A useful function, called the measure of V, is
(10)
2.2. Complex Toeplitz Methods 29
and K is called the modulus of numerical stability of U. When U is regular, K = 1.
Let '{JEm(r) C '{Js denote the space of all exponential sequences of the form
sn = S + cdi + czYz + '" + cmY;:', (12)
where cj =1= 0 is complex and Yj E I",a nonempty compact subset ofthe complex plane not containing O. We assume the Yj are distinct.
As the following theorem shows, the properties of the measure of U determine whether or not U is regular and accelerative for this important class of sequences.
Theorem 2. Let U be a triangle with measure 0"(..1.), .s4 = '{JEm(r).
(i) Let 0"(..1.) =1= I, AE r. Then U sums .s4 with s = s iff 0"(..1.) < 1, AE r. (ii) Let 0"(..1.) =1= A, AEre N. Then U is accelerative for .s4iff 0"(..1.) < IAI,
AE r. Proof The basic inequality from which these statements follow is
(0"(..1.) - e)" <: IPiA) I <. [0"(..1.) + e]", (13)
for any s > O. To show sufficiency in (ii), for instance, choose an s so that the above holds for Z = Yl, Yz,.··, Ym' and let y* = suplv.]. Next pick 1 > b > 0 so that
One has
8n - S \ CI8"-sl C~ Icrl[IYrl(l-b)+e]" ~. *n <. 1... *n
5" - 5 Y r= 1 Y
<. C(1 - b + e/y*)" for every e > O. (15)
But this must hold for every s > 0, 15, y* being fixed. Thus
1(8" - S)/(5n - s)1 = O(r") for some r, 0 < r < 1, (16)
which shows not only that U accelerates convergence of each sequence in .s4 but does so exponentially.
Remaining details of the proof are left to the reader. •
If something is known about the zeros of PiA), one can say something about the kinds of exponential sequences U sums. In what follows, define the distance functions d, D for sets C'{J by
d(A, B) = inf lz - w], zEA WEB
D(A, B) = supiz - w], zEA WEB
(17)
and let A = Pond. Then clearly,
d(r, A)/D({I}, A) S; 1O'(A) 1 S; D(r, A)/d({I}, A), AEr, (18)
so if the maximum distance of r to {And is less than the minimum distance of {l} to {AmJ, U sums C6'Em(r). Also O'(A*) = 0 only if A* is a limit point of {And, provided the latter is bounded.
The regularity of U is particularly easy to characterize when Ank does not depend on n. These are the so-called (j, -Ak ) means (Jakimovski, 1959).
n ~ 1,
Theorem 3. Let the triangle U be defined by Po = 1 and
n A - AkPiA) = n~, k= 1 k
(19)
where Ak =f= 1.
(i) Let 0 <. Ak < M; then U is regular iffL Ak converges. (ii) Let Ak <.0; then U is regular iff L Ak 1 diverges.
Proof We prove statement (i) first. Let the last Ak S; 0 be Am' Clearly, the regularity of the triangle associated
with PiA) is unaffected if each factor (A - Ak)/(1 - Ak) in Pn(A), 1 S; k S; m, is replaced by, say, 2A - 1.Thus we can assume, without loss of generality, that Ak > O. Note also that already
n
I }ink = Pn(1) = 1. k=O
=>: Since Ak > 0, the coefficients in Pn(A) alternate in sign. Thus
n 1 1 + AI
nIPi-l)1 = n 1- ,k = I l}inkl < M. k= 1 /l.k k=O
(20)
(21)
=:
2.2. Complex Toeplitz Methods 31
But the convergence ofL Ak guarantees the convergence of the above product, since 0< Ak < M (Knopp, 1974, p. 274). Thus
Iltnkl :$; ARk- n (24)
and
/I A(R n + 1 - 1) AR k~olltnkl:$; Rn(R - 1) < R - l'
and letting n --+ 00 in (24) finishes the first part of the proof. For (ii), let -Ak = 'k > 0, without loss of generality. Note LIltnkl =
L Itnk = 1. =:
n ( I)-IItno = n 1 + - . k= 1 'k
If Itno --+ 0, then L ,;; 1 must be divergent, since the product in (26) must diverge (to zero).
<=: As before
1 Iii n (t + 'k) IIltnkl = - k+1 n -- dt 2n Itl=et k=1 1 + 'k
0<8<1. (27)
The product diverges as n --+ 00 if L (1 + 'k) - 1 is divergent. But, as is easily seen, this is true for L ,;; 1 divergent. Thus the product diverges and obviously to °since its terms are < 1. This means Itnk --+ 0, so V is regular. •
For this very simple class of triangles, the measure can be computed explicitly.
Theorem 4. Let the triangle V be defined by Po = 1 and
n (A - Ak ) Pn(A) = )J
1 1 - A
where Ak i= 1, k ;:::.: 1, and Ak = 0(1). Then
a(A) = 1.11. Proof
n ;:::.: 1, (28)
32 2. Linear Transformations
or the left-hand side is the exponential of the Cesaro means of the sequence s~ = lnl(A - An)/(1 - An)l. This sequence is convergent-in fact, s* = In IA1- .,,,rl the theorem results. •
Example 1. Let Ak = (J-k, (J > 1, U is then the triangle corresponding to the Romberg integration procedure (see Section 3.1).
In the next result, the zeros of Pnare allowed to depend on n.
Theorem 5. Let Pn(A) be as in (28) but with A = Ank' Furthermore, let all but a finite number of the AnkE~, which is a compact subset of {Re z :s; O}, and let
n
(31)
(32)
Then U is regular. Furthermore, if ~ = [ -a, OJ, a > 0, (31) may be omitted.
Proof Using the previous contour integral, we find
k n nn 11 - tAnj Il!lnkl :S;.R - sup 1 _ A JII=Rj=l nj
= RkfI [(1/R - Re Anif + (1m AnjtJ1/2. j=l (1 - Re An) + (Im An)
But since Re Ani' 1m Anj are bounded and Re Anj :s; 0, each term in the product is less than or equal to 1'/ < 1, so
in n. (33)
M n = Pn(1) = 1. • (34)
Example 2. The case where {Pn} is a system orthonormal with respect to a distribution function t/J E 'P with support (that is, the set of points of increase) in [ -a, OJ is very important (Section 2.3.6).
Of course, it is also important to know when a method is not regular.
Theorem 6. Let AnkE. [0, 00J, k 2': 1,and let m(n) of these be bounded and bounded away from zero, m(n) -- 00 as n -- 00. Then U is not regular.
Proof
(35)
(37)
Theorem 7. Anyone of the n + 1 conditions below is necessary for U to be regular:
n
n
l~p~n,
(38)
where Sp is the pth symmetric function of the roots of Pn(A).
Proof Obvious. •
Definition. Let U be a triangle. U is said to be equivalent to convergence iflimn~oo s; = S ifflimn~oo sn = s. [Note that this definition requires s; (or sn) to exist only for n sufficiently large.]
Triangles equivalent to convergence are, generally speaking, pretty weak computationally-they, as it were, try to do too much. The triangle U tends to be heavily weighted toward the diagonal [flii] and so gives excessive weight to the latest member used in the sequence {s.}. But the latest member of the sequence carries very little information. The following criterion is due to Agnew (1952).
Theorem 8. Let U be a regular triangle and
lim (2flnn - Mn) > 0. n-e oo
Then U is equivalent to convergence. Proof See Agnew (1952). •
2.3. Important Triangles
2.3.1. Weighted Means
(39)
(1)
(2)
34 2. Linear Transformations
U is regular ifand only ifIz=o IPkl = O(Pn) and P; ---+ 00. When Pn > 0, U is regular if and only if Pn ---+ 00.
2.3.2. Euler Means
Ci(A) = I~ : ; I· (2)
When P = 1, U is called the binomial method. For further properties of U, see Section 12.2.
2.3.3. HausdorffTransformations
Let ¢ be of bounded variation in [0, 1], ¢(O) = 0, ¢(1) = 1, fb d¢ = 1, and define
(1)
Theorem 1. U is regular iff ¢ is continuous at O.
Proof
= Lld¢1 < 00.
Thus U is regular iff limn~ co f!nk = O. If ¢ is continuous at 0,
(2)
(3)
Illnkl ~ f:1d¢, + f (~)Xk(1 - x)n-kjd¢1
~ J:1d¢1 + (~)(1 - e)n-k L'd¢'
= J:1d¢1 + 0(1), n --+ 00, (4)
and soit follows that limn _ oo Ilnk = O,k ~ O.Conversely,foreverye,O < I: < 1,
Illnd ~ IJ:d¢ I-I f (~)Xk(1 - x)n-k d¢I ~ IJ:d¢ 1- (~)(1 - I:)n-k L'd¢'
= ILd¢ I+ 0(1), n --+ 00. (5)
If limn _ 00 Ilnk = 0 and ¢(O) = 0, it follows that ¢ must be continuous at O. • Hausdorff weights yield interesting quadrature formulas for Stieltjes
integrals.
Theorem 2. Let f E CEO, 1]. Then
lim ±f(~)llnk = II f d¢. (6) n-e co k=O n 0
Proof The proofiselementary. By the uniform approximating properties of the Bernstein polynomials (Davis, 1963),
Bn(j; x) = kt(~)f(~)Xk(1 - x)n-k. (7)
Note that ¢ == x yields the trapezoidal formula. • For additional properties of Hausdorff transformations, see Petersen
(1966) and Peyerimhoff (1969).
= (_l)n+k (y + k)n (n)Ilnk n! k ' y>O (1)
36 2. Linear Transformations
(Salzer, 1955, 1956; Wynn, 1956a; Salzer and Kimbro, 1961; Wimp, 1972, 1975). U is not regular since
n
In fact the following result holds.
(2)
Theorem. Let A t= °be arbitrary complex. Then for U defined by (1),
a(A) = W -1, (3)
where W(A) is the modulus of the smallest (in magnitude) root(s) of
e- Z + Aez = 0. In particular,
K(U) = o( -1) = 3.5911.
i oo ~
I - 2 . n+ 1 dp,n. tu c r- i a: P
we have
t = Y + k, c> 0, (6)
(-1)" fC+;oo ePY (1 - Aep)nPn(A) = --. - dp, 2m c r- La» p P
(7)
and the theorem follows by a straightforward application of the method of steepest descents.
Rouche's theorem shows that Eq. (4) for A. = -I has exactly one root in Re z > - 1. This root is real,
zo = - 0.278464543, (8)
and since the Ilnk alternate, Eq. (5) follows immediately. •
Equation (4) is rather interesting, and has received much attention. If°< A < 1, it has no real roots. For all A. t= 0, it has a string of roots lying asymptotically within an arbitrarily narrow sector enclosing the imaginary axis. An asymptotic formula for these roots is known; see Bellman and Cooke (1963) and Wright (1955).
Equation (5) indicates the method obtained from the Salzer weights is numerically very unstable. What happens, of course, is that the weights grow large and alternate in sign.
These considerations would seem reason enough to dismiss U as a summation method suitable for any practical applications. The reader will therefore be surprised to learn that U is one of the most important summation
2.3. Important Triangles 37
methods. It is regular for a large and important class of sequences, and per- forms better in summing these sequences than even the most powerful nonlinear methods. Furthermore, those sequences, which have the property that they approach their limits algebraically (and are thus logarithmically convergent), are the sequences which pose the greatest challenge to any summation method.
To explore this idea, observe that
n 1I Ilnb + k)-r = ,{\nyn-r = boT' 0 s r S n. (9) k=O n.
This means that, ultimately, U is exact when applied to sequences that are in Lin(1,(y + n)-l,(y + n)-2, ... ,(y + n)-m); i.e., sn =.8. [In fact, this is a consequence of the manner of derivation ofthe method; see Wimp (1975) for details.]
Actually U is effective-i-but not necessarily exact-i-on a much larger class of sequences.
Let d be the class of sequences s with 00 C
s, = 8 + I ( r !3)" r=1 n + n 2 0, (10)
where C E C(}s and the series converges absolutely for n = O. Assume 0 < !3 < y. Rearranging (10) gives (the also absolutely convergent series)
00 c* s; = 8 + I ( r )" n 2 O.
r=1 n + y With the representation
1 foo(k + y)-r = -- e-(k+ Y)ltr- 1 dt, T(r) 0
we have
( l)n 00 c* foo- I ~ e- Yt( 1 - e- t)"tr - 1 dt, n! r=1 r(r) 0
so
(II)
(12)
(13)
1 - 1 < ~ ~ Ic:+nl foo -Yt('-1 dt = yn ~ Ic:+nl (14) r« - ,L. F(r) e ,L. r+nn' r=1 r 0 n· r=! Y
or
(15)
Table I
n s,
0 1 I 1.5 2 1.625 3 1.643518 4 1.644965 5 1.644951 6 1.644935185 7 1.644933943 8 1.644934041
Thus U is regular for d. In fact, we can find a constant C and an integer m such that
I(sn - s)/(sn - s) I ~ Cy"nm/n!, (16) m being the index of the first c~ i= O. Thus U is dramatically accelerative for d.
U also seems to do well on sequences which have Poincare asymptotic expansions given by the right-hand side of (10). However, without assuming something more about the character of s, there seems no way to establish regularity. Nevertheless, as an example, take
2 n 1 n Z Cl Cz
S = (PI) = L '" - + - + - + ... (17) n n k=O (k + l)z 6 n nZ
as the work of Section 1.6 shows. Taking}' = 1 gives the values shown in Table I. The error in the last entry is 2.6 x 10- 8 .
Can U sum divergent sequences? If IIlI > 1, (4) has one zero in N, so for these values of Il,a(A) > 1. Thus U sums no divergent exponential sequences. If s; is a real convergent alternating sequence, s, = (- A)", 0 < A < 1, a direct argument shows Salzer's method is regular when 0 < A < l/e and produces a divergent s; when l/e < A < 1.
The Salzer weights are best applied using a lozenge procedure; see Section 3.3, Example 3.
2.3.5. Other Nonreqular Methods
There is a class of nonregular methods that work on the same kind of sequences as the Salzer methods but that are easier to analyze theoretically. These are triangles given by
so _ (T + k>n( _l)n+k (n)
flnk - , k .n.
T> 0, (1)
2.3. Important Triangles 39
All the zeros ofPiA) lie in (0, 1) and, in fact, are equidistributed there. Thus U is not regular [Theorem 2.2(6)]. However, let .s;1 be the class of all sequences oftheform
00 C s; = S + I r,
r=l(n+T)r
Theorem 1. Let
Proof
r>n n . n21. (5)
_ (-ltr(n + T) 00 crr(r) r,> n! r~lr(r+T+n)r(r-n)
results by using the known formula for 2F1(I). Thus
If I < ~ I Icr+n+ll(n + r)! n - n! r =0 (n + r)n+r + 1r!
(6)
(8)
:::; Ipln+l ~~~Ivrl r(~(~ 1) <1>(n + 1, 2n + 1; Ipl), (7)
<1> being Tricomi's <1>-function, since (n + T)n+r+ 1 is increasing in T. However, each term in the Taylor series for <1> is decreasing in n. Letting n = 0 gives an upper bound, and the theorem results. •
The foregoing also shows that U is, ultimately, exact (rn =.0) for se- quences of the form
m C s, = S + I r.
r=l(n+T)r
Since the weights /lnk alternate in sign, the numerical stability of this method is
or
K(U) = 3 + J8 = 5.828,
even worse than the Salzer method. The method is regular for another important class of sequences.
(9)
(10)
40 2. Linear Transformations
Theorem 2. Let d be the class of S E ((js whose remainder sequences {(sn - s)} = r E ~TM' U is regular for d.
Proof For some t/J E '1',
and thus
= (_I)n+l f/~r-I'O)(t)d¢, ( 1 - t)¢(t) = t/J -2- .
(11)
(12)
tE[-I,l]; (13)
o= arccos t. (15)
p~r-I,O)(COS 8) ~ K(8)n- 1/ 2 cos[(n + r/2)8 + C], (14)
(14) holding uniformly on compact subsets of (0, n), K (>0) being integrable on such subsets. Pick b, 0 < b < 1, and write
Irnl s r.d¢ + C'n- 1/ 2 f_-I bK(8)d¢ + f-/¢,
Now pick b to make the first and last integrals < e/3; the second will be <. £/3. Thus \rnl <. £ or sn = S + 0(1). •
2.3.6. Orthoqonal Methods
Let a > 0 and {pnCt)} be a system of polynomials orthonormal with respect to some t/J E 'I' with support in [ - 1, 1]. Further, let
be bounded. If we define
JI In t/J'(t) r.- dt
-I V I - (2 (1)
O"n(a) = Pn(2/a + 1), (2)
Pn(A) has its zeros in (-a, 0), so a regular triangle is obtained, by virtue of
2.3. ImportantTriangles 41
Theorem 2.2(5). Further, by the well-known asymptotic properties of P« (Freud, 1966, p. 245ff.),
A~ [-a, OJ, P = 1 + J1+~ ~ 2,
(3)
branch cuts being taken between - a and O. The Toeplitz methods obtained by taking the polynomial P; in (2) as the
characteristic polynomial of U are perhaps the most powerful of all linear summation methods. Furthermore, they lend themselves to an elegant computational scheme that requires no explicit knowledge of the zeros of Pn
or the weights /1nk' This algorithm is derived in Chapter 3. Here, let us explore the regularity of methods based on (2). We first need
some geometric concepts. The values of A for which A(o) = IAI are given by the roots of
or by
A 1
P (4)
A (cos 2fJ - (2/p) cos fJ) - i(sin 2fJ - (2/p) sin fJ) a pZ - 4p cos fJ + 4
(5)
As f) varies between -arccos l/p and arccos LIp, the latter having its principal value, the A-values in Eq. (5) trace out the outer loop of a limacon- type figure intersecting the real axis at the points - a/pZ and 1.This figure is shown in Fig. 1 for a = 8.
Let 11(a) denote the region of the cut A-planeexterior to this curve. There (and only there) U(A) < IAI.
Let I z(a)denote the region interior to an ellipse, center ( - a/2, 0), major semiaxis on the real axis of length a/2 + 1, minor semiaxis parallel to the imaginary axis of length ja+l, and exterior to [-a, 0]. For AE Iz(a), U(A) < 1 (and only there). These observations and Theorem 2.2(2) lead to the following result.
Theorem 1. Let U be a triangle determined from Eq. (2). Then U sums rtE'n(!), I C I z, and accelerates convergence of rtErn(!) for I C r 1 n EI'
The asymptotic theory for Pn«2A/a) + 1) on the cut is rather complicated and to get an idea of what can happen, it is best to specialize, taking the case of the Jacobi polynomials, since these generate the most useful triangles. If
42 2. Linear Transformations
Fig. l.
Pn(X) = h; 1(2P~a.·P)(X), h; denoting the usual orthonormalization constant, then
v = ex + f3 + 1,(n)(n + V)k (A.)kj Jl.nk = k (ex + l ), a Tn(a),
Tn(a) = ±(n) (n + V)k (~)k. k = 0 k (ex + l)k a
From Erdelyi et al. (1953, vol. 11,10.14 (10»,
(A.) ""' C cos{[n + -!(ex + f3 + 1)]8 - (1ex + i)n} Pn ""' 2n 'p
e= arccoseaA. + 1)' A. E ( -a, 0).
(6)
(7)
cr(A) = p-2. (8)
If max ly.] in the sequence s, [see Eq. 2.2(12)] is larger than this, U will accelerate Sn' For a = 1, this value is 0.17157, and then U is accelerative if all YrE(-I, -0.17157). However, if some YrE(-0.17157, 1), then U is not accelerative and, in fact, an application of U will harm rather than help convergence, at least for such exponential sequences. This is consistent with the general philosophy that regular methods are ineffective for summing sequences that approach their limits monotonically, for instance, sequences like {1 + (!)n}. In such cases, one's recourse must be to nonregular methods.
2.3.7. The Chebyshev Weights
What is probably the best of all the positive triangles results when pix) of the previous section is chosen to be the Chebyshev polynomial y"(x) of degree n. The efficiency of U in this case is a consequence of the extraordinary interpolatory properties of y"(x) that cause that system of polynomials to play such an important role in numerical analysis and approximation theory. The definition is
T,,(x) = cos nO, 0 = arccos x, n 2': 0,
so To = 1, T1 = x, T2 = 2x 2 - 1,... . Obviously,
y,,(x) = 1[(x + i~)n + (x - ij1:::':;;2)n].
Another useful representation is
Letting x = 2A/a + 1 gives a positive triangle with entries
(1)
(2)
(3)
(4)
k, n 2': 0.
Usually one takes a = 1. As an example of the power of U take a = 1 in Eq. (4) and
n (-It Sn = (LN 2)n = L -- --+ In 2 = 0.69314718.
k=O k + I
(5)
Table II
n s,
0 1 1 0.66666667 2 0.68627451 3 0.69360269 4 0.69312536 5 0.69315096 6 0.69314685 7 0.69314723 8 0.69314717 9 0.69314718
Of course, U is a positive triangle, and thus works poorly on positive monotone sequences.
The application of U is best accomplished by a lozenge method rather than using LJlnkSk (see Chapter 3).
2.3.8. Lotockii's Method
For Lotockii's method,
p A. _ (A. + a)n _ (A. + a)(A. + a + 1)··· (A. + 1:1. + n - 1) a> O. (1) n( ) - (1 + a)n - (1 + a)(2 + 1:1.) ••• (n + a) ,
[See Lotockii (1953), Vuckovic (1958), and for applications and other refer- ences Cowling and King (1962/1963) and Agnew (1957).J Theorem 2.2(3) as it stands provides no information about the regularity of U, but, starting with Eq. 2.2(24) the proof is easily modified:
I I < Rknn (a + j - 1 + I/R) R > 1 Jlnk - ( +') ,
j~l a ]
= RkO(n(l/R)-l) = 0(1) III n. (2)
Since U is a positive triangle, this is all that is required to show that U is regular.
2.3.9. Romberg Weights
The Romberg weights are a triangle, not a positive one, that bears a close relationship to an extrapolation procedure attributed to Richardson and also
2.3. Important Triangles 45
to a method attributed to Romberg for improving the accuracy of integration by the trapezoidal rule. Both procedures are treated at length in most books on numerical analysis; see, for instance, Isaacson and Keller (1966), Bauer et al. (1963), or the articles of Bulirsch and Stoer (1966, 1967).
Take a > 1, Po = 1, and n A _ o ?
PiA) = JJ 1 _ a k' n ~ 1. (1)
/lnk then is the kth symmetric function of (17- 1,17- 2 , ••• , a- n); but it is not
necessary, nor even desirable, to compute /lnk to use U. It is much more con- venient to use a lozenge algorithm (see Chapter 3).
We have shown that U is regular. Furthermore
(2)
and the product on the right is convergent as n -+ 00. It cannot converge to zero unless one of the factors is zero. Thus U accelerates an s in the space of exponential sequences ~Em(r) iff the y, of largest modulus [see 2.1(12)] is equal to a- k for some k ~ 1.
Since (an + I - l)Pn+ l(A) = (Aan+ 1 - I)Pn(A),
one gets, by equating powers of A, the recursion formula
(an + 1 - 1)/ln+l.k = an+1/ln,k_l - Iln,k' n, k ~ 0
(Ilnk = 0, k < 0, k > n).
2.3.10. Hiqqins Weights
Higgins weights are designed for sequences having the following behavior.
(1)
cf. Eq. 2.3.3(10). In contrast to the summation formulas of Section 2.3.3, the method to be derived here is regular.
Let
By the same steepest descent argument used in Section 2.3.4,
vv,,/n! ~ j2n/n(zo + l)eY(Zo+ll( - zo)- n, Zo = -0.278464543. (3)
The Toeplitz limit theorem shows U is regular.
46 2. Linear Transformations
We now demonstrate two theorems about this process. According to Theorem 1.6(6), (1) may be rewritten
5" ~ 5 + (- 1)" f ( c~ r: (4) r= 1 n + y
or m c' ~(m)
5" =.5 + (-1)" L ( r y + ( " r+l' m 2 1, (5) r=1 n + y n + y
~(m) a bounded sequence. [(5) holds also when (4) is convergent.] Note that
" (_I)"+k llI r"k = 0, k=O (k + y)1
and this provides the following theorem.
1 :-s; j :-s; n, (6)
Theorem 1. Let (1) hold with y > 0. Then
Is" - 51 :-s; supl~~m)l/ym+1, m < n. "
(7)
Proof Left to the reader. •
When the series on the right-hand side of (4) is absolutely convergent for n = 0, much more can be said.
Theorem 2. Let the series on the right-hand side of (4) converge ab- solutely, y > 0. Then
(8)
where M is independent of nand k is the smallest integer such that ci of. 0.
Proof The integral representation 2.3.3(12) gives
(9)
(10)
Therefore,
1- I < 1 ~ Ic~ +" I < 1 ()" c(y) = ~ lerr' I.r; - W L. -r- - W C Y Y , L. " r= 1 y" r= 1 Y
Choosing k to be the smallest integer with c~ of. 0, we can describe the error sequence asymptotically by
Ir"1 ~ 1e~I/nk
(11)
2.3. Important Triangles 47
The computational gain on using the transformation (2) on series of the kind (l) is spectacular, the convergence being accelerated more than expo- nentially, actually, by a factor Anlnn. It is rare indeed that a linear method per- forms this well.
Of course since the method is a positive triangle, it is numerically stable (K = 1). The analysis of this procedure illustrates very well the gulfthat exists between the acceleration of sequences that oscillate about their limits and those that do not (logarithmically convergent sequences being cases of the latter). For the former, there exists a profusion of highly efficient summation procedures, while for the latter, the suitable methods are much less efficient and invariably nonregular. For sequences that neither oscillate about their limits nor approach their limits monotonically, almost all known methods fail. (Recent numerical evidence, however, indicates that the implicit pro- cedures of Chapter 9 hold some promise for such sequences.)
2.3.11. Inverse Methods
Some interest attaches to the inverse of a Toeplitz method V, i.e., the triangle U* = [Il~k] defined by
n
n
(1)
for all sequences s, s. The characteristic polynomials P~(A) of U* usually cannot be found
explicitly. In one case, however, this can be accomplished, that is, for the nonregular methods discussed in Section 2.3.5.
If (2)
and application of the formula in Erdelyi et al. (1953, vol. 2, 10.20(3» gives
where
Furthermore,
(3)
(4)
48 2. Linear Transformations
For litl < 1, (5) follows from (4) by dominated convergence and taking a termwise limit. For Iitl > 1, consider the integral
u« /3, it) = f(1 - t)n+a(1 + itt)n+P dt
= (1 + it)n+ Pfzn+>(1 - yz)n+ Pdz, y = it/(I + it)
= (1 + it)"+p[f IY
+ {~J = (1 + it)2n+a+ P+lit- n->-IB(n + a + 1, n + /3 + 1)
- (1/it)I(/3, a; Ilit) The use of Stirling's formula and the relation
(6)
(7)( - n - /3 1\ )(n + a + I)I(a, /3; it) = F n + a -l- 2 -it
shows (5) for Iitl > 1. A quick computation shows U* is regular, but it is a poor method to use
on exponential sequences since S will converge more slowly than itn for all litl < 1.
A considerable amount of research has been done on inverse methods. The paper by Wilansky and Zeller (1957) contains some important results and a number of useful references.
2.4. Toeplitz Methods Applied to Series of Variable Terms; Fourier Series and Lebesgue Constants
Often it is important to discern the effect of U on a series of variable terms:
co
n
n n
sn(z) = L J.1nk Sk(Z) = L Vndk(Z), k=O k=O
(1)
2.4. Toeplitz Methods Applied to Series of Variahle Terms 49
A straightforward application of Cauchy's integral formula shows that for U to sum a Taylor series about the origin anywhere within its circle of con- vergence, it is sufficient that Pn(A) -> 0 uniformly for all IAI ~ 1 - 15, for every o< 15 < 1. Obviously this is a weaker condition than regularity. Necessary and sufficient conditions are presented later [Theorem 4.3.1(1)].
Applications to Fourier series present somewhat different problems. Let f E L( - n, n) and let ak' bk be the Fourier coefficients generated by f,
Let
(2)
n
sn(x) = !ao + L (ak cos kx + b, sin kx). k=l
(3)
Assume that U is a real triangle and that six) is the result of applying the summability method U to sn(x). The convergence of s; can be related to the constants
(4)
called the Lebesgue constants for U. The standard theorem establishing the connection is due to Hardy and Rogosinki (1956).
Theorem 1. Let U be regular with Ln(U) bounded. Then sn(x) converges to
(5)
wherever this exists. If f is continuous on a compact set K c [ - n, n], then sn(x) converges uniformly to f(x) on K.
Conversely, if Ln(U) is unbounded there is an f E C[ - n, n] for which six) -f> f(x) at some point x E [ - z, n].
Proof See Hardy and Rogosinski (1956, pp. 58ff.). •
An important related result is due to Nikolskii (1948).
Theorem 2. limn~oo sn(x) = f(x) at every Lebesgue point of f iff
(i) limn~ 00 /lnk = 0 and (ii) Ln(U) is bounded.
Proof The proof is established by an appeal to results of Banach on weak convergence in Banach spaces [see Nikolskii (1948)]. •
(6)
50 2. Linear Transformations
As a philosophical consequence of such theorems, much research has centered on describing the asymptotic properties of L n( U) for various sum- mation methods.
Concerning the Hausdorff transformation
Ilnk = (~) f x\1 - x)n-k d</>
(see Section 2.3.3), Lorch and Newman (1961), improving the earlier work of Livingston (1954), have found the following result.
Theorem 3. Let U for (6) be regular. Then
Ln(U) = CcP In n + o(ln n),
where
(7)
(8)
the sum extending over all the discontinuities ~k (at most countable) of </>, and ~(f(x)) represents the mean value of the almost periodic function f(x).
Furthermore,
and CcP = 0 iff </> is continuous.
(9)
Theorem 4. Let E E e; be monotone. Then there exists an increasing absolutely continuous </> for which
LiU) i= O(En In n). (10)
This result establishes that the error term o(ln n) in (7) is the best possible and cannot be improved even for an increasing absolutely continuous </>.
For the Cesaro method
n tdn + 1) 0 sin? ()
~ f"/2 sin2(n + 1)0 _ f<n+ 1),,/2 sin? u < 1 02 dO - M --2- du
n + 0 0 u (11)
and the latter is a convergent integral. This yields the well-known fact that the Fourier series of a continuous function is Cesaro summable. On the other hand, the constants Ln(U) for the binomial method are not bounded.
(12)
2.4. Toeplitz Methods Applied to Series of Variable Terms 51
The Lebesgue constants for the methods displayed in Theorem 2.2(3),
n ,1,- Ak PiA) = }]1 1 - A
k '
have been discussed by Lorch and Newman (1962). (Note this includes Lotockif's method of Section 2.3.8.)
In what follows it is assumed that Ak < 0. Let
n 1 Un = 1 + 2 L --,.
k=1 1 - Ak (13)
Then, if Un is bounded,
while if Un is unbounded,
2 2 Ii sin t 2 leo 1 (2 )IX = - 2 Y + ~ - dt - - - - - Isin r] dt, TC TCo t TCltTC
(14)
(15)
(16)
Statement (14),coupled with Theorem 2.2(3ii),shows that for regular (f, - Yk) methods with u; bounded and Yk < 0, the Lebesgue constants are unbounded.
For generalizations to methods (not triangles) defined by
f f(A) - Ak = f Ak
k= d(1) - Ak k=/,nk'
f analytic at 0, see Shoop (1979). Concerning the behavior of the Lebesgue constants for the powerful
methods based on orthogonal polynomials (Section 2.3.6) nothing at all is known. Undoubtedly, they will prove to be unbounded, another consequence of the rule of thumb that only weak methods (Cesaro summability, for in- stance) are regular for large classes of sequences, in this case, the partial sums of the Fourier series for continuous functions.
How much improvement can be expected on applying a Toeplitz triangle to a Fourier series involves the concept of saturation. Let X denote either C[ - TC, TC] or L p [ - TC, TC], 1 ::::;; p < 00 with the norm defined in the usual man- ner. Let U be a triangle and
(17)
52 2. Linear Transformations
Theorem 5. Let f E X. If there is an a > 0 such that for each k > 0
lim nal Vnk - II > 0, (18)
then
(19)
Proof Let
Then
(20)
(21)
There exists a subsequence {nj}, nj -+ 00, such that both lim j_ oo njjvnj,k - 11 = Ck > 0 and also, by Holder's inequality, limj_oo njllsn/x) - f(x) II 1 = O. But this implies Ck Ifk I = 0 for each k > O. Since a function in X is uniquely characterized by its Fourier coefficients, f must be a constant. •
This shows that the approximation in norm of sn(x) to f(x) by Toeplitz methods satisfying (18) cannot be improved beyond the critical order n -r a:no matter how smooth f is. Saturation theory deals with the optimal order of approximation to functions E E c X by a triangle U. For instance, consider the Cesaro means, Vnk = (n + 1 - k)/(n + 1). One cannot have IISix)- f(x)11 = 0(n- 1 ) for f E C[ -n, nJ no matter how smooth f is, since IVnk - 11 = k/(n + 1) and a = 1 in the previous theorem. For all nonconstant functions in C[ - n, nJ, sn(x) approximates f with an order at most O(n- 1). In fact, this order is actually attained since for f = eix , Ilsn(x) - f(x)11 = I/(n + 1). One says the Cesaro triangle is saturated in C[ - n, nJ with order 0(n- 1 ) .
One problem is to characterize those elements in X for which the optimal order is attained. In some cases this can be done. Define
~ 1 f" tf(x) = 2n _/(x - t) cot "2 dt,
the integral being a principal value integral.
(22)
(23)
2.5 Toeplitz Methods and Rational Approximations; The Pade Table 53
Theorem 6. Let six) be the Cesaro means of the Fourier series for f(x), X = C[ -n, n]. Then
Ilsn(x) - f(x)11 = O(n- 1)
Proof See Butzer and Nessel (1971, Chap. 12). •
It can be shown that the typical means defined by Vnk = 1 - [k/(n + 1)]\ K > 0, are saturated in Cor LP with order n-".
Zemansky (1949) has studied the case
Vnk = g(k/n), (24)
where g is a polynomial with g(O) = 1, g(l) = 0, gUl(O) = 0, 1 ~ j ~ p - 1, g(Pl(O) #- 0. U is saturated in C[ - n, n] with order n- ". The saturation class of U is the subclass offunctions such that] (resp. f) is p - 1 times differentiable and satisfies a Lipschitz condition of order 1 for p even (resp. odd).
Many of the previous ideas can be generalized to abstract spaces. See Butzer and Nessel (1971).
Much work has been done on the summability of expansions in general orthogonal functions, for instance, by Olevskii (1975 and the references given there). A discussion ofthese results is outside the scope of this book. However, one result is particularly intriguing: if a E 12 and$(t) is an orthonormal system on [0, 1] and sit) = Lk= 1 ak¢k(t) is summable a.e. by a real regular triangle U, then some subsequence ofs(t) converges a.e. [see Cooke (1955, p. 90)].
2.5. Toeplitz Methods and Rational Approximations; The Pade Table
Let sn(z) denote the partial sums of the power series of a function analytic at 0,
00
n
(1)
Let y E '(j and (Jnk be an infinite lower triangular array of numbers. Define n
Aiz, y) = L y-k(JnkSk(Z), k=O
so that
n
(2)
s(z) = snCz, y) - fn(z, y), sn(z, y) = Aiz, y)/Biy), fiz, y) = Rn(z, y)/Bb)·
(3)
Explicitly,
n ()k n n n-r k (_)r A.(z, y) = L ak ~ L (In,r+kr: = L y-k L ar(Jn,r+k ~ .
k=O i r=O k=O r=O } (4)
For y fixed, snis, of course, just the Toeplitz means of SO, ••• , Sn with weights
I n
-k -kIlnk = Y (Jnk L y (Jnk . k=O
However, if we put y = z then, defining sn(z, z) = sn(z), we have
(5)
(6)
and the latter is the ratio of two polynomials in z and thus a rational ap- proximation. Clearly
[znB.(z)]s(z) - [z"An(z)] = Oiz"" 1), z~o; (7)
i.e., the rational approximation agrees with the power series through n + 1 terms. As will be shown, for certain functions s and certain choices of weights, considerably greater agreement is possible.
What is a "reasonable" choice for (Jnk? Certainly, some ofthe most power- ful Toeplitz methods are those based on orthogonal polynomials (Section 2.3.6). Thus one could take
_(n) (n + vM- 1)k (Jnk - k (f3 + l ), ' v = o: + f3 + 1, r:x, f3 > - 1. (8)
The characteristic polynomial for the method defined by (2) and (3) is then
(9)
so P; has its zeros on the ray connecting 0 and y. An argument based on Eq. 2.3.6(3) and Theorem 2.2(5) shows that U is regular iff y is real and y < O. In this case sn(z, y) ~ s(z) for all z interior to the circle of convergence of (1). Also, the rational approximation sn(z) will converge for all z real, negative, and interior to the circle of convergence of (1).
For many important functions, however, this appraisal of convergence is far too weak. These are the functions that have a representation as Stieltjes integrals
JOO dljt s(z) = --,
o 1 - zt Ijt E tJl*, z rt= Supp Ijt. (10)
2.5 Toeplitz Methods and Rational Approximations; The Pade Table 55
Theorem 1. Let the representation (5) hold and t/J have compact support. Define
a=sup{tltESUppt/J}. (11)
Let YE~, l1Y¢ [0, IJ, zalyER, °s zal» s 1, a> - t, /3 > -1. Then
1FnCz, y)1 s KnCz), (12)
I zt I fiall/y - 1 la/2+1/4Iyl-/l/2-1/4nQ+l/2
Kn(z) :'=:: 2 sup -- , o sr s« 1 - zt q!ly-1/2 + J 1/Y _ 11 2n+ v
q = max(a, /3, -1), n ---+ 00. (13)
Proof
Fn(z, y) = _R~a,/l)(I/y)-l LX) [ztl(1 - zt)JR~a,/l)(zt/y) dt/J. (14)
The proofwill require the following well-known estimate (Szego, 1959, p. 194). For wi (0, 1),
R~a,/l)(w) :'=:: _1_ (w _ 1)-a/2-1/4w-/lI2-1/4(w l /2 + ~-=-i)2n+v, (15) 2JM
branch cuts for (w - l )" and w" being taken along ( - 00, IJ and ( - 00, OJ, respectively, This result holds uniformly on compact subsets of ~ - [0, 1]. Using the fact that R~a,/l)(x)can be bounded algebraically (Erdelyi et aI., 1953, vol. II, 10.18(12)) completes the proof. •
Corollary. Under the conditions stated above, sn(z) converges ex- ponentially to s(z). Further, the rational approximations sn(z, az) converge uniformlytos(z)oneverycompactsubsetSof~- Dla, w),alsoexponential- ly;i.e.,
ZES,
(16)
for some M and e. Note 1" < 1.
Example. Let
s(z) = F(1, /3; v; z), v=a+/3+1. (17)
56 2. Linear Transformations
(18)
and
(19)
On expanding (1 - zt)-l in powers of t one finds the first n terms, that is, the coefficients of 1, z, ... , zn- 1 vanish by virtue of the orthogonality properties of R~IZ·/l)(t). Thus
[znBiz)]s(z) - [znAn(z)] = O(z2n+l); (20)
i.e., in this case the rational approximation yields the [n/n] entry in the Pade table for F(1, fJ; (f. + fJ + 1; z), These rational approximations, by virtue of the theorem, converge uniformly On compact subsets of C& - [1, 00). For an extensive discussion of the construction and properties of Pade approximants, see Chapter 6.
Using
( l)n d n
R~IZ·/l)(t) = ~ ~ (1 - t)-lZt-/l dt" [(1 - t)lZ+nt/l+n], (21)
a useful formula for rn can be derived by integration by parts:
rn(z) = .~n<z) - s(z) = Cnz"+ 1F(n + 1, n + fJ + 1; 2n + v + 1; z)/R~"·/l)(l/z),
c = -r(v)r(n + (f. + 1)r(n + P+ 1) (22) n r(P)r«(f. + 1)r(2n + v + 1) .
The F in (22) is easily estimated by Watson's formula (Luke, 1969, vol. I, p.237):
(23)r z = -2nr(v)(1 - z)"z2n+l [1 (I)J n() r(fJ)r«(f. + 1)[1 + ft-="Z]4n+2v + 0 ~ ,
the branch cut of J1=Z being taken along [1, (0).
A similar error formula holds when the Unk are the coefficients of any system of polynomials orthogonal with respect to a general distribution function t/J E '11* with bounded support.
(24)a ~ r ~ b, 1
z #- -, t
Let t/J E '11* with support in [a, b], - 00 ~ a < b ~ 00,
i b dt/J
2.5. Toeplitz Methods and Rational Approximations; The Pade Table 57
and let 0'nk be defined by n
Pn(t) = L O'nk tk k=O
(25)
where {Pn} is a system of polynomials