P1 2018 1 / 50

P1 Calculus of a single variable

Prof David Murray

david.murray@eng.ox.ac.ukwww.robots.ox.ac.uk/∼dwm/Courses/1CA1

4 lectures, MT 2018

P1 2018 2 / 50

Topic 3: Series, Convergence, andPolynomial Expansions

P1 2018 3 / 50

Introduction

The main part of this lecture is concerned with series expansions —Taylor’s and MacLaurin’s expansions or series — which tell us how toestimate functions in terms of polynomials.

Taylor and MacLaurin are really useful.

However, first we introduce the broader notions of series and convergence.

P1 2018 4 / 50

Lecture contents

3.1 Elementary Series3.2 Convergence and Divergence – in brief3.3 Taylor’s series expansion3.4 MacLaurin’s series expansion3.5 De L’Hôpital’s Rule3.6 Using Taylor’s expansion to estimate derivatives from samples

P1 2018 5 / 50

3.1 Series

P1 2018 6 / 50

Series: definition

Consider an ordered sequence of items a1, a2, etc, each of which isrelated to its neighbours in some defined way.

Adding the terms together forms a series.

If there are an infinite number of items the sum is an infinite series andthe sum is written

S∞ = a1 + a2 + a3 + ... =

∞∑i=1

ai .

If one adds up the sum to n terms

Sn = a1 + a2 + . . .+ an

the result is called a partial sum.

P1 2018 7 / 50

Convergence

An infinite series will either converge, which means

limn→∞ Sn → Finite value ,

or diverge.

A necessary condition for convergence is (obviously!) that

limn→∞ an → 0

but this is not a sufficient condition.

A divergent series might have partials sums that tend monotonically toeither +∞ or −∞, or its partial sums might oscillate between ±∞.

P1 2018 8 / 50

Types of convergence

A convergent series with positive and negative terms in it might beconditionally convergent or absolutely convergent.

If a convergent series

S∞ = a1 + a2 + a3 + . . .

contains positive and negative terms they must cancel each othersomewhat.

We could therefore ask a stronger question: does the following converge?

S ′∞ = |a1|+ |a2|+ |a3|+ . . . ?

If it does converge, the original series is absolutely convergent.

P1 2018 9 / 50

Eg#1 Arithmetic SeriesAn arithmetic series ois a sequence of n numbers successive members ofwhich differ by the same amount d .

{a1, (a1 + d), (a1 + 2d), . . . , (a1 + (n − 1)d)}

The i-th term: is ai = a1 + (i − 1)d .

Sum: Suppose we wanted the sum to the n-th term, Sn =∑n

i=1 ai .

Sn = a1 + (a1 + d) + (a1 + 2d) + . . .+ (a1 + (n − 1)d)= (a1 + (n − 1)d) + (a1 + (n − 2)d) + (a1 + (n − 3)d) + . . .+ a1

Add corresponding terms: n terms each of 2a1 + (n − 1)d ,so

Sn =12n(2a1 + (n − 1)d) =

12n(a1 + an) .

Convergence: The sum of this series always diverges as more terms areadded. The only series that remains finite is a string of zeros!

P1 2018 10 / 50

Eg#2 Geometric seriesA geometric series or progression is a sequence of n numbers successivemembers of which are multiplied by a common factor r .

{a1, ra1, r2a1, . . . , rn−1a1}

The i-th term: is ai = r i−1a1.Sum: The sum to n terms is

Sn = a1 + ra1 + r2a1 + . . . + rn−1a1⇒rSn = ra1 + r2a1 + . . . + rn−1a1 + rna1

SoSn(1− r) = a1(1− rn) ⇒Sn =

a1(1− rn)

1− r.

Convergence: The sum of an infinite number of terms will diverge to±∞ unless |r | < 1. If this condition is satisfied, rn → 0 and

S∞ =a1

1− r.

P1 2018 11 / 50

Eg#3 Harmonic Series

The harmonic series is11,12,13,14,15,

The i-th term is ai = 1/i .

The sum to n terms isn∑

i=1

1i=

11+

12+

13+

14+ . . .+

1n

— but that’s not much help!

Convergence: The sum to an infinite number of terms is, er ...

Yes, we have a problem. It really is not obvious.

P1 2018 12 / 50

Tests for convergence

Good news:there are a many tests you can apply to a series to test for convergence.Eg:

1. Basic last-term limit test 2. Term-by-term comparison test3. Ratio comparison test 4. D’Alembert’s ratio test5. Cauchy integral test 6. Alternating terms test

Bad news:it is never immediately obvious which will give you a helpful result.

Good news:from this year, you don’t have to learn them like a parrot.

Understand the concept of convergence, but don’t learn the tests.

P1 2018 13 / 50

Eg#4Let’s look at just one example.

Q: Does S = 1+ x +x2

2!+

x3

3!+

x4

4!+ . . . converge?

A: One might imagine that this series would converge for, say, x < 1, butdiverge for large otherwise. But let’s not guess ...

One possible test is D’Alembert’s ratio test, in which you test wherethe ratio of successive terms tends to zero.

But notice that x could be negative, and the terms oscillate in sign. Sowe might consider the modulus of all the terms in SAbs.

Apply it to SAbs.

limn→∞

∣∣∣∣ xn+1

(n + 1)!

∣∣∣∣ ∣∣∣∣ n!xn

∣∣∣∣ = |x |n + 1

.

This tends to zero as n→∞, so that the series isAbsolutely Convergent — and, surprisingly, for any finite x.

P1 2018 14 / 50

Eg#4 /ctd: Can we understand why?Suppose x = 50 in 1+ x + x2

2! +x3

3! +x4

4! + . . . The terms would increase atfirst — until

5050

50!is followed by

5051

51!So, we have a finite number of naughty terms, followed by an infinitenumber of well-behaved terms. The numbers are impressive ...

100

101

102

103

100

105

1010

1015

1020

1025

No of terms

Sum

of series for

x=

50

⇒ Believe the tests, not your intuition.

P1 2018 15 / 50

3.4 Taylor’s Series Expansion

The series requiring the special analysis justdiscussed tend to occur infrequently inengineering analysis.

The series expansions we are about todiscuss are frequently useful across theentire range of engineering specialities.

They are very important! Brook Taylor 1685-1731

Fresher at St John’s

Cambridge in 1703

P1 2018 16 / 50

Taylor’s series expansion

Suppose we know the value f (a) of a function f (x) at x=a ...

And suppose we know some of its derivatives f ′(a), f ′′(a), ..., at x=a.

Can we estimate the value of the function at x = a + h, where h is somesmall offset?

The answer is yes, using ...

P1 2018 17 / 50

Taylor’s series expansion

Taylor’s expansion to O(hn)

f (a+h) = f (a)+hf ′(a)+h2

2!f ′′(a)+

h3

3!f ′′′(a)+. . .+

hn

n!f (n)(a)+Rn+1 .

where the remainder

Rn+1 =hn+1

(n + 1)!f (n+1)(ζ), a 6 ζ 6 (a+h) .

You might ask, why not simply put a + h into the function and computethe answer? Two reasons:

1 You might not actually know what f (x) is — you may just havenumerical values at x=a.

2 The expansion provides an approximation to a function that ispolynomial in the small quantity h. We can compute polynomialsusing multiplication and addition!

P1 2018 18 / 50

[**] Explanation of the expansion

f (a + h) = f (a) + hf ′(a) + h2

2! f′′(a) + h3

3! f′′′(a) + . . .+ hn

n! f(n)(a) + Rn+1 .

* The “zeroth order” approximation would be to say that (a + h) is notfar from a, so f (a + h) ≈ f (a) .

* But if we know the first derivative at x = a, we can make a 1st ordercorrection of C1 = hf ′(a), leading to f (a + h) ≈ f (a) + hf ′(a).

x

f(a)

a a+h

x

f(a)

a a+h

correction

1st order

P1 2018 19 / 50

[**] Explanation of the expansion

x

f(a)

a a+h

pp hp+dp0

* Suppose we know f ′′(a) too. This allowsestimation of the extra correction arising fromthe change in gradient between a and a + h.

The “extra” gradient at x = a is zero.

What is the extra gradient f̂ ′ at a point p,where p lies between 0 to h?

It is f̂ ′(p) = p f ′′(a).

* Between p and p + dp this will introduce an infinitesimal extracorrection to the function

df̂ = f̂ ′(p)dp = p f ′′(a) dp .

* Integrating between 0 and h, the total extra correction is

C2 =

∫h

0df̂ =

∫h

0pf ′′(a) dp = f ′′(a)

∫h

0pdp =

h2

2f ′′(a) .

P1 2018 20 / 50

[**] Explanation of the expansionThe 3rd order correction requires us to integrate the “extra extragradient” ˆ̂f ′(p) due to f ′′′(a).

C3 =

∫h

0

ˆ̂f ′(p)dp .

To find ˆ̂f ′(p) at p we have to integrate f̂ ′′(q) from 0 to p. That is,

ˆ̂f ′(p) =∫p

0f̂ ′′(q)dq

But the extra f ′′ at q is

f̂ ′′(q) = f ′′′(a)q

Sticking this all together

C3 =

∫h

0

[∫p

0f ′′′(a)qdq

]dp = f ′′′(a)

∫h

0

[p2

2

]dp = f ′′′(a)

h3

3!

P1 2018 21 / 50

[**] Explanation of the expansion

We are on a roll now ...

C4 =

∫h

0

[∫p

0

[∫q

0f (4)(a)rdr

]dq]dp = f (4)(a)

∫h

0

[∫p

0

[q2

2

]dq]dp

= f (4)(a)∫h

0

[p3

3!

]dp = f (4)(a)

h4

4!

and so on and on!

Adding all the correction to f (a) gives us the result.

f (a + h) = f (a) + hf ′(a) +h2

2!f ′′(a) +

h3

3!f ′′′(a) + . . .

P1 2018 22 / 50

Taylor’s expansion: ♣ Example:

Reminder: f (a + h) = f (a) + hf ′(a) +h2

2!f ′′(a) +

h3

3!f ′′′(a) + . . .

Q: Find the series expansion of ex about x = 1.A: a = 1 and our small quantity is h.Build a table of derivatives and theirvalues:

f (x) ex f (1) = ef ′(x) ex f ′(1) = ef ′′(x) ex f ′′(1) = ef ′′′(x) ex f ′′′(1) = e

So the series is

e(1+h) =

[e+ he+

h2

2!e+

h3

3!e+ . . .

]= e

[1+ h +

h2

2!+

h3

3!+ . . .

]Suppose h = 0.8 and we use 5 terms The series says

e1.8 ≈ e [1+ 0.8+ 0.32+ 0.0853+ 0.0171 . . .] = 6.0411

and the exact value ise1.8 = 6.0496

P1 2018 23 / 50

♣ Example: #2Q: Find the series expansion of f (x) = 1/(1+ x) about x = a.A: The fixed point is a and the smallthing is h. The table of derivatives is ê⇒In general, we can write

f (n)(a) = (−1)nn!1

(1+ a)n+1

f (x) (1+ x)−1

f ′(x) (−1)(1+ x)−2

f ′′(x) (−1)(−2)(1+ x)−3

f ′′′(x) (−1)(−2)(−3)(1+ x)−4

So

f (a + h) = f (a) + hf ′(a) +h2

2!f ′′(a) +

h3

3!f ′′′(a) + . . .

=1

(1+ a)−

h(1+ a)2

+h2

(1+ a)3−

h3

(1+ a)4+ . . .

=1

(1+ a)

[1−

(h

1+ a

)+

(h

1+ a

)2

−

(h

1+ a

)3

+ . . .

].

P1 2018 24 / 50

#2 /ctd

Reminder ...

f (x) =1

1+ x

f (a + h) =1

(1+ a)

[1−

(h

1+ a

)+

(h

1+ a

)2

−

(h

1+ a

)3

+ . . .

].

To test with some numbers, let a = 0.5 and h = 0.2.

The series is

f (0.5+0.2) =23

[1−

(0.21.5

)+

(0.21.5

)2

−

(0.21.5

)3

+

(0.21.5

)4

− . . .

]= 0.5883 ,

while the exact value is 1/(1+ (0.5+0.2)) = 0.5882.

P1 2018 25 / 50

A source of confusion, cleared upDifferent books appear to express Taylor’s expansion differently.Sometimes f (a + h), other times f (x + h), and elsewhere f (a + x).

* This is merely a different choice of symbols for the fixed pointand small offset.

Decide what the fixed point is, decide what ze small thing is, and youcannot go wrong.

For example, for Greeks who want a fixed point of µ, and a small thing η:

φ(µ+ η) = φ(µ) + ηφ ′(µ) +η2

2!φ ′′(µ) + . . .

For Goths with fixed point A and small thing G:

F(A+ G) = F(A) + GF ′(A) +G 2

2!F ′′(A) + . . .

P1 2018 26 / 50

The remainder term explained

Rn+1(ζ) =hn+1

(n + 1)!f (n+1)(ζ)

You might care to read how the remainder term is arrived at.Here we only need to explain what it means.

* It’s NOT the “next term” (that would be “the leading error term”)If GREEN is your approximation, RED is your leading error term

f (a + h) = f (a) + hf ′(a) +h2

2!f ′′(a) +

h3

3!f ′′′(a) + . . .

* Instead, the remainder estimates ALL the neglected terms.

R3 ≈h3

3!f ′′′(a) +

h4

4!f (4)(a) +

h5

5!f (5)(a) + . . .

P1 2018 27 / 50

The remainder term explained

Rn+1(ζ) =hn+1

(n + 1)!f (n+1)(ζ)

* Notice that it is easy to remember. It looks almost exactly like “thenext term”

* But f (n+1) is not evaluated at x = a, ratherat x = ζ, where a 6 ζ 6 (a + h).

xa a+h

?

is somewhere

between

and

a

a+h

ζ

Obvious Question:What value does ζ haveexactly?

P1 2018 28 / 50

The remainder term explainedThe way to work it out is actually very straightforward.

As h is fixed, and n is known, it depends on f (n+1)(ζ) alone.

Rn+1(ζ) =hn+1

(n + 1)!f (n+1)(ζ)

So plot the value of f (n+1)(ζ) as ζ varies between a and a + h,then choose the ζ value which gives the largest magnitude.

a a+h

a a+hf(n+1)

(ζ )

ζ=(a+h)

ζ

ζSome possible plots ...

Plot

between

and

ζ =a

P1 2018 29 / 50

♣ Example: The remainder term explainedConsider the earlier example

f (a + h) =

[1

(1 + a)−

h(1 + a)2

+h2

(1 + a)3−

h3

(1 + a)4+

h4

(1 + a)5− . . .

]

We want R5, so put n = 5 into the general remainder:

Rn = (−1)nhn 1(1+ ζ)n+1 ⇒ R5 = (−1)50.25 1

(1+ ζ)6

ζ lies between a = 0.5 and (a + h) = (0.5+ 0.2). That is, 0.5 6 ζ 6 0.7

But dR5/dζ = (−1)50.25(−6)(1+ ζ)−7 is never zero: there’s no turningpoint.

⇒The largest magnitude of R5 is when ζ is smallest ⇒ζ = 0.5.

⇒ max |R5| =

∣∣∣∣(−1)5(0.2)51

(1+ 0.5)6

∣∣∣∣ ≈ 2.8× 10−5

P1 2018 30 / 50

The remainder term computedLet’s plot

the absolute value of R, Green Dashed, andthe absolute value of the True Error Red Solid

as a function of the number of terms n = 1, 2, . . . 10 used ...

0 2 4 6 8 1010

−10

10−8

10−6

10−4

10−2

100

abs(Remainder)

abs(Error)

1 1.5 2 2.5 310

−4

10−3

10−2

10−1

abs(Remainder)

abs(Error)

Importantlythe estimate of the Remainder is quite close to the actual Errorthe estimate of the Remainder is always bigger than the actualError. It is sensibly pessimistic.

P1 2018 31 / 50

3.5 MacLaurin’s Series Expansion

Colin Maclaurin 1698-1746

Wee Colin graduated from U

Glasgow aged 14

P1 2018 32 / 50

MacLaurin’s Series Expansion

GOOD NEWS! There is no further thought required to write downMacLaurin’s Series.

It is simply a Taylor Series about fixed point a = 0.

Because the fixed point is zero, this series is often written using x as thesmall quantity, rather than h.

MacLaurin’s Series to O(xn)

f (x) = f (0) + xf ′(0) +x2

2!f ′′(0) +

x3

3!f ′′′(0) + . . .+

xn

n!f (n)(0) +Rn+1 .

where the remainder

Rn+1 =xn+1

(n + 1)!f (n+1)(ζ), 0 6 ζ 6 x .

P1 2018 33 / 50

♣ Example: MacLaurin’s

Q: Determine the first four non-zero terms in the MacLaurin’s series forln(1+ x)

A: The table of derivatives is:n f (n)(x) f (n)(0)0 ln(1+ x) 01 1/(1+ x) 12 −1/(1+ x)2 -13 2/(1+ x)3 24 −(3)!/(1+ x)4 -6

Hence:

ln(1+ x) = x −12!x2 +

23!x3 −

64!x4 + . . .+ (−1)n+1 (n − 1)!

n!xn + . . .

= x −12x2 +

13x3 −

14x4 + . . .

P1 2018 34 / 50

♣ Example: MacLaurin’s

Q: Determine the MacLaurin expansion of sin(x)

A: The table of derivatives and their values isn f (n)(x) f (n)(0)0 sin(x) 01 cos(x) 12 − sin(x) 03 − cos(x) -14 sin(x) 0

f (x) = f (0) + f ′(0)x1!

+ f ′′(0)x2

2!+ f ′′′(0)

x3

3!+ f (4)(0)

x4

4!+ . . .

Hence

sin(x) = x −x3

3!+

x5

5!−

x7

7!+ . . .

P1 2018 35 / 50

3.6 de L’Hôpital’s Rule

Guillaume François AntoineMarquis de L’Hôpital, Marquisde Sainte-Mesme, Comted’Entremont and Seigneurd’Ouques-la-Chaise 1661-1704

P1 2018 36 / 50

de l’Hôpital’s Rule

You will recall that the 0/0 created by limx→0

sin(x)x

was equal to 1, but

that this was awaiting proof.

The underlying problem is that both numerator and denominator tend tozero, leaving and apparently indeterminate ratio.

Let’s consider the general problem of finding

limx→a

f (x)g(x)

where both f (a) = 0 and g(a) = 0.

P1 2018 37 / 50

de l’Hôpital’s Rule

We have exactly the tool to handle this in Taylor’s expansion

limx→a

f (x)g(x)

= limh→0

f (a) + hf ′(a) + (h2/2!)f ′′(a) + . . .

g(a) + hg ′(a) + (h2/2!)g ′′(a) + . . .

But both f (a) and g(a) are zero. So, dividing top and bottom by h,

limx→a

f (x)g(x)

= limh→0

f ′(a) + (h/2!)f ′′(a) + . . .

g ′(a) + (h/2!)g ′′(a) + . . .

But as h→ 0, all the terms multiplied by h, h2 will vanish, so ...

P1 2018 38 / 50

de l’Hôpital’s Rule

we reachde l’Hôpital’s ruleIf f (a) = g(a) = 0,

limx→a

f (x)g(x)

=f ′(a)g ′(a)

NB! that the two derivatives are taken separately — you do NOT usethe quotient rule!

P1 2018 39 / 50

♣ Example: de l’Hôpital’s Rule

Q: Find

limx→0

sin(x)x

A: Both numerator and denominator are zero,hence using de l’Hôpital’s rule

limx→0

sin(x)x

=f ′(0)g ′(0)

=cos(0)

1= 1 .

P1 2018 40 / 50

♣ Example: de l’Hôpital’s Rule

Q: Find

limx→0

sin(x2)

x2

A #1: If you are awake ... Substitute u = x2 and take the limit asu → 0. Obviously the answer is the same as before, unity.

A #2: However, suppose you hadn’t spotted that and just used del’Hôpital’s rule

limx→0

sin(x2)

x2 =f ′(0)g ′(0)

=2x cos(x2)

2x at x=0= cos(0) .

Now you are allowed to divide through by 2x and hence the answer iscos(0) = 1.

P1 2018 41 / 50

♣ Example: ctd

A #3: However however, suppose you had not spotted even that, andjust said that it is 0

0 .

Going back to the series expansion

limx→a

f (x)g(x)

= limh→0

f (a) + hf ′(a) + (h2/2!)f ′′(a) + . . .

g(a) + hg ′(a) + (h2/2!)g ′′(a) + . . .

you’ll see thatIf f (a) = g(a) = 0 AND f ′(a) = g ′(a) = 0 then

limx→a

f (x)g(x)

=f ′′(a)g ′′(a)

P1 2018 42 / 50

♣ Example: de l’Hôpital’s Rule

This allows us to write a supercharged version ...

de l’Hôpital’s rule (fuller statement)If f (a) = g(a) = 0 and all f (n) = g (n) = 0 up to some number n = j ,then

limx→a

f (x)g(x)

=f (j+1)(a)g (j+1)(a)

So in our example we can keep going!

limx→0

sin(x2)

x2 =f ′′(0)g ′′(0)

=2 cos(x2) − 4x2 sin(x2)

2 at x=0= 1 .

P1 2018 43 / 50

♣ Example:

Q: Find limx→0

ln(x + 1)x2

A1: Apply de l’Hôpital’s rule limx→0

ln(1+ x)x2 =

1/(x + 1)2x at x=0

→∞A2: Because de L’Hôpital involves series expansions, an equivalentmethod of solution is to write down the expansions.

As the limit in the last example is taken about x = 0, we can use theMacLaurin expansion for ln(1+ x) where x is small.

ln(1+ x)x2 =

x − 12x

2 + 13x

3 − . . .

x2 =1x−

12+

13x −

14x2 . . .

which obviously shoots off to infinity when x → 0.

P1 2018 44 / 50

3.7 Using Taylor’s to estimate derivatives numerically

P1 2018 45 / 50

Using Taylor’s to estimate derivatives numerically

Another very useful application of Taylor’s expansion is in the estimationof the derivatives of a unknown function for which one only hasinformation at discrete values of the independent variable

This is very useful computationally, as measurements are often obtainedat discrete points or at discrete times.

a

f(x)

x

f(a)

a−h a+h a+2h

We know f (a − h), f (a), f (a + h), and so on, but we want to estimatethe derivatives.

P1 2018 46 / 50

Using Taylor’s to estimate derivatives numericallyUsing Taylor’s expansion

f (a + h) = f (a) + hf ′(a) +h2

2!f ′′(a) +

h3

3!f ′′′(a) + . . .

So we could estimate

f (a + h) − f (a) ≈ hf ′(a) +h2

2!f ′′(a) + . . .

or

f ′(a) ≈[f (a + h) − f (a)

h

]+

h2!f ′′(a) + . . .

The leading error term h2! f

′′(a) is O(h) (“order h”, or proportional to h).

This means that if you make h smaller by a factor of 10, the error in theestimate is expected to reduce by a factor of 10.

P1 2018 47 / 50

But we can do better! Again using Taylor’s expansion

f (a + h) = f (a) + hf ′(a) +h2

2!f ′′(a) +

h3

3!f ′′′(a) + . . .

f (a − h) = f (a) − hf ′(a) +h2

2!f ′′(a) −

h3

3!f ′′′(a) + . . .

Now subtract ...f (a + h) − f (a − h) = 2hf ′(a) + 2

h3

3!f ′′′(a) + . . .

⇒[f (a + h) − f (a − h)

2h

]≈ f ′(a) +

h2

3!f ′′′(a) .

So the estimate is

f ′(a) ≈[f (a + h) − f (a − h)

2h

]+h2

3!f ′′′(a) + . . .

The leading error term is now O(h2)Sampling 10 times more closely makes the error decrease by 102.

P1 2018 48 / 50

Summary

We reviewed the concepts of convergence, divergence, absolute and andconditional convergence.

We introduced an unhealthy number of tests for convergence.

We then looked saw Taylor’s expansion: estimates function by a powerseries in a small deviation about a fixed point. Found that MacLaurin’sseries is just Taylor’s expansion applied at x = 0.

Both these series are very valuable, as they allow functions to beapproximated as polynomials.

We saw how Taylor’s expansion can explain de l’Hôpital’s Rule.

Finally we saw how to use samples from a function to approximatederivatives.

P1 2018 49 / 50

If you have been ...

... thanks for listening

P1 2018 50 / 50