module 9: proof techniques (part 2) mathematical induction

CPSC 121 – 2014W T2 1

CPSC 121: Models of Computation

Module 9: Proof Techniques (part 2)Mathematical Induction

CPSC 121 – 2014W T2 2

Module 9: Mathematical Induction

Announcements:

In-class test #2:

Wednesday March 18th.

Format similar to that of in-class test #1.

You can bring one 8.5 x 11in (21.59 x 27.94cm) two sided sheet of paper.

No textbook, calculator or other electronic equipment is allowed

It covers modules 7 and 8.

CPSC 121 – 2014W T2 3


By the start of class, you should be able to

Convert sequences to and from explicit formulas that describe the sequence.

Convert sums to and from summation/Σ notation.

Convert products to and from product/π notation.

Manipulate formulas in summation/product notation by adjusting their bounds, merging or splitting summations/products, and factoring out values.

CPSC 121 – 2014W T2 4


Pre-class quiz #9

Very well done.

Being comfortable with summations is important not only for CPSC 121, but for following courses too!

As usual, we will revisit the open-ended question shortly.

CPSC 121 – 2014W T2


???

?

?

? ??

?

???

?

?

?

?

?

CPSC 121: the BIG questions:

1. How can we convince ourselves that an algorithm does what it's supposed to do?

2. How do we determine whether or not one algorithm is better than another one?

Mathematical induction is a veryvery useful tool when proving the correctness or efficiency of an algorithm.

We will see several examples of this.

CPSC 121 – 2014W T2 6


Module Summary

Example: single-elimination tournaments.

Defining and validating mathematical induction.

More examples where we can use induction.

A slightly different type of induction.

CPSC 121 – 2014W T2 7


Problem: single-elimination tournament

Teams play one another in pairs

The winner of each pair advances to the next round

Montréal

Montréal

Los Angeles

Montréal

Toronto

New York Isl.

Vancouver

St. Louis

Pittsburgh

Buffalo

Los Angeles

Los Angeles

Toronto

New York Isl.Montréal

Round 2 Round 3 Round 4

CPSC 121 – 2014W T2 8


Question: if we have n rounds of playoffs, how many teams can participate?

a) n

b) 2n

c) n2

d) 2n

e) None of the above.

CPSC 121 – 2014W T2 9


How can we prove this result formally?

We will use a technique called mathematical induction.

We will convince ourselves that it is a valid proof technique.

Many theorems involve summations, so it is important to be comfortable manipulating them.

Hence the reading for pre-class quiz #9.

CPSC 121 – 2014W T2 10


First we look at the following question instead:

If t teams can participate in a playoff with n rounds, how many teams can participate in a playoff with n+1 rounds?

a) t+1

b) 2t

c) t2

d) 2t


CPSC 121 – 2014W T2 11


Proof (with holes):

Consider an unspecified playoff with n rounds. Assume that

We can think of a playoff with n+1 rounds as follows:

Two playoffs with n rounds proceed in parallel.

The two winners then

Since each playoff with n rounds has

CPSC 121 – 2014W T2 12


How do we use this to prove what we want?

Recall Universal Modus Ponens:P(a)∀x D, P(x) → Q(x)∈Q(a)

If we define

P(t, n): Up to t teams can participate in a playoff with n rounds

then we have proved

∀t Z∈ + n Z∀ ∈ + , P(t, n)→P(2t, n+1)

CPSC 121 – 2014W T2 13


So

We know that only 2 teams can play with 1 round.

This is P(2, 1).

Given

P(2, 1)

∀t Z∈ + n Z∀ ∈ +, P(t, n) → P(2t, n+1)

We can deduce:

P(4, 2)

CPSC 121 – 2014W T2 14


Given

P(4, 2)∀t Z∈ + n Z∀ ∈ +, P(t, n) → P(2t, n+1)

We can deduce:

P(8, 3)

Given

P(8, 3)∀t Z∈ + n Z∀ ∈ +, P(t, n) → P(2t, n+1)

We can deduce:

P(16, 4)

CPSC 121 – 2014W T2 15


Given

P(16, 4)∀t Z∈ + n Z∀ ∈ +, P(t, n) → P(2t, n+1)

We can deduce:

P(32, 5)

Etc...

Therefore we can show that

∀n Z∈ +, P(2n, n)

CPSC 121 – 2014W T2 16


Another way to this about this:

Given an unspecified positive integer n.

To prove P(2n+1, n+1):

First we prove P(2n, n).

Then we use the fact that

∀t Z∈ + n Z∀ ∈ +, P(t, n) → P(2t, n+1)

Hence induction is more or less the same as recursion!

A fun example involving induction: http://www.youtube.com/watch?v=daRS98L9rHs&feature=related

CPSC 121 – 2014W T2 17


Further readings: the rest of the chapters on mathematical induction (Epp4, 5.2 to 5.4, etc).

After this reading, you will be able to:

Given a theorem to prove stated in terms of its induction variable (i.e., usually, in terms of n), write out the skeleton of an inductive proof including:

the base case(s) that need to be proven,

the induction hypothesis, and

the inductive step that needs to be proven.

CPSC 121 – 2014W T2 18


By the end of this module, you should be able to:

Establish properties of self-referential structures using inductive proofs that naturally build on those self-references.

Critique formal inductive proofs to determine whether they are valid and where the error(s) lie if they are invalid.

Prove properties of the non-negative integers (or a subset) with appropriate self-referential structure using weak or strong induction as needed.

CPSC 121 – 2014W T2 19


Module Summary





CPSC 121 – 2014W T2 20


Define Q(n) by:

Q(n): P(2n,n)

In the previous example, we proved

Q(1)

∀n Z∈ +, Q(n) → Q(n+1)

and deduced that

∀n Z∈ +, Q(n)

Why is this a valid proof technique?

CPSC 121 – 2014W T2 21


Theorem:

Q(1) ^ ( n Z∀ ∈ +, Q(n) → Q(n+1)) → n Z∀ ∈ +, Q(n)

Proof:

We use a proof by contradiction.

Suppose that the premises hold, but that n Z∀ ∈ +, Q(n) is false. There is at least one value of n for which Q(n) does not hold.

Let y be the smallest such value. Clearly y > 1, so y – 1 Z∈ +.

CPSC 121 – 2014W T2 22


Proof (continued):

Because y is the smallest value for which Q(n) is false, and y – 1 Z∈ +, Q(y - 1) is true.

But then the 2nd premise implies that Q(y) is true, a contradiction (since we assumed Q(y) was false).

Therefore the conclusion must be true, that is n ∀ ∈Z+, Q(n) holds.

CPSC 121 – 2014W T2 23


So a proof by mathematical induction has

One (or more) base case(s): Q(1)

Usually very straightforward to prove.

An induction step: n Z∀ ∈ +, Q(n) → Q(n+1)

We can use any proof technique we know for it.

Usually a direct proof will work well.

CPSC 121 – 2014W T2 24


Module Summary





CPSC 121 – 2014W T2 25


Example 1: recall the exercises from our first class:

Make group of 5 students.

Then order each group from in order of day of birth by swapping pairs of adjacent students.

We claimed the maximum number of swaps for n students is n(n-1)/2.

CPSC 121 – 2014W T2 26


How many swaps do we need?

Suppose we place students from left to right.

The students already placed are ordered by day of birth.

We swap each new student with his/her neighbour until he/she is at the right place.

The ith student may be swapped with all previous i-1 students.

So the total number of swaps is

Hence we need to prove that

∑i=0

n−1

i

∑i=0

n−1

i=n(n−1)

2

CPSC 121 – 2014W T2 27


Which fact(s) do we need to prove?

a)

b)

c)

d) Both (a) and (b)

e) Both (a) and (c)

f) None of the above.

∑i=0

0

i=0

∀n∈ℤ+ (∑

i=0

n−1

i=n(n−1)

2 )→(∑i=0

n

i=(n+ 1)n

2 )(∀n∈ℤ

+ ,∑i=0

n−1

i=n(n−1)

2 )→(∀ n∈ℤ+ ,∑

i=0

n

i=(n+1)n

2 )

CPSC 121 – 2014W T2 28


Proof:

Base case: n = 1

Clearly, with only 1 student, no swap is needed: which is equal to 1 (1-1)/2

Induction step:

Pick an unspecified n ≥ 1. Assume that when we have n students, we need at most n(n-1)/2 swaps. Equivalently we assume that

This is called the Induction Hypothesis.

∑i=0

0

i=0

∑i=0

n−1

i=n(n−1)

2

CPSC 121 – 2014W T2 29


Proof (continued)

Induction step (continued)

For n+1 students, we need at most swaps.

By the induction hypothesis, this is equal to

swaps, as required for the induction step.

Hence by the principle of M.I., the theorem holds. QED

∑i=0

n

i=(∑i=0

n−1

i)+ n

n(n−1)

2+ n=

(n+ 1)n2

CPSC 121 – 2014W T2 30


Example 2: geometric series

We will prove that for every value of a ≠0, 1:

These summations occur frequently when we need to determine the running time of divide-and-conquer algorithms.

∑i=0

t

a i=

a t+ 1−1

a−1

CPSC 121 – 2014W T2 31


Proof:

Base case: t = 0

In this case the summation is a0 = 1, and

Induction step:

Pick an unspecified t ≥ 0. Assume that

Now

Hence by the principle of M.I., the theorem holds. QED

a1−1

a−1=1

CPSC 121 – 2014W T2 32


Example 3: prove that n ≥ 4,2n < n!

Rules for inequalities:

Start from one side (say the left side)

Work step by step towards the other.

When dealing with <, you are allowed to make the expression larger, but never smaller.

Example: if I am smaller than you, then I am still smaller than you when you stand on a bench.

CPSC 121 – 2014W T2 33


Proof: by induction on n.

Base case: n =

Induction step: we want to prove that n ≥ , if2n-1 < (n-1)! then 2n < n!

Consider an unspecified n ≥ .

Induction hypothesis: assume that 2n-1 < (n-1)!

Then 2n = 2(2n-1) < 2(n-1)! < n(n-1)! = n! this is where we “approximate” the induction hypothesis is used here

Hence by the principle of M.I., n ≥ 4,2n < n!

CPSC 121 – 2014W T2 34


Example 4: prove that for every n ≥ 1,

Note:

∑i=1

n1i2

≤2−1n

∑i=1

∞ 1

i2=π

2

6

CPSC 121 – 2014W T2 35


Example 5: consider the following DFA:

3

20

4

1

CPSC 121 – 2014W T2 36


Which strings does this DFA accept?

a) Strings whose length is divisible by 5.

b) Unsigned binary integers that are prime numbers.

c) Strings where every 1 is followed by 01.

d) Unsigned binary integers that are divisible by 5.


CPSC 121 – 2014W T2 37


What should we prove (ignoring empty strings)?

a) The DFA is in state 0 if and only if it has seen one of the strings 1010, 1111 or 11001.

b) The DFA is in state 0 if and only if s is divisible by 5.

c) The DFA is in state 0 if and only if s is terminated by the string 101.

d) The DFA is in state 0 if and only if s contains the substring 101 followed by zero or more 0's.


CPSC 121 – 2014W T2 38


Theorem: The DFA ends up in state r after reading a string s if and only if q Z, s = 5q+r∈ .

Proof: by induction on the length n of s.

Base case: n = 1

This can be verified easily by looking at the DFA.

Induction step: consider an unspecified n > 1.

Induction hypothesis: the theorem holds for strings with n – 1 characters.

Consider an unspecified string s with n bits.

CPSC 121 – 2014W T2 39


Proof (continued):

Suppose that s = bn-1bn-2...b2b1b0.

After reading bn-1bn-2...b2b1, the DFA is in state r. By the induction hypothesis, bn-1bn-2...b2b1 = 5q + r for some integer q. Now, s = 2 (bn-1bn-2...b2b1) + b0 = 2(5q+r) + b0 = 10q + 2r + b0 and so s = 5 (2q) + (2r + b0). So our DFA should have the following transitions:

r b0

2r+b0

0 0 0

1 0 2

2 0 4

3 0 5 + 1

4 0 5 + 3

r b0

2r+b0

0 1 1

1 1 3

2 1 5 + 0

3 1 5 + 2

4 1 5 + 4

CPSC 121 – 2014W T2 40


Proof (continued):

It does, and hence after reading all n bits of s, the DFA ends up in state r if and only if q Z, s = 5q+r∈ .

Hence by the principle of mathematical induction, the DFA ends up in state r after reading a string s if and only if q Z, s = 5q+r∈ . QED

Corollary: the DFA is in state 0 if and only if s is divisible by 5.

CPSC 121 – 2014W T2 41


Mathematical induction is how we analyze recursive algorithms:

Example:

(define (sum n)

(if (= n 0)

0

(+ n (sum (- n 1)))))

The type of induction we discussed so far works when the recursive call is with (- n 1)

∑i=0

n

i=n+ (∑i=0

n−1

i)

CPSC 121 – 2014W T2 42


Module Summary





CPSC 121 – 2014W T2 43


How do we handle more general recursions?

A recursive function/method has

One or more base casesWhen it can compute the answer directly

An induction stepIt computes recursively the solutions to one or more “smaller” sub-problems, and combines them to obtain its answer.

Our induction proofs mimic this structure.

It is similar to the templates you used in CPSC 110.

CPSC 121 – 2014W T2 44


We use a slightly different induction hypothesis.

Instead of proving that

∀x Z∈ +, Q(x) → Q(x+1)

We prove that

∀x Z∈ +, (Q(1) ^ Q(2) ^ ... ^ Q(x)) → Q(x+1)

We can also show that this type of induction is a valid proof technique.

The proof is the same proof by contradiction that we looked at earlier.

CPSC 121 – 2014W T2 45


CPSC 110 review: A binary tree is a data structure that is defined recursively: it is either

Empty, or

A node with some data, and two children that are themselves trees.

20

9 2 15

5

10

17 7

CPSC 121 – 2014W T2 46


Example 6: consider the following function:

(define (tree-size t)

(if (null? t)

0

(+ 1 (tree-size (left-child t))

(tree-size (right-child t)))))

How can we prove that it correctly computes the number of (non-null) nodes of the tree?

CPSC 121 – 2014W T2 47


We prove this using mathematical induction on the size of the tree.

Base case: t is null

In this case t contains exactly 0 nodes.

Induction step:

Assume the algorithm works for trees that are smaller than t.

Because the left sub-tree of t is smaller than t, the 1st recursive calls returns the size of the left sub-tree of t.

CPSC 121 – 2014W T2 48


Proof (continued)

Induction step (continued)

Similarly the right sub-tree of t is smaller than t, and so the 2nd recursive call returns the size of the right sub-tree of t.

But we return 1 + the sum of the values returned by the recursive calls.

This is exactly the size of t (1 for the root, and the sum of the sizes of the two sub-trees).

Hence by the principle of M.I., our algorithm computes correctly the size of every tree. QED

CPSC 121 – 2014W T2 49


Announcements:

Pre-class quiz #10: due Tuesday March 31st at 19:00.

Textbook sections:Epp, 4th edition: 6.1, 7.1Epp, 3rd edition: 5.1, 6.1Rosen, 6th edition: 2.1, 2.3 up to the top of page 136.Rosen, 7th edition: 2.1, 2.3 down to the bottom of page 141.

Assignment #5: due Tuesday April 7th at 17:00.

CPSC 121 – 2014W T2 50


Example 7: binary search

Suppose we have something like a list, but whose ith element and length can be found in a single step.

This structure is called a vector in Racket.

It is similar to an ArrayList in Java.

We assume that we have such a vector, sorted in increasing order.

Examples: (“Anh”, “Charles”, “Dora”, “Gregor”, “Wei”).

We want to find the position of a given element (for instance, “Dora”).

CPSC 121 – 2014W T2 51


Claim: the following algorithm (formerly known as B) works:

(define (binary-search avector first-pos last-pos x)

(if (> first-pos last-pos)

#f

(if (= first-pos last-pos)

(let ((mid-pos (quotient (+ first-pos last-pos) 2)))

(if (= x (vector-ref avector mid-pos))

(if (< x (vector-ref avector mid-pos))

CPSC 121 – 2014W T2 52


Proof: by induction on the size of the part of the vector that we are searching.

Base cases: size ≤ 1

If size is 0, then

If size is 1, then

Induction step: size ≥ 2

Suppose that the algorithm will find x (if it is in the vector) for every vector with fewer than size elements.

CPSC 121 – 2014W T2 53


Proof (continued)If x is at position mid-pos, then

Otherwise, x is either smaller than the element at position mid-pos, or larger.

If x is smaller then

If x is larger then

Hence by the principle of M.I., the algorithm returns the correct value. QED

CPSC 121 – 2014W T2 54


Example 8: every positive integer n greater than 1 can be written as a product of primes.

What base case(s) should we use?

a) n = 1

b) n = 2

c) n = 2, 3 or 5.

d) n < 100

e) n is prime.

f) None of the above.

CPSC 121 – 2014W T2 55


Proof: we prove the result by induction on n.

Base case: n is prime.

We can write n = n and n is a product of primes (with only 1 prime).

Induction step: n is composite.

Suppose that every value from 2 to n – 1 is a product of primes.

Since n is compositeWe can write n =

CPSC 121 – 2014W T2 56


Proof (continued)Since 2 < a < n,

Since 2 < b < n,

Then

Hence by the principle of M.I., every positive integer larger than 1 can be written as a product of primes. QED

CPSC 121 – 2014W T2 57


Example 9: prove that binary search makes at most ⌈log

2 (size+1)⌉ comparisons if size ≥ 1.

The proof is once again by induction on size.

Base case: size =

The algorithm uses comparison, which is

Induction step:

Consider an unspecified size .

Assume that

module 9: proof techniques (part 2) mathematical induction

Documents