chapter 3 3.1 algorithms 3.2 the growth of functions 3.3 complexity of algorithms 3.4 the integers...

1

Chapter 3

3.1 Algorithms3.2 The Growth of Functions3.3 Complexity of Algorithms3.4 The Integers and Division3.5 Primes and Greatest Common Divisors3.6 Integers and Algorithms3.7 Applications of Number Theory3.8 Matrices

2

Chapter 3

3.1 Algorithms– Searching Algorithms– Greedy Algorithms– The Halting Problem

3

Algorithm

• Definition 1: An algorithm is a finite set of precise instructions for performing a computation or for solving a problem.

• Example 1: Describe an algorithm for finding the maximum (largest) value in a finite sequence of integers.

4

• We perform the following steps1. Set the temporary maximum equal to the first

integer in the sequence. (the temporary maximum will be the largest integer examined at any stage of the procedure.)

2. Compare the next integer in the sequence to the temporary maximum, and if it is larger than the temporary maximum, set the temporary maximum equal to this integer.

3. Repeat the previous step if there are more integers in the sequence.

4. Stop when there are no integers left in the sequence. The temporary maximum at this point is the largest integer in the sequence.

5

Pseudocode• Pseudocode provides an intermediate step between an

English language description of an algorithm and an implementation of this algorithm in a programming language.

• Algorithm 1: Finding the maximum element in a finite sequence.

procedure max(a1, a2, . . . ,an: integers)

max := a1

for i: =2 to n if max < ai then max := ai

{max is the largest element}

6

Property of AlgorithmInput. Output. Definiteness. The steps of an algorithm must be defined precisely.Correctness. Finiteness. Effectiveness. Generality. The procedure should be applicable for all problems of the desired form, not just for a particular set of input values.

7

Searching Algorithms

Search Problem: Locating an element in an (ordered) list.

• Linear search• Binary search (ordered list)

8

The linear search• Algorithm 2 : the linear search algorithm procedure linear search (x: integer, a1, a2, …,an: distinct

integers) i :=1; while ( i ≤n and x ≠ ai)

i := i + 1If i ≤ n then location := iElse location := 0{location is the subscript of the term that equals x , or is

0 if x is not found}

9

The binary search• Algorithm 3: the binary search algorithmProcedure binary search (x: integer, a1, a2, …,an: increasing integers)

i :=1 { i is left endpoint of search interval} j :=n { j is right endpoint of search interval}While i < jbegin m := (i+j)/2 if x > am then i := m+1

else j := mendIf x = ai then location := i

else location :=0{location is the subscript of the term equal to x, or 0 if x is not found}

Example 3: to search for 19 in the list 1 2 3 5 6 7 8 10 12 13 15 16 18 19 20 22

10

Sorting• Sort:– Sorting is putting elements into a list in which the

elements are in increasing order.• E.g. 1) 7,2,1,4,5,9 -> 1,2,4,5,7,92) d,h,c,a,f -> a,c,d,f,h.

• Bubble sort• Insertion sort

11

Bubble Sort• ALGORITHM 4: The Bubble Sortprocedure bubble sort (a1, a2, …,an: real numbers with n ≥2)

for i := 1 to n-1 for j := 1 to n- i if aj > aj+1 then interchange aj and aj+1

{a1, a2, …,an is in increasing order}

• Example 4: Use the sort to put 3, 2, 4, 1, 5 into increasing order.

Bubble Sort

13

Insertion Sort• Algorithm 5: The Insertion Sortprocedure insertion sort (a1, a2, …,an: real numbers with n ≥2)

for j := 2 to nbegin i := 1 while aj > ai

i := i + 1 m := aj

for k :=0 to j-i-1 aj-k := a j-k-1

ai := m

end {a1, a2, …,an are sorted}

Example 5: Use the insertion sort to put the elements of the list 3, 2, 4, 1, 5 into increasing order.

14

Greedy Algorithm

• Optimization Problem: find the best solution.

• Algorithms that make what seems to be the best choice at each step are called greedy algorithms.

15

• Example 6: Consider the problem of making n cents change with quarters, dimes, nickels, and pennies, and using the least total number of coins.

• Algorithm 6: Greedy Change-Marking Algorithmprocedure change (c1, c2, …, cr: values of denominations of

coins, where c1 > c2 > … > cr ; n: a positive integer)

for i := 1 to r while n ≥ ci

begin add a coin with value ci to the change

n := n – ci

end

16

The Halting Problem • There is a problem that cannot be solved using any

procedure. • That is, there are unsolvable problems.• Halting Problem

FIGURE 2 Showing that the Halting Problem is Unsolvable.

Chapter 3

3.1 Algorithms3.2 The Growth of Functions3.3 Complexity of Algorithms3.4 The Integers and Division3.5 Primes and Greatest Common Divisors3.6 Integers and Algorithms3.7 Applications of Number Theory3.8 Matrices

17

Chapter 3

• 3.2 The Growth of Functions– Big-O Notation– Some Important Big-O Results– The Growth of Combinations of Functions– Big-Omega and Big-Theta Nation

18

The Growth of Functions

We quantify the concept that g grows at least as fast as f.What really matters in comparing the complexity of

algorithms?• We only care about the behavior for large problems.• Even bad algorithms can be used to solve small

problems.• Ignore implementation details such as loop counter

incrementation, etc. we can straight-line any loop.

19

Big-O Notation• Definition 1: let f and g functions from the set of integers or

the set of real numbers to the set of real number. We say that f(x) is O(g(x)) if there are constants C and k such that |f(x)| ≤ C |g(x)| whenever x > k.

• This is read as “ f(x) is big-oh of g(x) ”.• The constants C and k in the definition of big-O notation are

called witnesses to the relationship f(x) is O(g(x)).• Note: – Choose k– Choose C ; it may depend on your choice of k– Once you choose k and C, you must prove the truth of the

implication (often by induction).• Example 1: show that f(x)= x2+ 2x + 1 is O(x2) 20

Big-O Notation

FIGURE 1 The Function x2 + 2x + 1 is O(x2).

21

Big-O Notation

FIGURE 2 The Function f(x) is O(g(x)).22

Big-O Notation

• Example 2: show that 7x2 is O( x3 ).

• Example 4: Is it also true that x3 is O(7x2)?

• Example 3: show that n2 is not O(n).

23

Little-O Notation• An alternative for those with a calculus background:

• Definition: if then f is o(g), called little-o of g.

0)(

)(lim

ng

nf

n

24

• Theorem: if f is o(g) then f is O(g).• Proof: by definition of limit as n goes to infinity,

f(n)/g(n) gets arbitrarily small.

That is for any ε >0 , there must be n integer N such that when n > N, | f(n)/g(n) | < ε.Hence, choose C = ε and k= N . Q.E.D.

It is usually easier to prove f is o(g)• Using the theory of limits • Using L’Hospital’s rule• Using the properties of logarithmsetc 25

• Example : 3n + 5 is O(n2).• Proof: it’s easy to show using

the theory of limits.Hence, 3n+5 is o(n2) and so it is O(n2).Q.E.D.

053

2lim

n

n

n

26

Some Important Big-O Results

• Theorem 1: let where a0, a1, . . .,an-1 , an are real numbers

then f(x) is O(xn) .

• Example 5: how can big-O notation be used to estimate the sum of the first n positive integers?

011

1)( axaxaxaxf nn

nn

27

• Example 6: give big-O estimates for the factorial function and the logarithm of the factorial function, where the factorial function f(n) =n! is defined by

n! = 1* 2 * 3 * . . .*nWhenever n is a positive integer, and 0!=1.

28


• Example 7: In Section 4.1 ,we will show that n <2n whenever n is a positive integer.

Show that this inequality implies that n is O(2n) , and use this inequality to show that log n is O(n).

29


The Growth of Combinations of Functions

1 logn n n log n n2 2n n!

FIGURE 3 A Display of the Growth of Functions Commonly Used in Big-O Estimates.

30

Important Complexity Classes

Where j > 2 and c> 1.

• Example :Find the complexity class of the function

• Solution: this means to simplify the expression.

Throw out stuff which you know doesn’t grow as fast.

We are using the property that if f is O(g) then f + g is O(g).

)!()()()(

)log()()(log)1(2 nOcOnOnO

nnOnOnOOnj

)2)(33!( 1002 nnn nnnnn

31

if a flop takes a nanosecond, how big can a problem be solved (the value of n ) in

a minute? a day? a year?For the complexity class O(n n! nn)

32


a minute= 60*109= 6*1010 flopsa day= 24*60*60= 8.65*1013 flops a year= 365*24*60*60*109= 3.1536*1016 flopsWe want to find the maximal integer so that

n*n!*nn < 6*1010

n*n!*nn < 8.65*1013

n*n!*nn < 3.1536*1016

33


Maple Program:for k from 1 to 10 do (k,k*factorial(k)*kk)end do;

1, 12, 16

3, 4864, 24576

5, 1875006, 201553920

7, 29054597040 8, 5411658792960

9, 126528432343488010, 362880000000000000

So, n=7,8,9 for a minute, a day, and a year.34



• Theorem 2: suppose that f1(x) is O(g1(x)) and f2(x) is O(g2(x)). Then (f1 + f2)(x) is O(max( |g1(x)| , |g2(x)| )).

• Corollary 1: suppose that f1(x) and f2(x) are both O(g(x)). Then (f1 + f2)(x) is O(g(x)).

35

• Theorem: If f1 is O(g1) and f2 is O(g2) then

1. f1 f2 is O(g1g2)

2. f1+f2 is O(max {g1 ,g2})

36


• Theorem 3 :suppose that f1(x) is O(g1(x)) and f2(x) is O(g2(x)).

Then (f1f2)(x) is O(g1(x) g2(x)).

• Example 8: give a big-O estimate for f(n)=3n log(n!) + (n2 +3) log n where n is a positive integer.• Example 9: give a big-O estimate for f(x)=(x+1)log(x2+1) + 3x2

37

Properties of Big-O• f is O(g) iff • If f is O(g) and g is O(f) then

• The set O(g) is closed under addition: if f is O(g) and h is O(g) then f+h is O(g) • The set O(g) is closed under multiplication by a scalar a (real

number):if f is O(g) then af is O(g) That is ,O(g) is a vector space. (The proof is in the book.)

Also, as you would expect,• If f is O(g) and g is O(h), then f is O(h) .In particular

)()( gOfO )()( gOfO

)()()( hOgOfO 38

• Note : we often want to compare algorithms in the same complexity class

• Example:Suppose Algorithm 1 has complexity n2 – n +1

Algorithm 2 has complexity n2/2 + 3n + 2Then both are O(n2) but Algorithm 2 has a smaller

leading coefficient and will be faster for large problems.

Hence we writeAlgorithm 1 has complexity n2 +O(n)

Algorithm 2 has complexity n2/2 + O(n)

39

Big-Omega and Big-Theta Nation• Definition 2: Let f and g be functions from the set of integers

or the set of real numbers to the set of real numbers. • We say that f(x) is Ω(g(x)) if there are positive constants C and

k such that |f(x)|≥ C|g(x)| Whenever x > k. ( this is read as “f(x) is big-Omega of g(x)” .)

• Example 10 :The function f(x) =8x3+ 5x2 +7 is Ω(g(x)) , where g(x) is the function g(x) =x3.

• This is easy to see because f(x) =8x3+ 5x2 +7 ≥ x3 for all positive real numbers x. this is equivalent to saying that

g(x) = x3 is O(8x3+ 5x2 +7 ) ,which can be established directly by turning the inequality around.

40

• Definition 3: Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers.

• We say that f(x) is Θ(g(x)) if f(x) is O(g(x)) and f(x) is Ω(g(x)). • When f(x) is Θ(g(x)) , we say that” f is big-Theta of g(x)” and

we also say that f(x) is of order g(x).

• Example 11: we showed (in example 5) that the sum of the first n positive integers is O(n2). Is this sum of order n2?

• Example 12: show that 3x2 + 8x(logx) is Θ(x2).

41

• Theorem 4: let , where a0, a1, . . .,an-1 , an are real numbers with

an≠0 . Then f(x) is of order xn .

• Example 13: the ploynomials 3x8+10x7+221x2+1444

x19-18x4-10112 -x99+40001x98+100003x

are of orders x8, x19 and x99 ,respectively.

011

1)( axaxaxaxf nn

nn

42

Chapter 3

3.3 Complexity of Algorithms– Time Complexity– Understanding the complexity of Algorithms

43

Complexity of Algorithm

• Computational Complexity (of the Algorithm)• Time Complexity: Analysis of the time required.• Space Complexity: Analysis of the memory

required.

44

Time Complexity• Example 1: Describe the time complexity of Algorithm

1 of section 3.1 for finding the maximum element in a set (in terms of number of comparisons).

• Algorithm 1: Finding the maximum element in a finite sequence.

procedure max(a1, a2, . . . ,an: integers)

max := a1

for i: =2 to n if max < ai then max := ai

{max is the largest element}45

• Example 2: Describe the time complexity of the linear search algorithm.

• Algorithm 2 : the linear search algorithm procedure linear search (x: integer, a1, a2, …,an: distinct integers)

i :=1; while ( i ≤n and x ≠ ai)

i := i + 1If i ≤ n then location := iElse location := 0{location is the subscript of the term that equals x , or is 0 if x isnot found}

46

• Example 3: Describe the time complexity of the binary search algorithm in terms of the number of comparisons used .

(and ignoring the time required to compute m= in each iteration of the loop in the algorithm)

• Algorithm 3: the binary search algorithmProcedure binary search (x: integer, a1, a2, …,an: increasing integers)

i :=1 { i is left endpoint of search interval} j :=n { j is right endpoint of search interval}While i < jbegin m := if x > am then i := m+1

else j := mendIf x = ai then location := I

else location :=0{location is the subscript of the term equal to x, or 0 if x is not found}

2/)( ji

2/)( ji

47

• Example 4: Describe the average-case performance of the linear search algorithm, assuming that the element x is in the list.

• Example 5: What is the worst-case complexity of the bubble sort in terms of the number of comparisons made?

• ALGORITHM 4: The Bubble Sortprocedure bubble sort (a1, a2, …,an: real numbers with n ≥2)

for i := 1 to n-1 for j := 1 to n- i if aj > aj+1 then interchange aj and aj+1

{a1, a2, …,an is in increasing order}48

• Example 6: What is the worst-case complexity of the insertion sort in terms of the number of comparisons made?

• Algorithm 5: The Insertion Sortprocedure insertion sort (a1, a2, …,an: real numbers with n ≥2)

for j := 2 to nbegin i := 1 while aj > ai

i := i + 1 m := aj

for k :=0 to j-i-1 aj-k := a j-k-1

ai := m

end {a1, a2, …,an are sorted}49

Understanding the complexity of Algorithms

50

• Solvable (in polynomial time, or in exponential time)

• Tractable: A problem that is solvable using an algorithm with polynomial worst-case complexity.

• Intractable: The situation is much worse for problems that cannot be solved using an algorithm with worst-case polynomial time complexity. The problems are called intractable.

• NP problem.• NP-complete problem.• Unsolvable problem: no algorithm to solve them.

51

• Big-O estimate on the time complexity of an algorithm provides an upper, but not a lower, bound on the worst-case time required for the algorithm as a function of the input size.

• Table 2 displays the time needed to solve problems of various sizes with an algorithm using the indicated number of bit operations. Every bit operation takes nanosecond. Times of more than 10100 years are indicated with an asterisk.

52

Chapter 3

3.4 The Integers and Division– Division– The Division Algorithm– Modular Arithmetic– Applications of Congruences– Cryptology

53

Division• Definition 1: if a and b are integers with a≠0, we say

that a divides b if there is an integer c such that b=ac. When a divides b we say that a is a factor of b and that b is a multiple of a. the notation a|b denotes that a divides b. we write a | b when a does not divide b.

• Example 1: Determine whether 3|7 and whether 3|12.

• Example: Determine whether 3|0.

/

54

The Division Algorithm• Theorem 2 the division algorithm :let a be an integer

and d a positive integer. Then there are unique integers q and r, with 0 ≤ r < d, such that

a= dq+r• Definition 2: In the equality give in the division

algorithm, d is called the divisor, a is called the dividend, q is called the quotient, and r is called the remainder. This notation is used to express the quotient and remainder.

q = a div d, r = a mod d.• Example 4: What are the quotient and remainder when

-11 is divided by 3?56

Modular Arithmetic• Definition 3: if a and b are integers and m is a

positive integer, then a is congruent to b modulo m if m divides a - b.

• we use the notation a≡b (mod m) to indicate that a is congruent to b modulo m.

• if a and b are not congruent modulo m, we write a ≡b (mod m) ./

57

Modular Arithmetic

• Theorem 3: let a and b be integers, and let m be a positive integer. Then a≡b (mod m) if and only if a mod m = b mod m .

• Example 5: determine whether 17 is congruent to 5 modulo 6 and whether 24 and 14 are congruent modulo 6.

58

• Theorem 4 : let m be positive integer. The integers a and b are congruent modulo m if and only if there is an integer k such that a = b + km .

• Theorem 5: let m be a positive integer. If a≡b(mod m ) and c ≡d (mod m), then

a+c≡b+d (mod m) , ac ≡ bd (mod m)

• Example 6: because 7≡2 (mod 5) and 11≡1 (mod 5) , it follows from theorem 5 that

18=7+11 ≡2+1=3(mod 5) , and that 77=7*11 ≡2*1=2 (mod 5)

59

Modular Arithmetic

• Corollary 2: let m be a positive integer and let a and b be integers. Then

(a+b) mod m = ((a mod m)+(b mod m)) mod mAnd

ab mod m =((a mod m)(b mod m)) mod m.

60

Applications of Congruences

• Hashing Functions• Pseudorandom Numbers• Cryptology

61

Hashing Functions

• How can memory locations be assigned so that customer records can be retrieved quickly?

• Hashing function and key

• h(k) = k mod m; m is the number of available memory locations.

• Collision: one way to re solve a collision is to assign the first free location.

62

Pseudorandom Numbers• The numbers generated by systematic method are not

truly random, they are called pseudorandom numbers.• Linear Congruential Method(m, a, c, x0 :integers):• Modulus m• Multiplier a, 2 a < m• Increment c, 0 c < m• Seed x0 , 0 x0 < m

• xn+1= (axn+c) mod m

• For example: m=9, a=7, c=4, x0 =3, then

(x1, x2, x3, x4, x5, x6, x7, x8, x9)=(7, 8, 6, 1, 2, 0, 4, 5, 3)

x10=x163

Cryptology• Important Application of Congruences• Earliest known uses by Julius Caesar.• Shifting each letter three letters forward in the alphabet. • To express the process mathematically:• Let U={0,.., 25}, V={A, .., Z} and g: V -> U is a bijection

function defined as the table below.• Define function f : U -> U, where f(p)=(p+3) mod 26.• The Encryption function h:V->V, where h(x)=g-1( f(g(x) ) )• The decryption function f-1(p)=(p-3) mod 26.

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

D E F G H I J K L M N O P Q R S T U V W X Y Z A B C64

Applications of Congruences

• Example 9:• What is the secret message produced from

the message “MEET YOU IN THE PARK” using the Caesar cipher.

• HW: Example 10, p208

65

Chapter 3

3.5 Primes and Greatest Common Divisors‒ Primes‒ Greatest common divisors and least

common multiples

66

Primes

• Definition 1: A positive integer p greater than 1 is called prime if the only positive factors of p are 1 and p. A positive integer that is greater than 1 and is not prime is called composite.

• Remark: The integer n is composite if and only if there exists an integer a such that a|n and 1< a < n.

• Example 1: The integer 7 is prime because its only positive factors are 1 and 7, whereas the integer 9 is composite because it is divisible by 3.

67

Primes• Theorem 1: The fundamental theorem of arithmetic Every positive integer greater than 1 can be written uniquely as

a prime or as the product of two or more primes where the prime factors are written in order of nondecreasing size.

• Example 2: The prime factorizations of 100, 641 , 999 and 1024 are given by

100=2*2*5*5=2252

641=641 999=3*3*3*37=33*37

1024=2*2*2*2*2*2*2*2*2*2=210

68

Primes

• Theorem 2: If n is a composite integer , then n has a prime divisor less than or equal to .

• Example 3: Show that 101 is prime.

• Example 4: Find the prime factorization of 7007.

n

69

Primes

• Theorem 3: There are infinitely many primes .• Proof: We will prove this theorem using a proof by

contradiction. We assume that there are only finitely many primes, p1, p2, … , pn.

Let Q=

1...21 nppp

70

Greatest Common Divisors

• Definition 2: Let a and b be integers, not both zero.• The largest integer d such that d|a d|b is called the

greatest common divisor of a and b.• The greatest common divisor of a and b is denoted

by gcd(a,b).

• Example 10: what is the greatest common divisor of

24 and 36?

71


• Definition 3: The integers a and b are relatively prime if their greatest common divisor is 1.

• Example 12: Prove that y the integers 17 and 22 are relatively prime.

72


• Definition 4: The integers a1,a2 …,an are pairwise relatively prime if gcd(ai , aj)=1 whenever 1 i≦ <j ≦n.

• Example 13: determine whether the integers 10 , 17 and 21 are pairwise relatively prime and whether the integers 10 , 19 and 24 are pairwise relatively prime.

• Example 14: Because the prime factorizations of 120 and 500 are 120=23*3*5 and 500=22*53, the greatest common divisor is

gcd(120,500)=2 min(3 , 2) 3 min(1 , 0) 5 min(1,3)=223051=2073

Least Common Multiples

• Definition 5: The least common multiple of the positive integers a and b is the smallest positive integer that is divisible by both a and b.

• The least common multiple of a and b is denoted by

lcm(a , b).• Example 15: What is the gcd and lcm of 233572 and

2433?

74

Greatest Common Divisors andLeast Common Multiples

• Theorem 5: Let a and b be positive integers. Then ab = gcd(a ,b)* lcm(a , b)

75

Chapter 3

3.6 Integers and Algorithms‒ Representations of integers‒ Algorithms for integer operations‒ Modular Exponentiation‒ The Euclidean Algorithm

76

Representations of integers

• Theorem 1: Let b be a positive integer greater than 1. Then if n is a positive integer, it can be expressed uniquely in the form

where k is a nonnegative integer, a0, a1, …,ak are nonnegative integers less than b, and ak ≠0.

011

1 ... abababan kk

kk

77

• Example 1: What is the decimal expansion of the integer that has (1 0101 1111)2 as its binary expansion?

• Example 2: What is the decimal expansion of the hexadecimal expansion of (2AE0B)16 ?

78

• Example 3: Find the base 8, or octal, expansion of (12345)10

• Example 4: Find the hexadecimal expansion of (177130)10?

79

• Algorithm 1: Construction Base b Expansions procedure base b expansion(n:positive integer) q: = n k: =0 while q ≠ 0 begin ak : =q mod b

q: = k: =k+1 end {the base b expansion of n is (ak-1 . . . a1 a0)b}

bq /

80

Algorithms for integer operations• Algorithm 2: Addition of Integers Procedure add(a , b:positive integers) {the binary expansions of a and b are (an-1 . . . a1 a0)2 and

(bn-1 . . . b1 b0)2 respectively}

c : =0for j: =0 to n-1Begin d : = sj : = aj+bj+c-2d

c : =dendsn:=c {the binary expansion of the sum if (sn sn-1. . . s1 s0)2 }

2/)( cba jj

82

Algorithms for integer operations

83

Example 7: Add a=(1110)2 and b=(1011)2.


• Algorithm 3 : Multiplying Integers procedure multiply(a, b : positive integers) {the binary expansions of and b are(an-1 . . . a1 a0)2 and

(bn-1 . . . b1 b0)2 respectively}

for j:=0 to n-1 Begin if bj =1 then cj=a shifted j places

else cj:=0

end{c0 c1 . . . cn-1 are the partial products}

p :=0 for j:=0 to n-1 p: = p +cj {p is the value of ab} 84


85

Example 9: Find the product of a= (110)2 and b=(101)2

Algorithms for integer operations• Algorithm 4 : Computing div and mod

procedure division algorithm(a :integers ,d: positive integer) q: =0 r: =|a| while r≧d begin r := r-d q :=q+1 end if a<0 then

if r=0 then q:=-q else begin r := d-r q := -(q+1) end {q = a div d is the quotient, r = a mod d is the remainder} 86

Modular Exponentiation• In cryptography it is important to be able to find bn mod

m efficiently, where b, n and m and large integers. It’s impractical to first compute bn and then find its remainder when divided by m because bn will be a huge number. Instead, we can use an algorithm that employ expansion of the exponent n , say n = (ak-1 . . . a1 a0)2 .

• Before we present this algorithm, we illustrate its basic idea. We will explain how to use the binary expansion of n to compute bn .First , note that

011

1011

1 222...2 ... aaaaaan bbbbbk

kk

k

87

Modular Exponentiation

• To compute bn , we find the values of b, b2,(b2)2=b4, (b4)2=b8, . . . , .

• We multiply the terms in this list, where aj=1 .

This gives us .

• For example, to compute 311 we first note that 11 = (1011)2, so that 311= 383231.

• By successively squaring, we find that 32=9, 34=81, 38=6561.

• Consequently,311=383231=6561*9*3= 177,147

k

b2

j

b2

nb

88

Modular Exponentiation• Algorithm 5: Modular Exponentiation

procedure modular exponentiation(b:integer ,

n=(ak-1 . . . a1 a0)2 ,m: positive integer)

x: = 1

power := b mod m

for i=0 to k-1

begin

for ai =1 then x :=(x*power) mod m

power :=(power*power) mod m

End

{x equals bn mod m} 89

Example 11: Use Algorithm 5 to find 3644 mod 645.

The Euclidean Algorithm

• Lemma 1: Let a=bq+r ,where a, b, q, and r are integers. Then gcd(a,b)=gcd(b,r).

• Algorithm 6: The Euclidean Algorithm

procedure gcd(a.b:integers)

x: = a

y: = b

while y0

begin

r := x mod y

x := y

y := r

end {gcd(a,b) is x} 90

The Euclidean Algorithm

• Example 12: Find the GCD of 414 and 662 using the Euclidean Algorithm.

91

Chapter 3

3.7 Applications of Number Theory‒ Some Useful Results‒ Linear Congruences‒ The Chinese Remainder Theorem‒ Computer Arithmetic with Large Integers‒ Pseudoprimes‒ Public Key Cryptography

92

Some Useful Results

• Theorem 1: If a and b are positive integers, then there exist integers s and t such that gcd(a ,b) = sa+tb .

• Example 1: express gcd(252 , 198) =18 as a linear combination of 252 and 198 .

93

Some Useful Results

• Lemma 1: If a, b, and c are positive integers such that gcd(a , b) = 1 and a|bc, then a|c .

• Lemma 2 : If p is a prime and p|a1a2. . .an, where each ai is an integer , then p|ai for some i.

• Theorem 2: Let m be a positive integer and let a, b ,and c be integers. If ac≡ bc (mod m) and gcd(c, m) = 1 , then a≡b (mod m).

94

Linear Congruences• A congruence of the form ax≡b (mod m) where m is a positive

integer , a and b are integers , and x is variable, is called a linear congruence.

• Such congruences arise throughout number theory and its applications.

• How can we solve the linear congruence ax≡b (mod m) ? That is, find the x that satisfy this congruence.

• One method that we will describe uses an integer ā such that aā≡1 (mod m), if such an integer exist.

• Such an integer ā is said to be an inverse of a modulo m.• Theorem 3 guarantees that an inverse of a modulo m exists

whenever a and m are relatively prime.95

Linear Congruences• Theorem 3: If a and m are relatively prime integers

and m>1, then an inverse of modulo m exist. Furthermore, this inverse is unique modulo m.

(there is a unique positive integer ā less than m that is an inverse of a modulo m and every other inverse of a modulo m is congruent to ā modulo m.)

When we have an inverse of a modulo m, that is, ax≡1 (mod m) , we can easily solve the congruence ax≡b (mod m).

96

The Chinese Remainder Theorem• Example 3: Find an inverse of 3 modulo 7?

Theorem 5, section 3.4, p204.Let m be a positive integer. If a≡b (mod m) and c≡d (mod m), then

a+c≡b+d (mod m) and ac≡bd (mod m). • Example 4: What are the solutions of the liner congruence 3x ≡4 ( mod

7)?

• Example 5: In the first century, the Chinese mathematician Sun-Tsu asked:

There are certain things whose number is unknown. When divider by 3, the remainder is 2; when divided by 5, the remainder is 3; and when divided by 7 , the remainder is 2. What will be the number of things?

97

The Chinese Remainder Theorem

• Theorem 4: The Chinese Remainder Theorem Let m1, m2, . . . ,mn be pairwise relative prime positive

integers and a1, a2,. . . ,an arbitrary integers. Then the system x≡a1 ( mod m1)

x≡a2 ( mod m2)

… x≡an ( mod mn)

has a unique solution modulo m= m1, m2, . . . ,mn .

(That is , there is solution x with 0 x ≦ ＜ m, and all other solutions are congruent modulo m to this solution.) 98

射雕英雄傳第一千四比二十七首瑛姑說道 : 『 . . . 今有物不知其數，三三數支謄二，

五五數之謄三，七七數之謄二，問物幾何 ? 』

黃蓉笑道 : 『這容易得緊，以三三數之，餘數乘以七十 ; 五五數之，餘數乘以二十一，七七數之，餘數乘以十五。三者相加，如不大於一百零五，即為答數 ; 否則須減去一百零五或其倍數。』

黃蓉道 : 『也不用這般硬記，我念一首詩給你聽，那就容易記了 : 三人同行七十稀，五樹梅花二一枝，七子團員正半月，餘百零五便得知。』

99

The Chinese Remainder Theorem

• Example 6: Solve the system of congruences in Example 5 by using theorem 4.

• Example 5:there are certain things whose number is unknown. When divider by 3, the remainder is 2; when divided by 5, the remainder is 3; and when divided by 7 , the remainder is 2. What will be the number of things?

100

Computer Arithmetic with Large Integers

• Suppose that m1, m2, . . . ,mn are pairwise relatively prime integers greater than or equal to 2 and let m be their product. By the Chinese Remainder Theorem, we can show that an integer a with 0≤ a < m can be uniquely represented by the n-tuple consisting of its remainders upon division by mi , i= 1, 2,. . .,n.

• We can uniquely represent a by (a mod m1, a mod m2, . . ., a mod mn)

101


• Example 7: What are the pairs used to represent the nonnegative integers less than 12 when they are represented by the ordered pair where the first component is the remainder of the integer upon division by 3 and the second component is the remainder of the integer upon division by 4?

102

Pseudoprimes • Theorem 5: Fermat’s Little Theorem If p is prime and a is an integer not divisible by p, then

ap-1 ≡1 (mod p) Furthermore, for every integer a we have

ap ≡a (mod p)• Unfortunately, there are composite integer n, such

that 2n-1≡1 (mod p). Such integers are called pseudoprimes to the base 2.

• Example 9: Explain why the integer 341 is a pseudoprime to the base 2.

103


• Definition 1: Let b be a positive integer. If n is a composite positive integer, and bn-1 ≡1 (mod n), then n is called a pseudoprime to the base b.

• Definition 2: A composite integer n that satisfies the congruence bn-1 ≡1 (mod n) for all positive integers b with gcd(b , n)=1 is called a Carmichael number.

• (This numbers are named after Robert Carmichael, who studied them in the early twentieth century)

• Example 10: The integer 561 is a Carmichael number.

104

Private key cryptosystems (Section 3.4, Example 9, p207)

• c=(p+k) mod 26, where p, c represent a letter, k is an encryption key.

• Everybody knowing this key can both encrypt and decrypt messages easily.Private

• Two people need to securely exchange the key in advance.

105

Private Key Cryptography

• In 1976, three researchers at M.I.T. – Ronald Rivest, Adi Shamir, and Leonard Adleman – introduced to the world a public key cryptosystem, known as the RSA system.

• The RSA cryptosystem is based on modular exponentiation modulo the product of two large primes, which can be done rapidly using Algorithm 5 in section 3.6.

• Each individual has an encryption key consisting of a modulus n=pq, where p and q are large primes, say, with 200 digits each, and an exponent e that is relatively prime to (p-1)(q-1).

106

Public Key Cryptography

• To produce a usable key, two large primes must be found. This can be done quickly on a computer using probabilistic primality test. (Example 16, Section 6.2, p 412-413 text book)

• However, the product of these primes n=pq, with approximately 400 digits, cannot be factored in a reasonable length of time. This is an important reason why decryption cannot be done quickly without a separate decryption key.

107

Public Key Cryptography

• In the RSA encryption method, messages are translated into sequences of integers.

• These integers are grouped together to form larger integers, each representing a block of letters.

• The encryption proceeds by transforming the integer M, representing the plaintext (the original message), to an integer C, representing the ciphertext (the encryption message), using the function C=Me mod n.

108

RSA Encryption

• Example 11: Encrypt the message STOP using the RSA cryptosystem with p=43 and q=59, so that n=43 x 59 = 2537, and with e=13. Note that

Gcd(e, (p-1)(q-1)) = gcd(13, 42 x 58)=1.

109

RSA Encryption

• The plaintext message can be quickly recovered when the decryption key d, an inverse of e modulo (p-1)(q-1), is known. Such inverse exist because gcd(e, (p-1)(q-1))=1).

• de≡1 (mod (p-1)(q-1)), there exist an integer k, such that de=k(p-1)(q-1)+1.

• It follows that Cd≡(Me)d=Mde=M1+k(p-1)(q-1) (mod n).• By Fermat’s Little Theorem (theorem 5)[assuming

that gcd(M,p)=gcd(M,q)=1, which holds except in rare cases], it follows that Mp-1≡1 (mod p) and Mq-1 ≡1 (mod q).

110

RSA Decryption

• Consequently,Cd ≡M(Mp-1)k(q-1) ≡M (mod p)

and

Cd ≡M(Mq-1)k(p-1) ≡M (mod q) • Because gcd(p,q)=1, it follows by the Chinese

Remainder Theorem that Cd ≡M (mod pq)

111

RSA Decryption

• Example 12: We receive the encrypted message 0981 0461. What is the decrypted message if it was encrypted using the RSA cipher form example 11.

112

RSA Decryption

Chapter 3

3.8 Matrices‒ Matrix Arithmetic‒ Algorithms for Matrix Multiplication‒ Transposes and Powers of Matrices‒ Zero-One Matrices

113

Matrix Arithmetic

• Definition 1: • A matrix is a rectangular array of numbers. • A matrix with m rows and n columns is called an m × n matrix. • The plural of matrix is matrices. A matrix with the

same number of rows as columns is called square. • Two matrices are equal if they have the same

number of rows and the same number of columns and the corresponding entries in every position are equal.

114

Matrix Arithmetic• Definition 2: Let • The ith row of A is the

1 x n matrix [ai1,ai2,. . .,ain].• The jth column of A is

the n x 1 matrix

nnnn

n

n

aaa

aaa

aaa

A

21

22221

11211

nj

j

j

a

a

a

2

1

The (i, j)th element or entry of is the element aij , that is , the number in the ith row and jth column of A.A convenient shorthand notation for expressing the matrix A is to write A =[aij], which indicates that A is the matrix with its (i, j)th element equal to aij.

115

Matrix Arithmetic

• Definition 3: Let A=[aij] and B=[bij] be m x n matrices. The sum of A and B, denoted by A+B,

is the m x n matrix that has aij+bij as its (i, j)th element. In other words, A+B= [aij+bij].

• Example 2: we have

211

031

143

043

322

101

252

313

244

116

Matrix Arithmetic

• Definition 4: • Let A be an m x k matrix and B be k x n matrix.• The product of A and B, denoted by AB, is the m x n

matrix with its (i , j )th entry equal to the sum of the products of the corresponding elements from the

ith row of A and the jth column of B. • In other words, if AB=[cij], then

cij = ai1b1j + ai2b2j +. . . +aikbkj

117

Matrix Arithmetic

118

Algorithms for Matrix Multiplication• Algorithm 1 : Matrix

Multiplicationprocedure matrix multiplication (A, B: matrices)for i := 1 to m for j := 1 to n begin cij :=0

for q := 1 to k cij :=cij + aiqbqj

end

{C= [cij] is the product of A and B}

• Example 6: In which order should the matrices A1, A2, and A3, where

• A1 is 30x20 , A2 is 20x40 , A3 is 40x10,

• all with integer entries – be multiplied to use the least number of multiplications of integers?

119

Transposes and Powers of Matrices

• Definition 5: the identity matrix of order n is the n x n matrix

In = [δij]

where δij =1 if i = j and δij = 0 if i ≠ j. Hence,

1000

0010

0001

nI

120

Transposes and Powers of Matrices

• Definition 6: Let A=[aij] be an m x n matrix.• The transpose of A, denoted by At, is the n x m matrix

obtained by interchanging the rows and columns of A .

• In other words, if At=[bij], then bij = aji for i=1,2,. . .,n and j = 1,2,. . .,m .

• Definition 7: A square matrix A is called symmetric if A = At. • Thus A =[aij] is symmetric if aij = aji for all i and j with

1≤ i ≤ n and 1 ≤ j ≤ n .121

Symmetric Matrix

122

Zero-One Matrices

• Definition 8: Let A=[aij] and B=[bij] be m x n zero-one matrices.

• Then the join of A and B is the zero-one matrix with (i , j )th entry aij v bij.

The join of A and B is denoted by A v B.• The meet of A and B is the zero-one matrix with (i ,

j )th entry aij Λ bij.

The meet of A and B is denoted by A Λ B.

123

Zero-One Matrices

• Definition 9: Let A=[aij] be an m x k zero-one matrix and B=[bij] be a k x n zero-one matrix .

• Then the boolean product of A and B,denote by A⊙B , is the m x n matrix with with (i , j)th entry cij where

• Example 10: find the Boolean product of A and B, where

)ba( . . .)ba ()b(a =c kjik 2ji2 1ji1 ij

110

011,

01

10

01

BA

124

Zero-One Matrices

• Algorithm 2: The Boolean Productprocedure Boolean product(A, B: zero-one matrices)for i := 1 to m for j := 1 to n begin cij :=0

for q := 1 to k end

{C= [cij] is the Boolean product of A and B}125

)b(ac :=c qjiq ij ij

Zero-One Matrices

• Definition 10: Let A be a square zero-one matrix ant let r be a positive integer.

• The rth Boolean power of A is the Boolean product of r factors of A. The rth Boolean product of A is denoted by A[r]

• Hence,

• (this is well defined because the Boolean product of matrices is associative.)

• We also define A[0] to be In126

r times

[r] A⊙⊙ C⊙A ⊙AA

Zero-One Matrices

• Example 11: Let .

Find A[n] for all positive integers n.

127

011

001

100

A

chapter 3 3.1 algorithms 3.2 the growth of functions 3.3 complexity of algorithms 3.4 the integers...

Documents