advanced algebraic algorithms on integers and polynomials prepared by john reif, ph.d. analysis of...
TRANSCRIPT
Advanced Algebraic Algorithms on Integers and
Polynomials
Prepared by
John Reif, Ph.D.
Analysis of Algorithms
Integer and Polynomial Computations
a) Newton Iteration: application to divisionb) Evaluation and Interpolation via Chinese
Remaindering
Main Lecture Material: Algorithms on Integers
Advanced Lecture Material:Extension of Algorithms to Polynomials
Taylor Expansion
20
0 0 0 0
(x-x )(x) = (x ) + (x - x ) ' (x ) + " (x ) + ...
2f f f f
Taylor Expansion (cont’d)
• To find root f(x), use Newton Iteration:
• Example:– To find reciprocal of x choose
find root
i1
i
(x ) = -
'(x )i i
fx x
f
1( ) 1f y
xy
1y
x
1
( )= (2 )
'( )i
i i i ii
f yy y y y x
f y
Application of Newton Iteration to Reciprocal of an Integer• Input integer x, accuracy bound k• Initialize if x has n bits
0 2 ny
1
2
1 1
2 2 20 0
i=1 to k
(2 )
1 where 1
2 let error = 1
then = 1 = 1 (2 )
1 = ( ) = ( ) 2 since
2
k
k k
i i i
k k
k k
k k k k
k
for do
y y y x
output y y x
proof y x
y x xy y x
Application of Newton Iteration to Reciprocal of an Integer (cont’d)• Theorem Integer Reciprocal can
be computed to accuracy 2-n in O(log n) integer mults and additions.
Steven Cooks’s Improvement (His Harvard Ph.D. Thesis)
• Key Trick: Since
we need only compute yk up to 2k+1 bit accuracy of error on kth stage
• Total Time Cost
• Where M(n) is time cost to multiply two n bit integers
2
11
2kky x
log1
0
(2 ) ( ( ))n
k
k
c M O M n
Other Applications of Newton Iterations on Integers
• O(M(n)) time algorithms:• Quotient + divisor of integer
division• Square root• Sin, cosine, etc.
• Used in practice!
• Reciprocal = polynomial r(x)
• where
and (x) has degree < n-1
2 2
( ( ))( )
nxP x
P x
2 2( ) ( ) ( )nr x p x x x
Advanced Topic: Extension to Polynomial Reciprocal
Definition of Polynomial Reciprocal
Algorithm: Reciprocal (P(x))
• Input polynomial
degree n-1, n is power of 2
1
0
( )n
ii
i
P x a x
0
12
1 1 1/ 2
(3/ 2) 2 21 1
2
1[1] 1 else
[2] (x) Reciprocal ( ( )) where ( )
[3] ( ) 2 ( ) ( ) ( )
( )[4] ( )
nn i
ii n
n
n
if n then returna
r P x P x a x
R x r x x r x P x
R xreturn r x
x
Proof of the Reciprocal Algorithm• Theorem: The Algorithm Correctly
Computes Reciprocal(P(x))
• Proof by induction
00
21 2
1 2
11 ( ) ( )
let ( ) ( ) ( )
where deg ( ) 1, deg ( ) 12 2
n
basis n P x a so r xa
inductive step P x P x x P x
n nP P
Proof Algorithm Computes Reciprocal (P(x)) (cont’d)
• By induction hypothesis, if
1 1
21 1 1
1
( ) = reciprocal ( ( )) then
( ) ( ) ( )
nwhere ( ) has degree < 1
2
n
r x P x
r x P x x x
x
Proof Algorithm Computes Reciprocal (P(x)) (cont’d)
• At line [3] we compute
(3/ 2) 2 21 1
21 2
2 2 (3/ 2) 21 1 1 2
2
21 1 1 2
( ) 2 ( ) ( ) ( )
since ( ) ( ) ( )
( ) ( ) 2 ( ) ( ) 2 ( ) ( )
( ) ( ) ( ) ( )
n
n
n n
n
R x r x x r x P x
P x P x x P x
P x R x r x P x x r x P x x
r x p x x r x p x
Proof Algorithm Computes Reciprocal (P(x)) (cont’d)
21 1 1
2
3 4 21 1 2
3 4 2 4
2
2 2 2
Substituting + (x) for ( ) ( ),
we get
( ) ( ) ( ) ( ) ( )
= 0( )
( )But r( ) so
( ) ( ) ( )
n
nn
n n
n
n n
x r x p x
R x p x x x x r x p x
x x
R xx
x
r x p x x o x
Modular Arithmetic
• Assume relatively prime P0, P1, …, Pk-1
• Let given x, 0 x < p
1
0
k
ii
p p
0 -11 1 ( , ..., ) where
mod for 0, ..., -1k
i i
x x x
x x p i k
Applications to Arithmetic
• But doesn’t extend to division (overflow problems)
i i i i
i i
i i
Compute u op v by computing for i=0,...,k-1
w = u op v mod p
where u = u mod p
v = v mod p
op {+, -, }
Super Moduli Computation
• Input p0, p1, …, pk-1, where pi < 2b
• Output Super modular Tree:
Super Moduli Computation (cont’d)
• Time Cost
log
0
22
( ( ) log )
ki
ii
kM b
O M k b k
Algorithm Residue Computation
• Input
• Output x0, x1, …, xk-1, whenxi =x mod Pi i=0, and k-1
• Recursive algorithm[1] compute quotient and remainders:
1
0
, 0k
ii
x x P P
( 1) / 2
1 1 1 10
1
2 2 2 2( 1)
12
,
,
k
ii
k
ik
i
x q v r v P
x q v r v P
Algorithm Residue Computation (cont’d)
[2] recursively compute
[3] output for i = 0, 1, …, k-1
1
2
( 1)(2.1) mod for 0,...,
2( 1)
(2.2) mod for 1,..., 12
i
i
kr P i
kr P i k
1
2
( 1)mod for
2mod( 1)
mod for 2
i
i
i
kr P i
x Pk
r P i
Time Cost for Residue Computation
• Let D(n) = time cost for integer division = O(M(n))
• Total Time for input size n = k· b
( ) 2 2 ( ) ( )2
2 ( ( ))2
( ( ) log )
nT n T kD b O n
nT O M n
O M n n
Proof of Algorithm for Residue Computation
• Idea of Proof of algorithmUses fact: if x = q v + r and v mod Pi = 0,
then X mod Pi = r mod Pi
Advanced Topic: Residue Computation on Polynomials
• Input moduli P0(x), P1(x), …, Pk-1(x) and each degree d and relatively prime
• Algorithm uses similar Super modular Tree tre , but using polynomials rather than integers
Advanced Topic: Residue Computation on Polynomials (cont’d)• Output for i=0, …, k-1
Qi(x) = Q(x) mod Pi(x)Q(x) has degree kd
• Theorem The Residue Computation can be done in time O(M(n) log n) where n = k· d
• Proof Idea use same algorithm as in integer case
Advanced Topic: Multipoint Evaluation of Polynomials by Residue Computation
• Input polynomial f(x) degree n-1 and points x0, x1, …, xn-1
[1] for i=0, …, n-1 let Pi(x) = (x-xi)[2] By Residue Algorithm Computer for i=0, …, n-1
f(xi) = f(x) mod Pi(x)
[3] output f(x0), …, f(xn-1)
• Time Cost O(M(n) log n), = O(n(log n)2)
Polynomial Interpolation
• Input evaluation points x0, …, xn-1
values y0, …, yn-1
• Output P(x) where yk = P(xk) for k=0, …, n-1
Polynomial Interpolation (cont’d)
• Interpolation formula:
• Where
1
0
( ) ( )n
k k ik i k
P x y a x x
1
( )kk i
i k
ax x
Proof of Polynomial Interpolation
• Proof uses identities:
( ) mod( ) 1
( ) mod( ) 0 for j k
k i ki k
k i ji k
a x x x x
a x x x x
Using Chinese Remaindering for Integer Interpolation
• Input relatively prime P0, P1, …, Pn-1 and yi {0,…Pi-1} for i=0,…,n-1
• Problem compute y < Pi s.t. yi = y mod Pi i=0, …, n-1
• Generalized Interpolation Formula:
• Where
• proof
,
1,
and
( ) mod
k i ki k
i k i k
a s
s P P
1mod
0k i ji k
j ka P P
j k
Using Chinese Remaindering for Integer Interpolation
1
0
n
k k ik i k
y y a P
Advanced Topic: Preconditioned Interpolation
Preconditioned Case assumes coefficients {ak | k=0, …, n-1} precomputed
• Use Divide & Conquer
1
0
n
k k ik i k
y y a P
( 1) / 2 1 11
( 1)0 10 02
n n nn
k k i k k ink i k i kki i
y a P y a P
Preconditioned Interpolation (cont’d)
( 1) / 2 ( 1)1
( 1)0 1 ( 1)2 12
( 1) / 2( 1) / 21
( 1) 01 02
n nn
i k k ini i kk n
i
nnn
i k k in k i ki i
P y a P
P y a P
Time Cost for Preconditioned Interpolation
• Assuming {a0, …, an-1} precomputed
( ) 2 22 2
( ( ) log )
n nT n T M
O M n n
Precomputation of {a0, …, an-1}
1) Compute
2) Compute bk wherebkPk = P mod (Pk)2
by Residue Computation O(M(n) log n)
3) Compute ak = (bk)-1 mod Pk by Extended GCD algorithm
1
0
n
ii
P P
Proof of Precomputation of {a0, …, an-1}
• proof
2k k k
2k k k
i k k
k i k
since b P = P mod (P )
then P = d (P ) + b P
so P d P + b
so b P mod P
i k
i k
Precomputation of {a0, …, an-1} for Polynomial Interpolations• Here Pi = (x-xi) for i=0, …, n-1
reduces to multipoint evaluation of derivative of Q(x) O(M(n) log n) time!
n-1
k jj=0k
kk
k
k
Q(x)b where Q(x)= (x-x )
(x - x )
Q(x) - Q(x ) = since Q(x ) = 0
(x - x )
d Q(x) =
dx x = x
Conclusion
• Polynomial and Integer Computations use similar divide and conquer techniques to solve:
1. Multiplication2. Division3. Interpolation and evaluation
• Open Problem Reduce from time O(M(n)log n) to O(M(n))
Newton Iteration and Polynomial Computation
Prepared by
John Reif, Ph.D.
Analysis of Algorithms