choose your representation! great theoretical ideas in computer science steven rudichcs 15-251...

Choose Your Representation!

Great Theoretical Ideas In Computer Science

Steven Rudich

CS 15-251 Spring 2005

Lecture 14Lecture 14 Feb 24, 2005Feb 24, 2005 Carnegie Mellon University

We have seen that the same idea can be represented in

many different ways.

Natural Numbers

UnaryBinaryBase b

Mathematical Prehistory:30,000 BC

Paleolithic peoples in Europe record Paleolithic peoples in Europe record unaryunary numbers on bones. numbers on bones.

1 represented by 1 mark1 represented by 1 mark

2 represented by 2 marks2 represented by 2 marks



……

Prehistoric Unary

11

22

33

44

PowerPoint Unary

11

22

33

44

1 2 . . . . . . . . n

= number of white dots.1 + 2 + 3 + . . . + n-1 + n = S

n + n-1 + n-2 + . . . + 2 + 1 = S

(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S

n (n+1) = 2S

1 + 2 + 3 + . . . + n-1 + n = S

n + n-1 + n-2 + . . . + 2 + 1 = S

(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S

n (n+1) = 2S

1 2 . . . . . . . . n

= number of white dots

= number of yellow dots

n . . . . . . . 2 1

1 + 2 + 3 + . . . + n-1 + n = S

n + n-1 + n-2 + . . . + 2 + 1 = S

(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S

n (n+1) = 2S

n+1 n+1 n+1 n+1 n+1



n

n

n

n

n

n

There are n(n+1) dots in the grid

1 + 2 + 3 + . . . + n-1 + n = S

n + n-1 + n-2 + . . . + 2 + 1 = S

(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S

n (n+1) = 2S

n+1 n+1 n+1 n+1 n+1



n

n

n

n

n

n

2

1)(n n S

Reals

Standard decimal or binary notation.

Continued fractions

A Periodic CF

3 13 13

12 31

31

31

31

31

31

31

33 ....

Choice TreesBlock Walking ModelBinomial Expansion

Vectors

Manhattan

k’th Avenue01

23

4

Level n0

1

34

2

………

…………………… …………………………

…………………………………

………….

Multiple Representations

means that we have our CHOICE of which

one we will use.

How to play the 9 stone game?

9 stones, numbered 1-9 9 stones, numbered 1-9

Two players alternate moves. Two players alternate moves.

Each move a player gets to take a new Each move a player gets to take a new stonestone

Any subset ofAny subset of 3 stones adding to 153 stones adding to 15, wins., wins.

1 23

45

67 8

9

For enlightenment, let’s look to ancient China in

the days of Emperor Yu.

A tortoise emerged from the river Lo…

Magic Square: Brought to humanity on the back of a tortoise from the river Lo in the days of Emperor Yu

49 2

35

7

8 16

Magic Square: Any 3 in a vertical, horizontal, or diagonal line add up to

15.

44 99 22

33 55 77

88 11 66

Conversely, any 3 that add to 15 must be on a

line.

44 99 22

33 55 77

88 11 66

44 99 22

33 55 77

88 11 66

TIC-TAC-TOE on a Magic SquareRepresents The Nine Stone Game

Alternate taking squares 1-9. Get 3 in a row to win.

BIG IDEA!

Don’t stick with the representation

in which you encounter problems!

Always seek the more useful one!

This IDEA takes practice,

practice, practice

to understand and use.

Natural Numbers

UnaryBinaryBase b

Mathematical Prehistory:30,000 BC

Paleolithic peoples in Europe record Paleolithic peoples in Europe record unaryunary numbers on bones. numbers on bones.

1 represented by 1 mark1 represented by 1 mark




……

Prehistoric Unary

11

22

33

44

PowerPoint Unary

11

22

33

44

1 2 . . . . . . . . n

= number of white dots.1 + 2 + 3 + . . . + n-1 + n = S

n + n-1 + n-2 + . . . + 2 + 1 = S

(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S

n (n+1) = 2S

1 + 2 + 3 + . . . + n-1 + n = S

n + n-1 + n-2 + . . . + 2 + 1 = S

(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S

n (n+1) = 2S

1 2 . . . . . . . . n



n . . . . . . . 2 1

1 + 2 + 3 + . . . + n-1 + n = S

n + n-1 + n-2 + . . . + 2 + 1 = S

(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S

n (n+1) = 2S

n+1 n+1 n+1 n+1 n+1



n

n

n

n

n

n

There are n(n+1) dots in the grid

1 + 2 + 3 + . . . + n-1 + n = S

n + n-1 + n-2 + . . . + 2 + 1 = S

(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S

n (n+1) = 2S

n+1 n+1 n+1 n+1 n+1



n

n

n

n

n

n

2

1)(n n S

A case study.

Anagram Programming Task.

You are given a 70,000 word dictionary. Write an anagram utility that given a word as input returns all anagrams of that word appearing in the dictionary.

Examples

Input: CATInput: CAT

Output: ACT, CAT, TACOutput: ACT, CAT, TAC

Input: SUBESSENTIALInput: SUBESSENTIAL

Output: SUITABLENESSOutput: SUITABLENESS

Impatient Hacker (Novice Level Solution)

Loop through all possible ways of Loop through all possible ways of rearranging the input wordrearranging the input word

Use binary search to look up Use binary search to look up resulting word in dictionary.resulting word in dictionary.

If found, output itIf found, output it

Performance AnalysisCounting without executing

On the word “microphotographic”, we loop On the word “microphotographic”, we loop 17! 17! 3 * 10 3 * 10114 times.4 times.

Even at 1 microsecond per iteration, this will Even at 1 microsecond per iteration, this will take 3 *10take 3 *1088 seconds. seconds.

Almost a decade! Almost a decade!

(There are about (There are about seconds in a seconds in a nanocentury.)nanocentury.)

“Expert” Hacker

Module ANAGRAM(X,Y) returns Module ANAGRAM(X,Y) returns TRUETRUE exactly when X and Y are anagrams. exactly when X and Y are anagrams.

(Works by sorting the letters in X and Y)(Works by sorting the letters in X and Y)

Input XInput X

Loop through all dictionary words YLoop through all dictionary words Y

If ANAGRAM(X,Y) output YIf ANAGRAM(X,Y) output Y

The hacker is satisfied and reflects no futher

Comparing an input word with each Comparing an input word with each of 70,000 dictionary entries takes of 70,000 dictionary entries takes

about 15 seconds. about 15 seconds.

The master keeps trying to refine the solution.

The master’s program runs in less The master’s program runs in less than 1/1000 seconds.than 1/1000 seconds.

Master Solution

Don’t keep the dictionary in sorted order!

Rearranging the dictionary into anagram classes will make the

original problem simple.

Suppose the dictionary was the list below.

ASPASPDOGDOGLURELUREGODGODNICENICERULERULESPASPA

After each word, write its “signature” (sort its letters)

ASPASP APSAPSDOGDOG DGODGOLURELURE ELRUELRUGODGOD DGODGONICENICE CEINCEINRULERULE ELRUELRUSPASPA APSAPS

Sort by the signatures

ASPASP APSAPSSPASPA APSAPSNICENICE CEINCEINDOGDOG DGODGOGODGOD DGODGOLURELURE ELRUELRURULERULE ELRUELRU

Master Program

Input word WInput word W

X := signature of WX := signature of W

Use binary search to find the Use binary search to find the anagram class of W and output it. anagram class of W and output it.

About microseconds

.0004 seconds

log ( , )2 70 000 25

Of course, it takes about 30 seconds to create the dictionary, but it is perfectly fair to think of this as programming time. The building of the dictionary is a one-time cost that is part of writing the program.

Neat! I wish I had thought of that.

Name Your Tools

Whenever you see something you wish you had thought of, try and formulate the minimal and most general lesson that will insure that you will not miss the same thing the next time. Name the lesson to make it easy to remember.

NAME: Preprocessing

It is sometimes possible to pay a reasonable, one-time preprocessing cost to reorganize your data in such a way as to use it more efficiently later. The extra time required to preprocess can be thought of as additional programming effort.

Of course, preprocessing is

just a special case of seeking the appropriate representation.

Don’t let the representation choose you,

CHOOSE THE REPRESENTATION!

Vector Programs

Let’s define a (parallel) Let’s define a (parallel) programming language called programming language called VECTOR that operates on possibly VECTOR that operates on possibly infinite vectors of numbers. Each infinite vectors of numbers. Each variable Vvariable V!! can be thought of as: can be thought of as:

< * , * , * , * , *, *, . . . . . . . . . >< * , * , * , * , *, *, . . . . . . . . . >

0 1 2 3 4 5 . . . . . . . . . 0 1 2 3 4 5 . . . . . . . . .

Vector Programs

Let k stand for a scalar constantLet k stand for a scalar constant

<k> will stand for the vector <k,0,0,0,…><k> will stand for the vector <k,0,0,0,…>

<0> = <0,0,0,0,….><0> = <0,0,0,0,….>

<1> = <1,0,0,0,…><1> = <1,0,0,0,…>

VV! ! ++ TT!! means to add the vectors position-wise. means to add the vectors position-wise.

<4,2,3,…> + <5,1,1,….> = <9,3,4,…><4,2,3,…> + <5,1,1,….> = <9,3,4,…>

Vector Programs

RIGHT(VRIGHT(V!!)) means to shift every number in V means to shift every number in V!! one position to the one position to the rightright and to place a 0 in and to place a 0 in position 0.position 0.

RIGHT( <1,2,3, …> ) = <0,1,2,3,. …>RIGHT( <1,2,3, …> ) = <0,1,2,3,. …>

Vector Programs

Example:Example:

VV!! := <6>; := <6>;

VV!! := RIGHT(V := RIGHT(V!!)) + <42>;+ <42>;

VV! ! := RIGHT(V:= RIGHT(V!!)) + <2>;+ <2>;

VV!! := RIGHT(V := RIGHT(V!!)) + <13>;+ <13>;

VV!! = < 13, 2, 42, 6, 0, 0, 0, . . . > = < 13, 2, 42, 6, 0, 0, 0, . . . >

StareStare

VV!! = = <6,0,0,0,..><6,0,0,0,..>

VV!! = = <42,6,0,0,..><42,6,0,0,..>

VV!! = = <2,42,6,0,..> V<2,42,6,0,..> V!!

= <13,2,42,6,.>= <13,2,42,6,.>

Vector Programs

Example:Example:

VV!! := <1>; := <1>;

Loop n times:Loop n times:

VV!! := V := V!! + + RIGHT(VRIGHT(V!!););

VV!! = n = nthth row of Pascal’s triangle. row of Pascal’s triangle.

StareStare

VV!! = <1,0,0,0,..> = <1,0,0,0,..>

VV!! = <1,1,0,0,..> = <1,1,0,0,..>

VV!! = <1,2,1,0,..> = <1,2,1,0,..> VV! ! = <1,3,3,1,.>= <1,3,3,1,.>

X1 X2 + + X3

Vector programs can be

implemented by polynomials!

Programs -----> Polynomials

The vector VThe vector V!! = < a = < a00, a, a11, a, a22, . . . > will be , . . . > will be represented by the polynomial:represented by the polynomial:

Formal Power Series

The vector VThe vector V!! = < a = < a00, a, a11, a, a22, . . . > will be , . . . > will be represented by the represented by the formal power seriesformal power series::

V! = < a0, a1, a2, . . . >

<0> is represented by 0<k> is represented by k

RIGHT(V! ) is represented by (PV X)

V! + T! is represented by (PV + PT)

Vector Programs

Example:Example:

VV!! := <1>; := <1>;




PV := 1;

PV := PV + PV X;

Vector Programs

Example:Example:

VV!! := <1>; := <1>;




PV := 1;

PV := PV (1+ X);

Vector Programs

Example:Example:

VV!! := <1>; := <1>;




PV = (1+ X)n

What is the coefficient of Xk in the expansion of:

( 1 + X + X2 + X3 + X4

+ . . . . )n ?

Each path in the choice tree for the cross terms has n choices of exponent e1, e2, . . . , en ¸ 0. Each exponent can be any natural number.

Coefficient of Xk is the number of non-negative solutions to:

e1 + e2 + . . . + en = k


( 1 + X + X2 + X3 + X4

+ . . . . )n ?

n

n -1

1k

( 1 + X + X2 + X3 + X4

+ . . . . )n =

k

k 0

nX

n -1

11

1n

k

X


(a0 + a1X + a2X2 + a3X3 + …) ( 1 + X + X2 + X3

+ . . . )

= (a0 + a1X + a2X2 + a3X3 + …) / (1 – X) ?

a0 + a1 + a2 + .. + ak

(a0 + a1X + a2X2 + a3X3 + …) / (1

– X)

= k

k 0 i 0

a X

i k

i

=

PREFIXSUM(a0 + a1X + a2X2 + a3X3 + …)

Let’s add an instruction called PREFIXSUM to our

VECTOR language.

W! := PREFIXSUM(V!)

means that the ith position of W contains the sum of all the numbers in V from

positions 0 to i.

What does this program output?

VV!! := 1 := 1!! ; ;

Loop k times: Loop k times: VV!! := PREFIXSUM(V := PREFIXSUM(V!!) ; ) ;

k’th Avenue01

23

4

Can you see how PREFIXSUM can be represented by a

familiar polynomial expression?

W! := PREFIXSUM(V!)

is represented by

PW = PV / (1-X)

= PV (1+X+X2+X3+

….. )

Al-Karaji Program

Zero_Ave := PREFIXSUM(<1>);Zero_Ave := PREFIXSUM(<1>);

First_Ave := PREFIXSUM(Zero_Ave);First_Ave := PREFIXSUM(Zero_Ave);

Second_Ave :=PREFIXSUM(First_Ave);Second_Ave :=PREFIXSUM(First_Ave);

Output:= Output:= First_Ave + 2*RIGHT(Second_Ave)First_Ave + 2*RIGHT(Second_Ave)

OUTPUTOUTPUT!! = = <1, 4, 9, 25, 36, 49, …. ><1, 4, 9, 25, 36, 49, …. >

Al-Karaji Program

Zero_Ave = 1/(1-X);Zero_Ave = 1/(1-X);

First_Ave = 1/(1-X)First_Ave = 1/(1-X)22;;

Second_Ave = 1/(1-X)Second_Ave = 1/(1-X)33;;

Output = Output = 1/(1-X)1/(1-X)22 + 2X/(1-X) + 2X/(1-X)33

(1-X)/(1-X)(1-X)/(1-X)3 3 + 2X/(1-X)+ 2X/(1-X)3 3

= (1+X)/(1-X)= (1+X)/(1-X)33

(1+X)/(1-X)(1+X)/(1-X)33

Zero_Ave := PREFIXSUM(<1>);Zero_Ave := PREFIXSUM(<1>);

First_Ave := PREFIXSUM(Zero_Ave);First_Ave := PREFIXSUM(Zero_Ave);

Second_Ave :=PREFIXSUM(First_Ave);Second_Ave :=PREFIXSUM(First_Ave);

Output:= Output:= RIGHT(Second_Ave) + Second_AveRIGHT(Second_Ave) + Second_Ave

Second_Ave = <1, 3, 6, 10, 15,. Second_Ave = <1, 3, 6, 10, 15,.

RIGHT(Second_Ave) = <0, 1, 3, 6, 10,.RIGHT(Second_Ave) = <0, 1, 3, 6, 10,.Output = <1, 4, 9, 16, 25

(1+X)/(1-X)(1+X)/(1-X)3 3 outputs <1, 4, 9, ..>outputs <1, 4, 9, ..>

X(1+X)/(1-X)X(1+X)/(1-X)3 3 outputs <0, 1, 4, 9, ..>outputs <0, 1, 4, 9, ..>

The kThe kthth entry is k entry is k22

X(1+X)/(1-X)X(1+X)/(1-X)3 3 = = k k22XXkk

What does X(1+X)/(1-X)What does X(1+X)/(1-X)44 do?do?

X(1+X)/(1-X)X(1+X)/(1-X)44 expands to : expands to :

SSkk X Xkk

where Swhere Skk is the sum of the is the sum of the first k squaresfirst k squares

Aha! Thus, if there is an Aha! Thus, if there is an alternative interpretation of alternative interpretation of

the kthe kthth coefficient of coefficient of X(1+X)/(1-X)X(1+X)/(1-X)44

we would have a new way we would have a new way to get a formula for the sum to get a formula for the sum

of the first k squares.of the first k squares.

Using pirates and gold we found that:

k

k 0

nX

n -1

11

1n

k

X

k

k 0

X3

4

31

1

k

X

THUS:

Coefficient of Xk in PV = (X2+X)(1-X)-4

is the sum of the first k squares:

k

k 0

X3

4

31

1

k

X

Vector programs -> Polynomials-> Closed form expression

Expressions of the formExpressions of the form

Finite Polynomial / Finite Finite Polynomial / Finite PolynomialPolynomial

are called are called Rational Polynomial Rational Polynomial ExpressionsExpressions. .

Clearly, these expressions have Clearly, these expressions have some deeper interpretation as a some deeper interpretation as a

programming language.programming language.

References

The Heritage of ThalesThe Heritage of Thales, by W. S. Anglin , by W. S. Anglin and F. Lambekand F. Lambek

The Book Of NumbersThe Book Of Numbers, by J. Conway and , by J. Conway and R. GuyR. Guy

Programming PearlsProgramming Pearls, by J. Bentley, by J. Bentley

History of Mathematics, Histories of History of Mathematics, Histories of Problems,Problems, by The Inter-IREM Commission by The Inter-IREM Commission

choose your representation! great theoretical ideas in computer science steven rudichcs 15-251...

Documents