Number Representation

Part 2Floating Point Representations

Little-Endian vs. Big-Endian Representations

Galois Field Representations

ECE 645: Lecture 2

Required Reading

Chapter 17, Floating-Point Representations

Behrooz Parhami, Computer Arithmetic: Algorithms and Hardware Design

J-P. Deschamps, G. Bioul, G. Sutter, Synthesis of Arithmetic Circuits: FPGA, ASIC and Embedded Systems,

Chapter 3.3, Real Numbers

Recommended Reading

Recommended Reading(to be covered at the next lecture)

Chapter 5, Basic Addition and Counting

Behrooz Parhami, Computer Arithmetic: Algorithms and Hardware Design

J-P. Deschamps, G. Bioul, G. Sutter, Synthesis of Arithmetic Circuits: FPGA, ASIC and Embedded Systems,

Chapter 4.1.1 Basic AlgorithmChapter 11.1 Basic AdderChapter 11.2 Carry-Chain Adder

Floating Point Representations

The ANSI/IEEE standard floating-point number representation formats

Short (32-bit) format

Long (64-bit) format

Sign Exponent Significand

8 bits, bias = 127, –126 to 127

11 bits, bias = 1023, –1022 to 1023

52 bits for fractional part (plus hidden 1 in integer part)

23 bits for fractional part (plus hidden 1 in integer part)

Originally IEEE 754-1985.Superseded by IEEE 754-2008 Standard.

Table 17.1 Some features of the ANSI/IEEE standard floatingpoint number representation formats

00 01 7F FE FF7E 800 1 127 254 255126 128

–126 0 +127–1 +1

Decimal codeHex code

Exponent value

f = 0: Representation of 0f 0: Representation of denormals, 0.f 2–126

f = 0: Representation of f 0: Representation of NaNs

Exponent encoding in 8 bits for the single/short (32-bit) ANSI/IEEE format

1.f 2e

Exponent Encoding

Fig. 17.4 Denormals in the IEEE single-precision format.

The IEEE 754-2008 standard includes five rounding modes:

Round to nearest, ties away from 0 (rtna)

Round to nearest, ties to even (rtne) [default rounding mode]

Round toward zero (inward)

Round toward + (upward)

Round toward – (downward)

Rounding Modes

Round to Nearest Number

Fig. 17.7 Rounding of a signed-magnitude value to the nearest number.

Rounding has a slight upward bias.

Consider rounding (xk–1xk–2 ... x1x0 . x–1x–2)two to an integer (yk–1yk–2 ... y1y0 . )two

The four possible cases, and their representation errors are:

x–1x–2 Round Error 00 down 0 01 down –0.25 10 up 0.5 11 up 0.25

With equal prob., mean = 0.125For certain calculations, the probability of getting a midpoint value can be much higher than 2–l

rtn(x)

–4

–3

–2

–1

x –4 –3 –2 –1 4 3 2 1

4

3

2

1

rtna(x)

Directed Rounding: Motivation

We may need result errors to be in a known direction

Example: in computing upper bounds, larger results are acceptable, but results that are smaller than correct values could invalidate the upper bound

This leads to the definition of directed rounding modesupward-directed rounding (round toward +) and downward-directed rounding (round toward –)(required features of IEEE floating-point standard)

Directed Rounding: Visualization

Fig. 17.12 Upward-directed rounding or rounding toward +.

Fig. 17.6 Truncation or chopping of a 2’s-complement number (same as downward-directed rounding).

up(x)

–4

–3

–2

–1

x –4 –3 –2 –1 4 3 2 1

4

3

2

1

chop(x) = down(x)

–4

–3

–2

–1

x –4 –3 –2 –1 4 3 2 1

4

3

2

1

Requirements for Arithmetic

Results of the 4 basic arithmetic operations (+, , , ) as well as square-rooting must match those obtained if all intermediate computations were infinitely precise

That is, a floating-point arithmetic operation should introduce no more imprecision than the error attributable to the final rounding of a result that has no exact representation (this is the best possible)

Example:(1 + 21) (1 + 223 )

Rounded result 1 + 21 + 222 Error = ½ ulp

Exact result 1 + 21 + 223 + 224

New IEEE 754-2008 StandardBasic Formats

New IEEE 754-2008 StandardBinary Interchange Formats

Little-Endian vs. Big-Endian Representation of

Integers

Little-Endian vs. Big-Endian Representation

A0 B1 C2 D3 E4 F5 67 8916

LSBMSB

MSB = A0

B1

C2D3

E4F5

67LSB = 89

Big-Endian Little-Endian

LSB = 89

0

MAX

67

F5E4

D3C2

B1MSB = A0

address

Little-Endian vs. Big-Endian Camps

Big-Endian Little-Endian

0

MAX

address

MSB

LSB

. . .

LSB

MSB

. . .

Motorola 68xx, 680x0 Intel

IBM

Hewlett-PackardDEC VAX

Internet TCP/IP

Sun SuperSPARC

Bi-Endian

Motorola Power PC

Silicon Graphics MIPS

RS 232

AMD

Origin of the termsLittle-Endian vs. Big-Endian

Jonathan Swift, Gulliver’s Travels

• A law requiring all citizens of Lilliput to break their soft-eggs

at the little ends only

• A civil war breaking between the Little Endians and

the Big-Endians, resulting in the Big Endians taking refuge on

a nearby island, the kingdom of Blefuscu

• Satire over holy wars between Protestant Church of England

and the Catholic Church of France

Little-Endian vs. Big-Endian

Big-Endian Little-Endian

• easier to determine a sign of the number

• easier to compare two numbers

• easier to divide two numbers

• easier to print

• easier addition and multiplication of multiprecision numbers

Advantages and Disadvantages

Pointers (1)

89

67

F5E4

D3C2

B1

A0

Big-Endian Little-Endian

0

MAX

address

int * iptr;

(* iptr) = 8967; (* iptr) = 6789;

iptr+1

Pointers (2)

89

67

F5E4

D3C2

B1

A0

Big-Endian Little-Endian

0

MAX

address

long int * lptr;

(* lptr) = 8967F5E4; (* lptr) = E4F56789;

lptr + 1

Polynomial Representationof the Galois Field

elements

Evariste Galois (1811-1832)

Evariste Galois (1811-1832)

Studied the problem of finding algebraic solutions for the general

equations of the degree 5, e.g.,

f(x) = a5x5+ a4x4+ a3x3+ a2x2+ a1x+ a0 = 0

Answered definitely the question which specific equations of

a given degree have algebraic solutions

On the way, he developed group theory,

one of the most important branches of modern mathematics.

Evariste Galois (1811-1832)

1829 Galois submits his results for the first time to the French Academy of Sciences

Reviewer 1 Augustin-Luis Cauchy forgot or lost the communication

1830 Galois submits the revised version of his manuscript,hoping to enter the competition for the Grand Prizein mathematics

Reviewer 2 Joseph Fourier – died shortly after receiving the manuscript

1831 Third submission to the French Academy of SciencesReviewer 3

Simeon-Denis Poisson – did not understand the manuscript and rejected it.

Evariste Galois (1811-1832)

May 1832 Galois provoked into a duel

The night before the duel he writes a letter to his friend containing the summary of his discoveries.

The letter ends with a plea: “Eventually there will be, I hope, some people who

will find it profitable to decipher this mess.”

May 30, 1832 Galois is grievously wounded in the duel and dies in the hospital the following day.

1843 Galois manuscript rediscovered by Joseph Liouville

1846 Galois manuscript published forthe first time in a mathematical journal

Field

Set F, and two operations typically denoted by (but not necessarily equivalent to)

+ and *

Set F, and definitions of these two operations must fulfill special conditions.

{ set Zp={0, 1, 2, … , p-1}, + (mod p): addition modulo p, * (mod p): multiplication modulo p}

Examples of fieldsInfinite fields

Finite fields

{ R= set of real numbers, + addition of real numbers * multiplication of real numbers}

Quotient and remainder

Given integers a and n, n>0

! q, r Z such that

a = q n + r and 0 r < n

q – quotient

r – remainder (of a divided by n)

q = an = a div n

r = a - q n = a – an

n =

= a mod n

32 mod 5 =

-32 mod 5 =

Integers coungruent modulo n

Two integers a and b are congruent modulo n

(equivalent modulo n)

written a b

iff

a mod n = b mod n

or

a = b + kn, k Z

or

n | a - b

Laws of modular arithmetic

Rules of addition, subtraction and multiplicationmodulo n

a + b mod n = ((a mod n) + (b mod n)) mod n

a - b mod n = ((a mod n) - (b mod n)) mod n

a b mod n = ((a mod n) (b mod n)) mod n

9 · 13 mod 5 =

25 · 25 mod 26 =

Laws of modular arithmetic

Modular addition

Modular multiplication

Regular addition

Regular multiplication

a+b = a+ciff

b=c

a+b a+c (mod n)iff

b c (mod n)

If a b = a c and a 0then b = c

If a b a c (mod n) and gcd (a, n) = 1then b c (mod n)

Modular Multiplication: Example

18 42 (mod 8) 6 3 6 7 (mod 8)

3 7 (mod 8)

x

6 x mod 8

0 1 2 3 4 5 6 7

0 6 4 2 0 6 4 2

x

5 x mod 8

0 1 2 3 4 5 6 7

0 5 2 7 4 1 6 3

Z[x] - polynomials with coefficients in Z,

Sets of polynomials

e.g., f(x) = -4 x3 + 254 x2 + 45 x + 7

Zn[x] - polynomials with coefficients in Zn

e.g., for n=15

f(x) = 3 x3 + 14 x2 + 4 x + 7

Z2[x] - polynomials with coefficients in Z2

e.g., f(x) = 1 x3 + 0 x2 + 1 x + 1 = x3 + x + 1

Division of Polynomials

Finite sets of polynomials

Z2[x]/f(x) - polynomials with coefficients in Z2

of degree less than n=deg f(x)

Zp[x]/f(x) - polynomials with coefficients in Zp

of degree less than n=deg f(x)

e.g., for f(x) = x3 + x + 1

g7(x) = x2 + x + 1g6(x) = x2 + x g5(x) = x2 + 1g4(x) = x2

g3(x) = x + 1g2(x) = x g1(x) = 1g0(x) = 0

e.g., for f(x) = x3 + x + 1, and p=3

g0(x) = 0….gM-1(x) = 2x2 + 2x + 2

Total: 3n polynomials

Finite Fields = Galois Fields

GF(p) GF(2m)

Polynomial basisrepresentation

Normal basisrepresentation

Fast in hardware

Arithmetic operations

presentin many libraries

Fast squaring

GF(pm)p – primepm – number of elements in the field

Most significantspecial cases

Elements of the Galois Field GF(2m)

Binary representation (used for storing and processing in computer systems):

Polynomial representation(used for the definition of basic arithmetic operations):

A = (am-1, am-2, …, a2, a1, a0) ai {0, 1}

A(x) = aixi = am-1xm-1 + am-2xm-2 + …+ a2x2 + a1x+a0

multiplication+ addition modulo 2 (XOR)

i=0

m-1

Addition and Multiplicationin the Galois Field GF(2m)

Inputs

A = (am-1, am-2, …, a2, a1, a0)B = (bm-1, bm-2, …, b2, b1, b0)

ai , bi {0, 1}

Output

C = (cm-1, cm-2, …, c2, c1, c0) ci {0, 1}

Addition

A A(x)B B(x)C C(x) = A(x) + B(x) = = (am-1+bm-1)xm-1 + (am-2+bm-2)xm-2+ …+ + (a2+b2)x2 + (a1+b1)x + (a0+b0) = = cm-1xm-1 + cm-2xm-2 + …+ c2x2 + c1x+c0

Addition in the Galois Field GF(2m)

multiplication+ addition modulo 2 (XOR)

ci = ai + bi = ai XOR bi

C = A XOR B

Multiplication

A A(x)B B(x)C C(x) = A(x) B(x) mod P(X) = cm-1xm-1 + cm-2xm-2 + …+ c2x2 + c1x+c0

Multiplication in the Galois Field GF(2m)

P(x) - irreducible polynomial of the degree m

P(x) = pmxm + pm-1xm-1 + …+ p2x2 + p1x+p0

Irreducible polynomial for AES

P(x) = m(x) = x8 + x4 + x3 + x + 1

m=8 Galois Field GF(28)

AES MixColumns Operation

a0,0 a0,1 a0,2 a0,3

a1,0 a1,1 a1,2 a1,3

a2,0 a2,1 a2,2 a2,3

a3,0 a3,1 a3,2 a3,3

b0,0 b0,1 a0,2 b0,3

b1,0 b1,1 a1,2 b1,3

b2,0 b2,1 a2,2 b2,3

b3,0 b3,1 a3,2 b3,3

a1,j

a0,j

a2,j

a3,j

b1,j

b0,j

b2,j

b3,j

2 3 1 1 1 2 3 11 1 2 33 1 1 2

All operations in the Galois Field GF(28)

AES InvMixColumns Operations

a0,0 a0,1 a0,2 a0,3

a1,0 a1,1 a1,2 a1,3

a2,0 a2,1 a2,2 a2,3

a3,0 a3,1 a3,2 a3,3

b0,0 b0,1 a0,2 b0,3

b1,0 b1,1 a1,2 b1,3

b2,0 b2,1 a2,2 b2,3

b3,0 b3,1 a3,2 b3,3

a1,j

a0,j

a2,j

a3,j

b1,j

b0,j

b2,j

b3,j

E B D 9 9 E B DD 9 E BB D 9 E

All operations in the Galois Field GF(28)

Multiplication by a constantin the Galois Field GF(28)

Hardware8 8

MUL GF(28)

X

Y

C = const

x0 x3 x7

y0

. . .

x0 x3 x7

y7

x4

8

Hardware implementation - MixColumns

architecture mul_03 of mul_03 isbegin

output(7) <= input(7) xor input(6);output(6) <= input(6) xor input(5);output(5) <= input(5) xor input(4);output(4) <= input(4) xor input(3) xor input(7);output(3) <= input(3) xor input(2) xor input(7);output(2) <= input(2) xor input(1);output(1) <= input(1) xor input(0) xor input(7);output(0) <= input(0) xor input(7);

end mul_03;

b0 <= a0_02 xor a1_03 xor a2 xor a3;

Hardware implementation - InvMixColumns

architecture mul_0E of mul_0E isbegin output(7) <= input(7) xor input(6) xor input(5) xor input(4); output(6) <= input(6) xor input(5) xor input(4) xor input(3) xor input(7); output(5) <= input(5) xor input(4) xor input(3) xor input(2) xor input(6); output(4) <= input(4) xor input(3) xor input(2) xor input(1) xor input(5); output(3) <= input(3) xor input(2) xor input(1) xor input(0) xor input(6) xor input(5); output(2) <= input(2) xor input(1) xor input(0) xor input(6); output(1) <= input(1) xor input(0) xor input(5); output(0) <= input(0) xor input(7) xor input(6) xor input(5);end mul_0E;

b0 <= a0_0E xor a1_0B xor a2_0D xor a3_09;

Conclusion: In hardware, InvMixColumns slower than MixColumns