math1003 1.17 - truncation, rounding, overflow, & conversion error

71
MATH1003 10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001 1.17 Truncation, Rounding, Overflow, and Conversion Error

Upload: gcmath1003

Post on 23-Jun-2015

648 views

Category:

Education


3 download

TRANSCRIPT

Page 1: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

1.17Truncation, Rounding, Overflow, and

Conversion Error

Page 2: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

Goal

To be able to explain and demonstrate the concepts of truncation, rounding, overflow, and conversion error.

Page 3: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

The computer is imperfect

Page 4: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

The computer is imperfect

No matter how large a computer is, it still has a limited amount of storage.

Page 5: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

The computer is imperfect

No matter how large a computer is, it still has a limited amount of storage.

Consider the result of dividing 2 by 3.

Page 6: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

The computer is imperfect

No matter how large a computer is, it still has a limited amount of storage.

Consider the result of dividing 2 by 3.

0.666666 is a repeating number.

Page 7: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

The computer is imperfect

No matter how large a computer is, it still has a limited amount of storage.

Consider the result of dividing 2 by 3.

0.666666 is a repeating number.

Regardless of how many bits we use to store this number, it will get “cut off” at some point. No computer

can accurately store this number.

Page 8: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Truncation

Page 9: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Truncation

To truncate a number means to simply ignore the extra digits that the computer cannot store.

Page 10: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Truncation

To truncate a number means to simply ignore the extra digits that the computer cannot store.

Truncate the following to 3 significant digits

0.2349

Page 11: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Truncation

To truncate a number means to simply ignore the extra digits that the computer cannot store.

Truncate the following to 3 significant digits

0.234

Page 12: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Truncation

To truncate a number means to simply ignore the extra digits that the computer cannot store.

Truncate the following to 5 significant digits

0.666666666666

Page 13: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Truncation

To truncate a number means to simply ignore the extra digits that the computer cannot store.

Truncate the following to 5 significant digits

0.66666

Page 14: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Truncation

To truncate a number means to simply ignore the extra digits that the computer cannot store.

Truncate the following to 8 significant binary digits

0.1010101111000101

Page 15: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Truncation

To truncate a number means to simply ignore the extra digits that the computer cannot store.

Truncate the following to 8 significant binary digits

0.10101011

Page 16: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

Real Numbers0 0 1 1 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0

Represent 0.02510 in IEEE standard1. 0.02510 = 0.00000112

2. normalized as 1.1001 x 2-6

3. set the sign bit4. store -6 in the exponent section as (-6 + 127 = 121) 011110012

5. store the normalized binary form

1.1001 x 2-60.0000011001100111.100110011001

Page 17: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

Overflow Error

Page 18: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

Overflow Error

Let’s assume that we are using 8 bits and 2’s complement to store integers (1 sign bit and 7 bits for the number).

Page 19: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

Overflow Error

Let’s assume that we are using 8 bits and 2’s complement to store integers (1 sign bit and 7 bits for the number).

Calculate 12610 + 1310

Page 20: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

Overflow Error

Let’s assume that we are using 8 bits and 2’s complement to store integers (1 sign bit and 7 bits for the number).

Calculate 12610 + 1310

12610 + 1310

Page 21: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

Overflow Error

Let’s assume that we are using 8 bits and 2’s complement to store integers (1 sign bit and 7 bits for the number).

Calculate 12610 + 1310

011111102

+000011012

12610 + 1310

Page 22: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

Overflow Error

Let’s assume that we are using 8 bits and 2’s complement to store integers (1 sign bit and 7 bits for the number).

Calculate 12610 + 1310

011111102

+000011012

100010112

12610 + 1310

Page 23: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

Overflow Error

Let’s assume that we are using 8 bits and 2’s complement to store integers (1 sign bit and 7 bits for the number).

Calculate 12610 + 1310

011111102

+000011012

100010112

12610 + 1310

sign

bit

Page 24: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

Overflow Error

Let’s assume that we are using 8 bits and 2’s complement to store integers (1 sign bit and 7 bits for the number).

Calculate 12610 + 1310

011111102

+000011012

100010112

12610 + 1310

sign

bit

this is -11710 in 2’s complement, but

the answer should be 13910.

What happened?

Page 25: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

Overflow Error

Let’s assume that we are using 8 bits and 2’s complement to store integers (1 sign bit and 7 bits for the number).

Calculate 12610 + 1310

011111102

+000011012

100010112

12610 + 1310

sign

bit

7 bits can only store up to 127.

Remember that the first bit is used as a sign

bit.

Page 26: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

Overflow Error

Let’s assume that we are using 8 bits and 2’s complement to store integers (1 sign bit and 7 bits for the number).

Calculate 12610 + 1310

011111102

+000011012

100010112

12610 + 1310

sign

bit

7 bits can only store up to 127.

We have an overflow problem.

Page 27: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

Page 28: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Page 29: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 2 significant digits

Page 30: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 2 significant digits

2.53

Page 31: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 2 significant digits

2.53

Page 32: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 2 significant digits

2.53

Since 3 is less than 5, this will be like

truncation

Page 33: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 2 significant digits

2.5

Since 3 is less than 5, this will be like

truncation

Page 34: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 2 significant digits

2.5

Page 35: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 3 significant digits

17.948

Page 36: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 3 significant digits

17.948

Page 37: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 3 significant digits

17.948

Since 4 is less than 5, this will be like

truncation

Page 38: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 3 significant digits

17.9

Page 39: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 2 significant digits

-0.002463

Page 40: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 2 significant digits

-0.002463

Since we are concerned only

about the significant digits, we will only

consider these digits

Page 41: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 2 significant digits

-0.002463

Page 42: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 2 significant digits

-0.002463

Since 6 is greater than or equal to 5, we “round up” the 4 to its

left to 5

Page 43: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 2 significant digits

-0.0025

Page 44: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 5 significant digits

0.173

Page 45: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 5 significant digits

0.173

Note that this is a repeating number.

We should expand to at least 6 significant digits before rounding to 5 significant

digits

Page 46: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 5 significant digits

0.173737

Note that this is a repeating number.

We should expand to at least 6 significant digits before rounding to 5 significant

digits

Page 47: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 5 significant digits

0.173737

Page 48: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 5 significant digits

0.173737

Page 49: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 5 significant digits

0.173737

Since 7 is greater than or equal to 5, we “round up” the 3 to its

left to 4

Page 50: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

2/3

Rounding

An alternative to truncation is rounding, where the last digit is “adjusted” to give a more accurate representation

of the number.

Round the following to 5 significant digits

0.17374

Page 51: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Let’s represent 0.110 with 4 bytes in IEEE standard form.

Page 52: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Let’s represent 0.110 with 4 bytes in IEEE standard form.1. 0.110 = 0.000112

Page 53: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Let’s represent 0.110 with 4 bytes in IEEE standard form.1. 0.110 = 0.000112

2. normalize 0.000112 = 1.10011 x 2-4

Page 54: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Let’s represent 0.110 with 4 bytes in IEEE standard form.1. 0.110 = 0.000112

2. normalize 0.000112 = 1.10011 x 2-4

1.10011 x 2-4

Page 55: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Let’s represent 0.110 with 4 bytes in IEEE standard form.1. 0.110 = 0.000112

2. normalize 0.000112 = 1.10011 x 2-4

3. set the sign bit

1.10011 x 2-4

Page 56: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Let’s represent 0.110 with 4 bytes in IEEE standard form.0

1. 0.110 = 0.000112

2. normalize 0.000112 = 1.10011 x 2-4

3. set the sign bit

1.10011 x 2-4

Page 57: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Let’s represent 0.110 with 4 bytes in IEEE standard form.0

1. 0.110 = 0.000112

2. normalize 0.000112 = 1.10011 x 2-4

3. set the sign bit4. store -4 in the exponent section

1.10011 x 2-4

Page 58: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Let’s represent 0.110 with 4 bytes in IEEE standard form.0

1. 0.110 = 0.000112

2. normalize 0.000112 = 1.10011 x 2-4

3. set the sign bit4. store -4 in the exponent section

1.10011 x 2-4

-4 + 127 = 123123 = 011110112

Page 59: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Let’s represent 0.110 with 4 bytes in IEEE standard form.0 0 1 1 1 1 0 1 1

1. 0.110 = 0.000112

2. normalize 0.000112 = 1.10011 x 2-4

3. set the sign bit4. store -4 in the exponent section

1.10011 x 2-4

-4 + 127 = 123123 = 011110112

Page 60: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Let’s represent 0.110 with 4 bytes in IEEE standard form.0 0 1 1 1 1 0 1 1

1. 0.110 = 0.000112

2. normalize 0.000112 = 1.10011 x 2-4

3. set the sign bit4. store -4 in the exponent section5. store the normalized binary form

1.10011 x 2-4

Page 61: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Let’s represent 0.110 with 4 bytes in IEEE standard form.0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0

1.10011 x 2-41. 0.110 = 0.000112

2. normalize 0.000112 = 1.10011 x 2-4

3. set the sign bit4. store -4 in the exponent section5. store the normalized binary form

Page 62: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0

Page 63: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Now, let’s convert this back to decimal0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0

Page 64: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Now, let’s convert this back to decimal0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0

1. since the sign bit is 0, we know this is a positive number

Page 65: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Now, let’s convert this back to decimal0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0

1. since the sign bit is 0, we know this is a positive number2. the exponent section is 01111011 (= 123)

Page 66: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Now, let’s convert this back to decimal0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0

1. since the sign bit is 0, we know this is a positive number2. the exponent section is 01111011 (= 123)3. the decimal exponent is 123 - 127 = -4

Page 67: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Now, let’s convert this back to decimal0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0

1. since the sign bit is 0, we know this is a positive number2. the exponent section is 01111011 (= 123)3. the decimal exponent is 123 - 127 = -44. from the number section, we have 1.10011001100110011001100

Page 68: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Now, let’s convert this back to decimal0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0

1. since the sign bit is 0, we know this is a positive number2. the exponent section is 01111011 (= 123)3. the decimal exponent is 123 - 127 = -44. from the number section, we have 1.100110011001100110011005. therefore the number is 1. 10011001100110011001100 x 2-4

Page 69: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Now, let’s convert this back to decimal0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0

1. since the sign bit is 0, we know this is a positive number2. the exponent section is 01111011 (= 123)3. the decimal exponent is 123 - 127 = -44. from the number section, we have 1.100110011001100110011005. therefore the number is 1. 10011001100110011001100 x 2-4

6. 1. 10011001100110011001100 = 1.59999990463 (approximately)

Page 70: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Now, let’s convert this back to decimal0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0

1. since the sign bit is 0, we know this is a positive number2. the exponent section is 01111011 (= 123)3. the decimal exponent is 123 - 127 = -44. from the number section, we have 1.100110011001100110011005. therefore the number is 1. 10011001100110011001100 x 2-4

6. 1. 10011001100110011001100 = 1.59999990463 (approximately)7. 1.59999990463 x 2-4 = 0.099999994

Page 71: Math1003 1.17 - Truncation, Rounding, Overflow, & Conversion Error

MATH1003

10110100101011010100101010111010101111011011101111011101110111101110111011110111111010110100101011110110110101111011010100111111011010100110101001

0.1

Conversion Error

Now, let’s convert this back to decimal0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0

1. since the sign bit is 0, we know this is a positive number2. the exponent section is 01111011 (= 123)3. the decimal exponent is 123 - 127 = -44. from the number section, we have 1.100110011001100110011005. therefore the number is 1. 10011001100110011001100 x 2-4

6. 1. 10011001100110011001100 = 1.59999990463 (approximately)7. 1.59999990463 x 2-4 = 0.099999994

This is a conversion error:

0.099999994 ≠ 0.1