manipulating bit fields in c noah mendelsohn tufts university email:...

Manipulating Bit Fields in C

Noah MendelsohnTufts UniversityEmail: noah@cs.tufts.eduWeb: http://www.cs.tufts.edu/~noah

COMP 40: Machine Structure and

Assembly Language Programming (Spring 2014)

© 2010 Noah Mendelsohn

Goals for this presentation

Learn to use C language to pack and extract bit fields

Also: learn exact width C integer types

2

© 2010 Noah Mendelsohn

Warning:

Because it makes the examples easier to understand, some of the code in these slides uses

the syntax:

0b11001001

for literals. This is a GNU extension to the C language that is not allowed in COMP 40 code. You

may use hex literals like this:

0xC9

as they are standard.

© 2010 Noah Mendelsohn

Exact Width Integer Types

© 2010 Noah Mendelsohn

Why exact width integer types?

The problem: C integer types aren’t fixed size– Size of char, int, long, etc. depends on platform and compiler

– Sometimes we need to get a known size

The solution: stdint.h defines fixed size integers– int32_t: 32 bit signed integer– uint32_t: 32 bit unsigned integer– int16_t: 16 bit signed integer– uint64_t: 64 bit unsigned integer– Etc.

When writing bit packing code, you need to know or account for the size of the integers you’re manipulating

5

© 2010 Noah Mendelsohn

Why Bother with Bit Fields?

© 2010 Noah Mendelsohn

Why use bit fields?

Save space:– Storing 10 million values each ranging from 0-10

– If each is a 32 bit int: 40 megabytes

– If each is a 4 bit int: 5 megabytes

Manipulate standard file formats and network packets

7

© 2010 Noah Mendelsohn

32 bits

V HDLN SVC TYPE LENGTH

ID FLGS FRAG OFFSET

TTL PROTOCOL HDR CHECKSUM

SOURCE ADDRESS

DESTINATION ADDRESS

OPTIONS

THE TCP OR UDP DATA (VARIABLE LEN)

Example: Internet Protocol Packet

8

How can we extract this 8 bit protocol number from the 32 bit field?

© 2010 Noah Mendelsohn

Extracting Bit Fields

© 2010 Noah Mendelsohn

Extracting a bit field

10

uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0; *

* Note that extra spaces have been used in these boolean literals to make them, easier to read. Such spaces are not allowed in code (and, as previously noted, the 0b1001 syntax is a GNU extension not allowed in COMP 40 submissions anyway).

© 2010 Noah Mendelsohn

Extracting a bit field

11

uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0;

uint16_t mask = 0b 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0;

“and” together i and mask

(i & mask) == 0b 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0;

Create mask to select bits we need

© 2010 Noah Mendelsohn

Extracting a bit field

12

uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0;

uint16_t mask = 0b 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0;

“and” together i and mask?

(i & mask) == 0b 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0;

… and shift to finish the job

(i & mask) >> 5 == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1;

© 2010 Noah Mendelsohn

Be careful with signed shifts

13

uint16_t u = 0b 1 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0;

uint16_t mask = 0b 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0;

int16_t i = 0b 1 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0;

Signed shifts usually propagate that bit!

u >> 13 == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1;i >> 13 == 0b 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1;

In this example, masking leaves on high order bit

(u &= mask) == 0b 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0;(i &= mask) == 0b 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0;

© 2010 Noah Mendelsohn

Be careful with signed shifts

14

uint16_t u = 0b 1 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0;

uint16_t mask = 0b 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0;

int16_t i = 0b 1 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0;

Signed shifts usually propagate that bit!

u >> 13 == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1;i >> 13 == 0b 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1;

In this example, masking leaves on high order bit

(u &= mask) == 0b 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0;(i &= mask) == 0b 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0;

Unsigned is correctSigned has unwanted leading 1’s

© 2010 Noah Mendelsohn

Be careful with signed shifts

15

uint16_t u = 0b 1 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0;

uint16_t mask = 0b 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0;

int16_t i = 0b 1 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0;

Signed shifts usually propagate that bit!

u >> 13 == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1;i >> 13 == 0b 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1;

In this example, masking leaves on high order bit

u &= mask: 0b 1 0 1 0 0 0 0 0 0 0 0 0 0 1 0 1;i &= mask == 0b 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1;

Huh? Turns out that right shifting of a signed integer is implementation define…but

why?

C wants these operations to be super-efficient, and on some hardware there’s no

efficient signed shift.

On our systems, the sign will propagate and you may count on that in your

homework submissions.

© 2010 Noah Mendelsohn

How can we choose the bits to extract at runtime?

16

unsigned short i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0;int width = 3; int offset = 5;

/* mask is now 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 */

mask = mask >> (16 - width) << (offset);

This time we’ll have to compute the mask

unsigned short mask = ~0;

/* mask is now 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 */

(i & mask) >> offset == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1;

Now, finish the job as before

© 2010 Noah Mendelsohn

We can adapt to integer size at runtime

17

unsigned short i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0;int width = 3; int offset = 5;

/* mask is now 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 */

mask = mask >> (number_of_bits - width) << (offset);

This time we’ll have to compute the mask

unsigned short mask = ~0;

/* mask is now 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 */

(i & mask) >> offset == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1;

Now, finish the job as before

const number_of_bits = sizeof(unsigned short) * 8;

© 2010 Noah Mendelsohn

Setting Bit Fields

© 2010 Noah Mendelsohn

Putting new value in a bit field

20

uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0;uint16_t new_value = 0b 0 1 0; /* value we want *

uint16_t mask = 0b 1 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1;

“and” together i and mask

(i & mask) == 0b 1 0 0 1 1 1 0 0 0 0 0 0 1 1 1 0;

© 2010 Noah Mendelsohn

Putting new value in a bit field

21

uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0;uint16_t new_value = 0b 0 1 0; /* value we want *

uint16_t mask = 0b 1 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1;

“and” together i and mask…

(i & mask) == 0b 1 0 0 1 1 1 0 0 0 0 0 0 1 1 1 0;

Important: we have zeros where new value is going

© 2010 Noah Mendelsohn

Putting new value in a bit field

22

uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0;uint16_t new_value = 0b 0 1 0; /* value we want *

uint16_t mask = 0b 1 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1;

“and” together i and mask…

(i & mask) == 0b 1 0 0 1 1 1 0 0 0 0 0 0 1 1 1 0;

… shift the new value and use “or” to combine

(i | (new_value << 5)) == 0b 1 0 0 1 1 1 0 0 0 1 0 0 1 1 1 0;

© 2010 Noah Mendelsohn

Flag Bits

© 2010 Noah Mendelsohn

Very common idiom – single bit flags

24

/* Flag definitions */const uint16_t HI_PRIORITY = 0x8000;const uint16_t SECURE = 0x4000;const uint16_t ARCHIVED = 0x2000;…const uint16_t BEAUTIFUL = 0x0002;const uint16_t SPECTACULAR = 0x0001;

One bit each

© 2010 Noah Mendelsohn

Very common idiom – single bit flags

25

/* Flag definitions */const uint16_t HI_PRIORITY = 0b1000000000000000;const uint16_t SECURE = 0b0100000000000000;const uint16_t ARCHIVED = 0b0010000000000000;…const uint16_t BEAUTIFUL = 0b0000000000000010;const uint16_t SPECTACULAR = 0b0000000000000001;

Can use binary instead of hex, but hard to count the

zeros!

© 2010 Noah Mendelsohn

Initializing flags

26

/* Flag definitions */const uint16_t HI_PRIORITY = 0b1000000000000000;const uint16_t SECURE = 0b0100000000000000;const uint16_t ARCHIVED = 0b0010000000000000;…const uint16_t BEAUTIFUL = 0b0000000000000010;const uint16_t SPECTACULAR = 0b0000000000000001;

Use | to combine bits

uint16_t myflags = SECURE | BEAUTIFUL; /* secure and beautiful */

© 2010 Noah Mendelsohn

Testing flags

27

/* Flag definitions */const uint16_t HI_PRIORITY = 0b1000000000000000;const uint16_t SECURE = 0b0100000000000000;const uint16_t ARCHIVED = 0b0010000000000000;…const uint16_t BEAUTIFUL = 0b0000000000000010;const uint16_t SPECTACULAR = 0b0000000000000001;

Test flags with & remember C treats

anything != 0 as true!

if (myflags & BEAUTIFUL) {…}; /* if beautiful */

© 2010 Noah Mendelsohn

Testing flags

28

/* Flag definitions */const uint16_t HI_PRIORITY = 0b1000000000000000;const uint16_t SECURE = 0b0100000000000000;const uint16_t ARCHIVED = 0b0010000000000000;…const uint16_t BEAUTIFUL = 0b0000000000000010;const uint16_t SPECTACULAR = 0b0000000000000001;

Testing multiple flags on

if ((myflags & (BEAUTIFUL | SECURE)) == (BEAUTIFUL | SECURE)) {…}; /* if beautiful and secure */

© 2010 Noah Mendelsohn

Turning flags on

29

/* Flag definitions */const uint16_t HI_PRIORITY = 0b1000000000000000;const uint16_t SECURE = 0b0100000000000000;const uint16_t ARCHIVED = 0b0010000000000000;…const uint16_t BEAUTIFUL = 0b0000000000000010;const uint16_t SPECTACULAR = 0b0000000000000001;

Us | to turn on additional flags

myflags |= ARCHIVED; /* now it’s archived too */

© 2010 Noah Mendelsohn

Turning flags off

30

/* Flag definitions */const uint16_t HI_PRIORITY = 0b1000000000000000;const uint16_t SECURE = 0b0100000000000000;const uint16_t ARCHIVED = 0b0010000000000000;…const uint16_t BEAUTIFUL = 0b0000000000000010;const uint16_t SPECTACULAR = 0b0000000000000001;

To turn off, & with all the other flags!

myflags &= ~BEAUTIFUL; /* but not beautiful anymore */