6.172 - mit opencourseware · binary representation let x = x w–1x w–2…x o be a w-bit...

70
!"##$ %&'&( ! "#) +)$#) +, -./01 6.172 Performance Engineering of Software Systems © 2008–2018 by the MIT 6.172 Lecturers LECTURE 3 Bit Hacks Julian Shun 1

Upload: others

Post on 03-Jul-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

!"##$*%&'&(!"#)*+)$#)*+,*-./01*

6.172 Performance Engineering of Software Systems

© 2008–2018 by the MIT 6.172 Lecturers

LECTURE 3 Bit Hacks

Julian Shun

1

Page 2: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Binary Representation Let x = ⟨xw–1xw–2…xO⟩ be a w-bit computer word. The unsigned integer value stored in x is

For example, the 8-bit word Ob1OO1O11O represents

The prefix Ob designates a

Boolean constant.

the unsigned value 15O = 2 + 4 + 16 + 128.

The signed integer (two’s complement) value stored in x is

For example, the 8-bit word Ob1OO1O11O represents the signed value –1O6 = 2 + 4 + 16 – 128.

sign bit

© 2008–2018 by the MIT 6.172 Lecturers 2

Page 3: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Two’s Complement

We have ObOO…O = O.

What is the value of x = Ob11...1?

© 2008–2018 by the MIT 6.172 Lecturers 3

Page 4: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Complementary Relationship

Important identity

Since we have x + ~x = -1, it follows that

-x = ~x + 1.

Example x = ObO11O11OOO

~x = Ob1OO1OO111 –x = Ob1OO1O1OOO

© 2008–2018 by the MIT 6.172 Lecturers 4

Page 5: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Binary and Hexadecimal Decimal Hex Binary Decimal Hex Binary

! ! ! ! ! ! " " # ! ! !# # ! ! ! # $ $ # ! ! #% % ! ! # ! #! & # ! # !' ' ! ! # # ## ( # ! # #) ) ! # ! ! #% * # # ! !

+ ! # ! # #' , # # ! #- ! # # ! #) . # # # !/ ! # # # #+ 0 # # # #

To translat from hex to binary, translate each hex digit to its binary equivalent, and concatenate the bits.

The prefix !1designates a hex constant.

Example: 0xDEC1DE2CODE4F00D is

© 2008–2018 by the MIT 6.172 Lecturers 5

Page 6: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

C Bitwise Operators

Operator Description

& AND

| OR

^ XOR (exclusive OR)

~ NOT (one’s complement)

<< shift left

>> shift right

Examples (8-bit word) A = Ob1O11OO11B = ObO11O1OO1

A&B = ObOO1OOOO1 ~A = ObO1OO11OOA|B = Ob11111O11 A >> 3 = ObOOO1O11OA^B = Ob11O11O1O A << 2 = Ob11OO11OO

© 2008–2018 by the MIT 6.172 Lecturers 6

Page 7: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Set the kth Bit Problem Set !th bit in a word "%to #.

Idea Shift and OR.

Example!%*%+%

,%*%"%'%(#%&&%!)-%

# $ # # # # $ # %$%# # $ # # $ # %$ $ $ $ $ $ $ $ %#%$ $ $ $ $ $ $ %# $ # # # # $ # %#%# # $ # # $ # %

© 2008–2018 by the MIT 6.172 Lecturers 7

"%#%&&%!%

"%'%(#%&&%!)%

Page 8: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Clear the kth Bit Problem Clear the !th bit in a word ".

Idea Shift, complement, and AND.

-%+%"%*%'(#%&&%!).%

Example!%+%,%

"% #$# # # # $ # %#%# # $ # # $ # %

© 2008–2018 by the MIT 6.172 Lecturers 8

"%*%'(#%&&%!)%

$ $ $ $ $ $ $ $ %#%$ $ $ $ $ $ $ %# # # # # # # # %$%# # # # # # # %# $ # # # # $ # %$%# # $ # # $ # %

#%&&%!%'(#%&&%!)%

Page 9: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Toggle the kth Bit Problem Flip the !th bit in a word ".

Idea Shift and XOR.

,%*%"%'%(#%&&%!)-%

Example ($%! #) !%*%+%

"% #$# # # # $ # %$%# # $ # # $ # %$ $ $ $ $ $ $ $ %#%$ $ $ $ $ $ $ %# $ # # # # $ # %#%# # $ # # $ # %

© 2008–2018 by the MIT 6.172 Lecturers 9

#%&&%!%"%'%(#%&&%!)%

Page 10: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Toggle the kth Bit Problem Flip the !th bit in a word ".

Idea Shift and XOR.

*%+%"%'%(#%&&%!),%

Example (#%! $) !%+%-%

"% #$# # # # $ # %#%# # $ # # $ # %$ $ $ $ $ $ $ $ %#%$ $ $ $ $ $ $ %# $ # # # # $ # %$%# # $ # # $ # %

© 2008–2018 by the MIT 6.172 Lecturers 10

#%&&%!%"%'%(#%&&%!)%

Page 11: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Extract a Bit Field Problem Extract a bit field from a word !.

Idea Mask and shift.

1!()($%&'2(**(&+,-.3(

Example&+,-.(/(0(

!( "#" " " (" # " # (" " # " " # " (# # # # # (" " " " (# # # # # # # (# # # # # (" # " # (# # # # # # # (# # # # # # # # # # # # (" # " # (

© 2008–2018 by the MIT 6.172 Lecturers 11

$%&'(!()($%&'(

!()($%&'(**(&+,-.(

Page 12: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Set a Bit Field Problem Set a bit field in a word !)to a value ".

Idea Invert mask to clear, and OR the shifted value.

!),)-!)*)+%&'(.)/)-")00)'1234.5)

Example'1234),)6)

!) #$# # # )# $ # $ )# # $ # # $ # )$ $ $ $ $ $ $ $ $ $ $ $ )$ $ # # )$ $ $ $ $ )# # # # )$ $ $ $ $ $ $ )# $ # # # )$ $ $ $ )# # $ # # $ # )# $ # # # )$ $ # # )# # $ # # $ # )!),)-!)*)+%&'(.)/)-")00)'1234.5)

© 2008–2018 by the MIT 6.172 Lecturers 12

")%&'()

!)*)+%&'()

Page 13: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Set a Bit Field Problem Set a bit field in a word !)to a val

Idea Invert mask to clear, and OR the shi d value.

!),)-!)*)+%&'(.)/)-")00)'1234.5)

For safety’s sake: --")00)'1234.)*)%&'(.)

Example'1234),)6)

!) #$# # # )# $ # $ )# # $ # # $ # )$ $ $ $ $ $ $ $ $ $ $ $ )$ $ # # )$ $ $ $ $ )# # # # )$ $ $ $ $ $ $ )# $ # # # )$ $ $ $ )# # $ # # $ # )# $ # # # )$ $ # # )# # $ # # $ # )!),)-!)*)+%&'(.)/)-")00)'1234.5)

© 2008–2018 by the MIT 6.172 Lecturers 13

")%&'()

!)*)+%&'()

Page 14: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Ordinary Swap Problem Swap two integers # and %.

! " #$# " %$% " !$

© 2008–2018 by the MIT 6.172 Lecturers 14

Page 15: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

No-Temp Swap Problem Swap ! and $ without using a temporary.

! " ! # $%$ " ! # $%! " ! # $%

Example ! & ' & & & & ' &$ ' ' & ' & & & '

© 2008–2018 by the MIT 6.172 Lecturers 15

Page 16: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

No-Temp Swap Problem Swap !"and %"without using a temporary.

!"#"!"$"%&"%"#"!"$"%&"!"#"!"$"%&"

Example !" ' ( ' ' ' ' ( ' " ' ( ( ' ( ( ' ' "%" ( ( ' ( ' ' ' ( "

© 2008–2018 by the MIT 6.172 Lecturers 16

( ( ' ( ' ' ' ( "

Page 17: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

No-Temp Swap Problem Swap !&and "&without using a temporary.

!&#&!&$&"%&"&#&!&$&"%&!&#&!&$&"%&

Example !& ' ( ' ' ' ' ( ' & ' ( ( ' ( ( ' ' & ' ( ( ' ( ( ' ' &"& ( ( ' ( ' ' ' ( & ( ( ' ( ' ' ' ( & ' ( ' ' ' ' ( ' &

© 2008–2018 by the MIT 6.172 Lecturers 17

Page 18: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

No-Temp Swap Problem Swap !(and $(without using a temporary.

!(%(!(&($'($(%(!(&($'(!(%(!(&($'(

Example !( " # " " " " # " ( " # # " # # " " ( " # # " # # " " ( # # " # " " " # ($( # # " # " " " # ( # # " # " " " # ( " # " " " " # " ( " # " " " " # " (

© 2008–2018 by the MIT 6.172 Lecturers 18

Page 19: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

No-Temp Swap Problem Swap ! and $ without using a temporary.

! " ! # $%$ " ! # $%! " ! # $%

Example ! & ' & & & & ' & & ' ' & ' ' & & & ' ' & ' ' & & ' ' & ' & & & '$ ' ' & ' & & & ' ' ' & ' & & & ' & ' & & & & ' & & ' & & & & ' &

Why it works !# "# !#$#"# %!#$#"&#$#"#

' ' ' 'XOR is its own inverse: ' & & '(! # $) # $ ! !& ' & && & ' &

© 2008–2018 by the MIT 6.172 Lecturers 19

Page 20: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

No-Temp Swap (Instant Replay) Problem Swap ! and $ without using a temporary.

! " ! # $%$ " ! # $%! " ! # $%

Example ! & ' & & & & ' &$ ' ' & ' & & & '

Why it works !# "# !#$#"# %!#$#"&#$#"#

' ' ' 'XOR is its own inverse: ' & & '(! # $) # $ ! !& ' & && & ' &

© 2008–2018 by the MIT 6.172 Lecturers 20

Page 21: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

No-Temp Swap (Instant Replay) Problem Swap !"and %"without using a temporary.

!"#"!"$"%&"%"#"!"$"%&"!"#"!"$"%&"

Mask with 1’s where bits differ.

Example !" ' ( ' ' ' ' ( ' " ' ( ( ' ( ( ' ' "%" ( ( ' ( ' ' ' ( "

Why it works !# "# !#$#"# %!#$#"&#$#"#

( ( (" ("XOR is its own inverse: ( ' '" (")!"$"%*"$"%"! !"' ( '" '"' ' (" '"

© 2008–2018 by the MIT 6.172 Lecturers 21

( ( ' ( ' ' ' ( "

Page 22: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

No-Temp Swap (Instant Replay) Problem Swap !&and $&without using a temporary.

!&"&!&#&$%&$&"&!&#&$%&!&"&!&#&$%&

Flip bits in $&that differ from !.

Example !& ) * ) ) ) ) * ) & ) * * ) * * ) ) & ) * * ) * * ) ) &$& * * ) * ) ) ) * & * * ) * ) ) ) * & ) * ) ) ) ) * ) &

Why it works !# "# !#$#"# %!#$#"&#$#"#

* * *& *&XOR is its own inverse: * ) )& *&'!&#&$(&#&$&! !&) * )& )&) ) *& )&

© 2008–2018 by the MIT 6.172 Lecturers 22

Page 23: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

No-Temp Swap (Instant Replay) Problem Swap !&and $&without using a temporary.

!&"&!&#&$%&$&"&!&#&$%&!&"&!&#&$%&

Flip bits in !&that differ from $.

Example !& ' ( ' ' ' ' ( ' & ' ( ( ' ( ( ' ' & ' ( ( ' ( ( ' ' & ( ( ' ( ' ' ' ( &$& ( ( ' ( ' ' ' ( & ( ( ' ( ' ' ' ( & ' ( ' ' ' ' ( ' & ' ( ' ' ' ' ( ' &

Why it works !# "# !#$#"# %!#$#"&#$#"#

( ( (& (&XOR is its own inverse: ( ' '& (&)!&#&$*&#&$&! !&' ( '& '&' ' (& '&

© 2008–2018 by the MIT 6.172 Lecturers 23

Page 24: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

No-Temp Swap (Instant Replay) Problem Swap " and $ without using a temporary.

" & " # $'$ & " # $'" & " # $'

Example " ( ) ( ( ( ( ) ( ( ) ) ( ) ) ( ( ( ) ) ( ) ) ( ( ) ) ( ) ( ( ( )$ ) ) ( ) ( ( ( ) ) ) ( ) ( ( ( ) ( ) ( ( ( ( ) ( ( ) ( ( ( ( ) (

Why it works XOR is its own inverse: !" # $% # $ ! "Performance !Poor at exploiting instruction-level parallelism (ILP).

© 2008–2018 by the MIT 6.172 Lecturers 24

Page 25: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Minimum of Two Integers

Problem Find the minimum ! of two integers " and #.

$% &" ' #(! ) "*

+,-+! ) #*

or ! ) &" ' #( . " / #*

Performance A mispredicted branch empties the processor pipeline.

Caveat The compiler is usually smart enough to optimize away the unpredictable branch, but maybe not.

© 2008–2018 by the MIT 6.172 Lecturers 25

Page 26: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

No-Branch Minimum

Problem Find the minimum ! of two integers " and # without using a branch.

! $ # % &&" % #' ( )&" * #''+

Why it works ! The C language represents the Booleans TRUE and

FALSE with the integers , and -, respectively.! If " * #, then )&" * #' ! .,, which is all ,’s in

two’s complement representation. Therefore, wehave # % &" % #' ! ".

! If " " #, then )&" * #' ! -. Therefore, we have# % - ! #.

© 2008–2018 by the MIT 6.172 Lecturers 26

Page 27: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Merging Two Sorted Arrays

J FI FG HK

H FH IF IJ

!"#"$% &'$( )*+,*-.'/, 0 11+*!"+$%" 23.'/, 0 11+*!"+$%" 43.'/, 0 11+*!"+$%" 53!$6*1" /#3!$6*1" /78 9

:;$.* -/# < = >> /7 < =8 9$? -04 @A 058 902BB A 04BBC /#DDC

E *.!* 902BB A 05BBC /7DDC

EE:;$.* -/# < =8 902BB A 04BBC/#DDC

E:;$.* -/7 < =8 902BB A 05BBC/7DDC

EE

© 2008–2018 by the MIT 6.172 Lecturers 27

Page 28: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Branching !"#"$% &'$( )*+,*-.'/, 0 11+*!"+$%" 23

.'/, 0 11+*!"+$%" 43

.'/, 0 11+*!"+$%" 53!$6*1" /#3!$6*1" /78 9

:;$.* -/# < = >> /7 < =8 9$? -04 @A 058 902BB A 04BBC /#DDC

E *.!* 902BB A 05BBC /7DDC

EE:;$.* -/# < =8 902BB A 04BBC/#DDC

E:;$.* -/7 < =8 902BB A 05BBC/7DDC

EE

1

2

3

4

© 2008–2018 by the MIT 6.172 Lecturers

Branch Predictable? 1 2 3 4

? ? ? ?

Yes Yes No Yes

28

Page 29: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Branchless !"#"$%H&'$(H)*+,*-.'/,H0H11+*!"+$%"H23H

.'/,H0H11+*!"+$%"H43H

.'/,H0H11+*!"+$%"H53H

!$6*1"H/#3H

!$6*1"H/78H9H

:;$.*H-/#H<H=H>>H/7H<H=8H9H

.'/,H%)?H@H-04HA@H058BH

.'/,H)$/H@H05HCH--05HCH048H>H-D%)?88BH

02EEH@H)$/BH

4HE@H %)?BH/#HD@H %)?BH

5HE@HF%)?BH/7HD@HF%)?BH

GH

:;$.*H-/#H<H=8H9H

02EEH@H04EEBH

/#DDBH

GH

:;$.*H-/7H<H=8H 9H

02EEH@H05EEBH

/7DDBH

GH

GH

This optimization works well on some machines, but on modern machines using %.#/,HD=I, the branchless version is usually slower than the branching version. ! Modern compilers can perform this optimization better than you can!

© 2008–2018 by the MIT 6.172 Lecturers 29

Page 30: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Why Learn Bit Hacks?

Why learn bit hacks if they don’t even work? • Because the compiler does them, and it will help to

understand what the compiler is doing when youlook at the assembly code.

• Because sometimes the compiler doesn’t optimize,and you have to do it yourself by hand.

• Because many bit hacks for words extend naturallyto bit and word hacks for vectors.

• Because these tricks arise in other domains, and soit pays to be educated about them.

• Because they’re fun!

© 2008–2018 by the MIT 6.172 Lecturers 30

Page 31: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Modular Addition

Problem Compute (!%+ ") mod #, assuming that $%! !%< #%and $%! "%< #.

&%'%(!%)%"*%+%#,% Division is expensive, unless by a power of 2.

-%'%!%)%",%&%'%(-%.%#*%/%-%0%-1#,%

-%'%!%)%",%&%'%-%1%(#%3%1(-%4'%#**,%

Unpredictablebranch is expensive.

Same trick as minimum.

© 2008–2018 by the MIT 6.172 Lecturers 31

Page 32: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Round up to a Power of 2 Problem Compute 2⌈lg n⌉. Notation

lg n = log2n

© 2008–2018 by the MIT 6.172 Lecturers 32

Page 33: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Round up to a Power of 2 Problem Compute ."lg ##.

!"#$%&'$ #(!))#(# *+ # ,, -(# *+ # ,, .(# *+ # ,, &(# *+ # ,, /(# *+ # ,, -%(# *+ # ,, 0.(11#(

Example 22-222222-2-2222

© 2008–2018 by the MIT 6.172 Lecturers 33

Page 34: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Round up to a Power of 2 Problem Compute ."lg ##.

!"#$%&'$ #(!))#(# *+ # ,, -(# *+ # ,, .(# *+ # ,, &(# *+ # ,, /(# *+ # ,, -%(# *+ # ,, 0.(11#(

Example 22-222222-2-222222-222222-22----

© 2008–2018 by the MIT 6.172 Lecturers 34

Page 35: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Round up to a Power of 2 Problem Compute /"lg ##.

!"#$%&'$*#(*!))#(*#*+,*#*--*.(*#*+,*#*--*/(*#*+,*#*--*&(*#*+,*#*--*0(*#*+,*#*--*.%(*#*+,*#*--*1/(*22#(*

Example 33.333333.3.3333*33.333333.33....*33..33333..3....*

© 2008–2018 by the MIT 6.172 Lecturers 35

Page 36: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Round up to a Power of 2 Problem Compute /"lg ##.

!"#$%&'$.#(.!))#(.#.*+.#.,,.-(.#.*+.#.,,./(.#.*+.#.,,.&(.#.*+.#.,,.0(.#.*+.#.,,.-%(.#.*+.#.,,.1/(.22#(.

Example 33-333333-3-3333.33-333333-33----.33--33333--3----.33----333-------.

© 2008–2018 by the MIT 6.172 Lecturers 36

Page 37: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Round up to a Power of 2 Problem Compute ."lg ##.

!"#$%&'$/#(/!))#(/#/*+/#/,,/-(/#/*+/#/,,/.(/#/*+/#/,,/&(/#/*+/#/,,/0(/#/*+/#/,,/-%(/#/*+/#/,,/1.(/22#(/

Example 33-333333-3-3333/33-333333-33----/33--33333--3----/33----333-------/33--------------/

© 2008–2018 by the MIT 6.172 Lecturers 37

Page 38: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Round up to a Power of 2 Problem Compute ."lg ##.

!"#$%&'$/#(/!))#(/#/*+/#/,,/-(/#/*+/#/,,/.(/#/*+/#/,,/&(/#/*+/#/,,/0(/#/*+/#/,,/-%(/#/*+/#/,,/1.(/22#(/

Example 33-333333-3-3333/33-333333-33----/33--33333--3----/33----333-------/33--------------/

© 2008–2018 by the MIT 6.172 Lecturers 38

Page 39: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Round up to a Power of 2 Problem Compute ."lg ##.

!"#$%&'$0#(0!))#(0#0*+0#0,,0-(0#0*+0#0,,0.(0#0*+0#0,,0&(0#0*+0#0,,0/(0#0*+0#0,,0-%(0#0*+0#0,,01.(022#(0

Example 33-333333-3-3333033-333333-33----033--33333--3----033----333-------033--------------0

© 2008–2018 by the MIT 6.172 Lecturers 39

Page 40: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Round up to a Power of 2 Problem Compute ."lg ##.

!"#$%&'$0#(0!))#(0#0*+0#0,,0-(0#0*+0#0,,0.(0#0*+0#0,,0&(0#0*+0#0,,0/(0#0*+0#0,,0-%(0#0*+0#0,,01.(022#(0

Example 33-333333-3-3333033-333333-33----033--33333--3----033----333-------033--------------0

© 2008–2018 by the MIT 6.172 Lecturers 40

Page 41: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Round up to a Power of 2 Problem Compute ."lg ##.

!"#$%&'$ #(!))#(# *+ # ,, -(# *+ # ,, .(# *+ # ,, &(# *+ # ,, /(# *+ # ,, -%(# *+ # ,, 0.(11#(

Example 22-222222-2-222222-222222-22----22--22222--2----22----222-------22--------------2-22222222222222

© 2008–2018 by the MIT 6.172 Lecturers 41

Page 42: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Round up to a Power of 2 Problem Compute !!lg "".

#$"%&'(% ")#**")" +, " -- .)" +, " -- !)" +, " -- ')" +, " -- /)" +, " -- .&)" +, " -- 0!)11")

Example

Bit !lg "" – . must be set

Populate all bits to the right with .Set bit !lg ""

22.222222.2.222222.222222.22....22..22222..2....22....222.......22..............2.22222222222222

© 2008–2018 by the MIT 6.172 Lecturers 42

Page 43: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Round up to a Power of 2 Problem Compute ."lg ##.

!"#$%&'$ #(!))#(# *+ # ,, -(# *+ # ,, .(# *+ # ,, &(# *+ # ,, /(# *+ # ,, -%(# *+ # ,, 0.(11#(

Why decrement?

Example 22-222222-2-222222-222222-22----22--22222--2----22----222-------22--------------2-22222222222222

To handle the boundary case when # is a power of .. © 2008–2018 by the MIT 6.172 Lecturers 43

Page 44: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Least-Significant 1 Problem Compute the mask of the least-significant ! in word ".

# $ " % &'"()

Example " **!******!*!****'" !!*!!!!!!*!!****

" % &'"( ***********!****

Why it works The binary representation of '" is (+")+!. Question How do you find the index of the bit, i.e., lg #?

© 2008–2018 by the MIT 6.172 Lecturers 44

Page 45: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Log Base 2 of a Power of 2

Problem Compute lg 2, where 2 is a power of 3.

!"#$% &'#%()*% +,-.&'/# 0 121334++(5!!6758(+9!"#$% '#% !"#:,.%;()< 0 =

1> ?> 3> 75> 5> @> 7)> 3@>)> 58> )?> 8> 5)> 77> )8> 38>

(3> 7> 56> )(> ))> )3> 33> 6>3)> 57> 76> 7(> )6> ?8> 36> ??>(5> 73> (> 3(> 5@> )1> 55> )@>(?> )7> )5> 3?> 35> 78> ?@> ?1>7?> 37> 5(> 53> (1> 31> 7@> ?(>71> 5?> ?6> ?7> 51> ?)> ?5> ?3

A9. 0 !"#:,.%;B2 C +,-.&'/#D EE 78<9

© 2008–2018 by the MIT 6.172 Lecturers 45

Page 46: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Mathemagic Trick

5 volunteers who can follow directions

Introducing Jess Ray, “The Golden Raytio”

© 2008–2018 by the MIT 6.172 Lecturers 46

Page 47: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Log Base 2 of a Power of 2

OOO111O1O OOO1 OO12 O113 1114 11O5 1O16 O1O7 1OO

⇒ Ob11O1OOOO

start with all O’s

Why it works Example: k=3 A deBruijn sequence s of length 2k is a cyclic O-1 sequence such that each of the 2k O-1 strings of length k occurs exactly once as a substring of s.

ObOOO111O1*24 Ob11O1OOOO >> 5 ⇒ 6 convert[6] ⇒ 4

Performance Limited by multiply and const int convert[8] table look-up. = {O,1,6,2,7,5,4,3};

© 2008–2018 by the MIT 6.172 Lecturers 47

Page 48: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Queens Problem

Problem Place ! queens on an ! ! ! chessboard so that no queen attacks another, i.e., no two queens in any row, column, or diagonal. Count the number of possible solutions.

© 2008–2018 by the MIT 6.172 Lecturers 48

Page 49: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Backtracking Search

Strategy Try placing queens row by row. If you can’t place a queen in a row, backtrack.

© 2008–2018 by the MIT 6.172 Lecturers 49

Page 50: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Backtracking Search

Strategy Try placing queens row by row. If you can’t place a queen in a row, backtrack.

© 2008–2018 by the MIT 6.172 Lecturers 50

Page 51: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Backtracking Search

Strategy Try placing queens row by row. If you can’t place a queen in a row, backtrack.

© 2008–2018 by the MIT 6.172 Lecturers 51

Page 52: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Backtracking Search

Strategy Try placing queens row by row. If you can’t place a queen in a row, backtrack.

© 2008–2018 by the MIT 6.172 Lecturers 52

Page 53: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Backtracking Search

Strategy Try placing queens row by row. If you can’t place a queen in a row, backtrack.

© 2008–2018 by the MIT 6.172 Lecturers 53

Page 54: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Backtracking Search

Strategy Try placing queens row by row. If you can’t place a queen in a row, backtrack.

Backtrack!

© 2008–2018 by the MIT 6.172 Lecturers 54

Page 55: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Backtracking Search

Strategy Try placing queens row by row. If you can’t place a queen in a row, backtrack.

© 2008–2018 by the MIT 6.172 Lecturers 55

Page 56: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Backtracking Search

Strategy Try placing queens row by row. If you can’t place a queen in a row, backtrack.

Backtrack!

© 2008–2018 by the MIT 6.172 Lecturers 56

Page 57: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Backtracking Search

Strategy Try placing queens row by row. If you can’t place a queen in a row, backtrack.

Backtrack!

© 2008–2018 by the MIT 6.172 Lecturers 57

Page 58: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Backtracking Search

Strategy Try placing queens row by row. If you can’t place a queen in a row, backtrack.

© 2008–2018 by the MIT 6.172 Lecturers 58

Page 59: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Board Representation

The backtrack search can be implemented as a simple recursive procedure, but how should the board be represented to facilitate queen placement? • array of n2 bytes?• array of n2 bits?• array of n bytes?• 3 bitvectors of size n, 2n-1, and 2n-1.

© 2008–2018 by the MIT 6.172 Lecturers 59

Page 60: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

!"

Bitvector Representation

Placing a queen in column '*is not safe if

#$%&*(*)"*++*',-*is nonzero.

" " ! " " " ! "*#$%&*

© 2008–2018 by the MIT 6.172 Lecturers 60

Page 61: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

"

Bitvector Representation

!+!"+!+

"+"+

!+

!+

Placing a queen in row '+and column (+is not safe if

#$%&+)+*!+,,+*'-(..+is nonzero.

" ! " " ! " " "+#$%&+

© 2008–2018 by the MIT 6.172 Lecturers 61

Page 62: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Bitvector Representation

" !

Placing a queen in row #+and !+ column (+is not safe if !+ #$%&'+)+*"+,,+*-.".#/(00+

is nonzero. "+!+"+"+!+

! ! ! ! " " ! "+#$%&'+

© 2008–2018 by the MIT 6.172 Lecturers 62

Page 63: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Population Count I Problem Count the number of ! bits in a word ".

#$% &%'() "*'() ++%," -' " . !)

Repeatedly eliminate the least-significant 1.

Example

((!(!!(!!!(!((((((!(!!(!!!((!!!!((!(!!(!!!((((((

Issue Fast if the popcount is small, but in the worst case, the running time is proportional to the number of bits in the word.

© 2008–2018 by the MIT 6.172 Lecturers 63

"" . !

" - &" . !,)

Page 64: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Population Count II

!"#"$%4%&'!"4$'"4%&('")*+,-4.4/4014214214*14214*14*14314214!1454674

8&94:$'"494.4074;4<.4074;4==.45>494?.4%&('");4@40;AA-74

Table look-up

Performance depends on the size of ;. The cost of memory operations is a major bottleneck. Typical costs: ! register: 24cycle,! L1-cache: B4cycles,! L2-cache: 204cycles,

per ,B-byte cache line ! L3-cache: +04cycles,! DRAM: 2+04cycles.

© 2008–2018 by the MIT 6.172 Lecturers 64

Page 65: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Population Count III Parallel divide-and-conquer

!!"#$%&'%"(&)*)"+,"-".//012"33"4526"""!!"745145"+8"-"+,"9"/+,"33"1:26"!!"/71:11:25"+4"-"+8"9"/+8"33";26" !!"/7;1;28"+5"-"+4"9"/+4"33"826" !!"/78182;"+1"-"+5"9"/+5"33"526" !!"/751521:"+7"-"+1"9"/+1"33"126""!!"/71245"!!"#<(=>'%"=<=?<>@'"A"-"//A"BB"12"C"+72"D"/A"C"+726"A"-"//A"BB"52"C"+12"D"/A"C"+126"A"-"//A"BB"82"D"A2"C"+56"A"-"//A"BB";2"D"A2"C"+46"A"-"//A"BB"1:2"D"A2"C"+86"A"-"//A"BB"452"D"A2"C"+,6"

Notation: E*"= EE!E"

*"times

© 2008–2018 by the MIT 6.172 Lecturers 65

Page 66: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Population Count III

1 1 0 0 0 0 1 0 0 1 0 1 1 0 1 1 1 1 1 1 0 1 0 0 0 1 1 1 1 0 0 0 1 0 0 0 1 1 0 1 1 1 1 0 1 1 0 0 x&MO

+ 1 0 0 1 0 0 1 1 1 1 0 0 0 1 1 0 (x>>1)&MO

0 0 0 1 0 1 1 0 1 0 0 0 1 0 0 0 x&M1

+ 1 0 0 0 0 1 0 1 1 0 0 1 0 1 0 1 (x>>2)&M1

0 0 0 1 0 0 1 1 0 0 0 1 0 0 0 1 x&M2

+ 0 0 1 0 0 0 1 0 0 1 0 0 0 0 1 1 (x>>4)&M2

0 1 0 1 1 0 1 1 0 0 0 0 0 1 0 0 x&M3

+ 1 1 0 0 0 0 1 0 0 0 0 0 0 1 0 1 (x>>8)&M3

0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 x&M4

+ 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 (x>>16)&M4

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1

17

© 2008–2018 by the MIT 6.172 Lecturers 66

Page 67: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Population Count III Parallel divide-and-conquer

Performance !(lg w) time, where w = word length.

!!"#$%&'%"(&)*)"+,"-".//012"33"4526"""!!"745145"+8"-"+,"9"/+,"33"1:26"!!"/71:11:25"+4"-"+8"9"/+8"33";26" !!"/7;1;28"+5"-"+4"9"/+4"33"826" !!"/78182;"+1"-"+5"9"/+5"33"526" !!"/751521:"+<"-"+1"9"/+1"33"126""!!"/71245"!!"#=(>?'%">=>@=?A'"B"-"//B"CC"12"D"+<2"E"/B"D"+<26"B"-"//B"CC"52"D"+12"E"/B"D"+126"B"-"//B"CC"82"E"B2"D"+56"B"-"//B"CC";2"E"B2"D"+46"B"-"//B"CC"1:2"E"B2"D"+86"B"-"//B"CC"452"E"B2"D"+,6"

Avoid overflow

No worryabout

overflow.

© 2008–2018 by the MIT 6.172 Lecturers 67

Page 68: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Popcount Instructions

Most modern machines provide popcount instructions, which operate much faster than anything you can code yourself. You can access them via compiler intrinsics, e.g., in GCC:

int __builtin_popcount (unsigned int x); Warning: You may need to enable certain compiler switches to access built-in functions, and your code may be less portable.

Exercise Compute the log base 2 of a power of 2 quickly using a popcount instruction.

© 2008–2018 by the MIT 6.172 Lecturers 68

Page 69: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

Further Reading

Sean Eron Anderson, “Bit twiddling hacks,” http://graphics.stanford.edu/~seander/bithacks.h

tml, 2009. Donald E. Knuth, The Art of Computer Programming , Volume 4A, Combinatorial Algorithms, Part 1 , Addison-Wesley, 2011, Section 7.1.3.

Henry S. Warren, Hacker’s Delight , Addison-Wesley, 2003.

Happy Bit-Hacking!

© 2008–2018 by the MIT 6.172 Lecturers 69

Page 70: 6.172 - MIT OpenCourseWare · Binary Representation Let x = x w–1x w–2…x O be a w-bit computer word.The unsigned integer value stored in x is Z Y 7 M / ZM MY For example, the

MIT OpenCourseWare https://ocw.mit.edu

6.172 Performance Engineering of Software Systems Fall 2018

For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.

70