visual reverse engineering of binary and data files · slashdot.org .txt constrained pairs...
TRANSCRIPT
![Page 1: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/1.jpg)
Visual Reverse Engineering of Binary and Data Files
Gregory Conti Erik Dean
Matthew Sinda Benjamin Sangster
United States Military Academy
West Point, New York
![Page 2: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/2.jpg)
file
data
exe
doc xls txt…
operated on by applications
executed by OS
core dump pagefile.sys hiberfil.sys…
ELF PE... 01010
10101 01010
memory
network
other special cases
process memory cache…
packets…
![Page 3: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/3.jpg)
Framework
• File Independent Level – Entropy – Byte Frequency – N-Gram Analysis – Strings – Hex / Decimal / ASCII – Bit Plot (2D/3D) – File Statistics
• File Specific Level – Complete or Partial Knowledge of File
Structure – For Example, Metadata
![Page 4: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/4.jpg)
Framework
• File Independent Level – Entropy – Byte Frequency – N-Gram Analysis – Strings – Hex / Decimal / ASCII – Bit Plot (2D/3D) – File Statistics
• File Specific Level – Complete or Partial Knowledge of File
Structure – For Example, Metadata
![Page 5: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/5.jpg)
Framework
• File Independent Level – Entropy – Byte Frequency – N-Gram Analysis – Strings – Hex / Decimal / ASCII – Bit Plot (2D/3D) – File Statistics
• File Specific Level – Complete or Partial Knowledge of File
Structure – For Example, Metadata
![Page 6: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/6.jpg)
Framework
• File Independent Level – Entropy – Byte Frequency – N-Gram Analysis – Strings – Hex / Decimal / ASCII – Bit Plot (2D/3D) – File Statistics
• File Specific Level – Complete or Partial Knowledge of File
Structure – For Example, Metadata
![Page 7: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/7.jpg)
Hex Editor Core
Textual Hex/ASCII Detail View
Traditional Textual Utilities
(strings...)
Graphical Displays
Machine Assisted Mapping and Navigation
![Page 8: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/8.jpg)
Hex Editor Core
Textual Hex/ASCII Detail View
Traditional Textual Utilities
(strings...)
Graphical Displays
Machine Assisted Mapping and Navigation
![Page 9: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/9.jpg)
Hex Editor Core
Textual Hex/ASCII Detail View
Traditional Textual Utilities
(strings...)
Graphical Displays
Machine Assisted Mapping and Navigation
![Page 10: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/10.jpg)
Hex Editor Core
Textual Hex/ASCII Detail View
Traditional Textual Utilities
(strings...)
Graphical Displays
Machine Assisted Mapping and Navigation
![Page 11: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/11.jpg)
Towards a Visual Hex Editor • Malware Analysis • Locate Embedded Objects
– Encoding / Encryption • Audit Files for Vulnerabilities • Compare files (Diffing) • Cracking • Analyze Unknown/Undocumented File
Format • Cryptanalysis • Perform Forensic Analysis • File System Analysis • Reporting • File Fuzzing
![Page 12: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/12.jpg)
• Textual: Text/ASCII, Strings, ByteCloud
• Graphical: Bitplot, BytePlot, RGBPlot, BytePresence, ByteFrequency, Digram, Dotplot
• Interaction: VCR, Memory Map, Color Coding
System Overview
![Page 13: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/13.jpg)
Digraph View
black hat bl (98,108) la (108,97) ac (97,99) ck (99,107) k_ (107,32) _h (32,104) ha (104,97) at (97,116)
![Page 14: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/14.jpg)
Digraph View
0,1, ... 255
Byte 0 Byte 1
...
Byte 255
98,108
32,108
![Page 15: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/15.jpg)
uuencoded
constrained pairs slashdot.org .txt
compression encryption
incrementing words
![Page 16: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/16.jpg)
Byte Plot
1 640
1
480
255 108 0 40 ...
![Page 17: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/17.jpg)
RGB Plot
1 640
1
480
255 108 0 40 128 255 0 0 0 200 0 0
![Page 18: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/18.jpg)
Byte Presence
0 255
![Page 19: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/19.jpg)
Display Comparison
Pixels/Byte 19” Monitor Gain
Textual Hex
300 pixels/byte
4.4 KB
N/A
Byte View
1 pixel/byte
1.3 MB
300x
RGB View
3 bytes/pixel
3.9 MB
900x
![Page 20: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/20.jpg)
Encryption
![Page 21: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/21.jpg)
unencrypted
XOR
![Page 22: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/22.jpg)
unencrypted
AES
![Page 23: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/23.jpg)
Fixed Length Structure
![Page 24: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/24.jpg)
Neverwinter Nights Database File
![Page 25: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/25.jpg)
Variable Length Structure
![Page 26: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/26.jpg)
Thumbs.db
See http://www.acquisitiondata.com/white_papers/thumbsdbfiles.pdf for a well written white paper.
![Page 27: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/27.jpg)
Demo (Firefox hdmp)
![Page 28: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/28.jpg)
Firefox .hdmp
![Page 29: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/29.jpg)
Firefox .hdmp
![Page 30: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/30.jpg)
Firefox .hdmp
![Page 31: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/31.jpg)
Firefox .hdmp
![Page 32: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/32.jpg)
Redacted PDF...
http://entertainment.slashdot.org/article.pl?sid=08/05/20/0228229
![Page 33: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/33.jpg)
Weaknesses
• entire file may be extracted from bit/byte/RGB – May trigger AV or IDS – 8bit/byte steg
• Screams for big monitor • Better memory management
– ~300MB+ • Unicode
![Page 34: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/34.jpg)
Future Work
• Plug-ins / Editable Config Files – Visualizations – Encodings
• Saving state – Memory Maps
• Improving Interaction – What works / What doesn’t
• Multiple Files / File Systems • REGEX search • Automated Memory Map Generation
![Page 35: Visual Reverse Engineering of Binary and Data Files · slashdot.org .txt constrained pairs compression encryption incrementing words . Byte Plot 1 640 1 480 255 108 0 40 ... RGB Plot](https://reader033.vdocument.in/reader033/viewer/2022050315/5f773aaaccfa2129ee4a65e5/html5/thumbnails/35.jpg)
Acknowledgements
Damon Becknell, Jon Bentley, Jean Blair, Sergey Bratus, Chris Compton, Tom Cross, Ron Dodge, Carrie Gates, Chris Gates, Joe Grand, Julian Grizzard, Toby Kohlenberg, Oleg Kolesnikov, Frank Mabry, Raffy Marty, Brent Nolan, Gene Ressler, Ben Sangster, Dino Schweitzer, Matt Sinda, and Ed Sobiesk