compilation and execution
TRANSCRIPT
Compiling Flow
• What happened from compiling the source code to executing
Source Code(.c)
Header Files(.h)
Preprocessingcpp
Preprocessed(.i)
Compilationgcc
Static Library(.a)
Object File(.o)
Linking(ld)
Assembleas
Executablea.out
Assembly(.s)
Preproccessing
• Preproccessing
– Extend and remove #define
– #if, #ifdef, #elif, …..
$gcc -E hello.c -o hello.i
Assembly
• Assemble assembly into machine code
• After assembly, we will have the ELF format file
$gcc -c hello.s -o hello.o
ELF Format
• Executable file format – Derived from COFF(Common Object File Format)
• Windows : PE (Portable Executable) • Linux: ELF (Executable Linkable Format)
– Dynamic Linking Library (DLL) • Windows (.dll); Linux (.so)
– Static Linking Library • Windows (.lib); Linux (.a)
– Object file • Windows (.obj); Linux (.o) • Like executable file format• Intermediate file between compilation and linking
File Content
• Machine code, data, symbol table, string table
• File header
– Basic file information
• File divided by sections
– Code Section (.code, .text)
– Data Section (.data)
– Special Section (.symtab, .strtab)
File Header
• File header contains following information
– Is executable
– Static Link or Dynamic Link
– Entry address
– Target hardware / OS
– Section Table
File Header Structure
• The structure of ELF header is defined as Elf_Ehdr
• e_ident
– The first 4 byte is‘\x7f’, ‘E’,’L’,’F’
– File signature
• e_type
typedef struct{
unsigned char e_ident[16]; /* ELF identification */Elf64_Half e_type; /* Object file type */Elf64_Half e_machine; /* Machine type */Elf64_Word e_version; /* Object file version */Elf64_Addr e_entry; /* Entry point address */Elf64_Off e_phoff; /* Program header offset */Elf64_Off e_shoff; /* Section header offset */Elf64_Word e_flags; /* Processor-specific flags */Elf64_Half e_ehsize; /* ELF header size */Elf64_Half e_phentsize; /* Size of program header entry */Elf64_Half e_phnum; /* Number of program header entries */Elf64_Half e_shentsize; /* Size of section header entry */Elf64_Half e_shnum; /* Number of section header entries */Elf64_Half e_shstrndx; /* Section name string table index */
} Elf64_Ehdr;
File Header Structure
• e_machine
– 62 for AMD x86-64 architecture
– 243 for RISC-V (HITCON 2015)
• e_entry
• e_shoff
– Follow this membercan find the sectiontable
typedef struct{
unsigned char e_ident[16]; /* ELF identification */Elf64_Half e_type; /* Object file type */Elf64_Half e_machine; /* Machine type */Elf64_Word e_version; /* Object file version */Elf64_Addr e_entry; /* Entry point address */Elf64_Off e_phoff; /* Program header offset */Elf64_Off e_shoff; /* Section header offset */Elf64_Word e_flags; /* Processor-specific flags */Elf64_Half e_ehsize; /* ELF header size */Elf64_Half e_phentsize; /* Size of program header entry */Elf64_Half e_phnum; /* Number of program header entries */Elf64_Half e_shentsize; /* Size of section header entry */Elf64_Half e_shnum; /* Number of section header entries */Elf64_Half e_shstrndx; /* Section name string table index */
} Elf64_Ehdr;
Sections Table
• Sections Table storethe array of section headers
– Each section is in type Elf64_Shdr
– Every element 64 bytes typedef struct{Elf64_Word sh_name; /* Section name */Elf64_Word sh_type; /* Section type */Elf64_Xword sh_flags; /* Section attributes */Elf64_Addr sh_addr; /* Virtual address in memory */Elf64_Off sh_offset; /* Offset in file */Elf64_Xword sh_size; /* Size of section */Elf64_Word sh_link; /* Link to other section */Elf64_Word sh_info; /* Miscellaneous information */Elf64_Xword sh_addralign; /* Address alignment boundary */Elf64_Xword sh_entsize; /* Size of entries, if section has table */} Elf64_Shdr;
e_shoff
.sh.strtab
.sh.strtab + 0x20
Code Section
• Code section, most case .text, is used to save the binary code
• objdump –s
– Display the full contents of all sections
• objdump –d
– Display assembler contents of executable sections
Data Section
• There are several sections to store program’s data
– .data → Initialized global variable & static variable
– .rodata → save the constant value in the program
– .bss → save the uninitialized variables
Bss section
• BSS section is used to save uninitialized data or data filled with zero
• This section will not occupy space in the ELF file
– But have space when loading into memory
Other
• .rela.text, .symtab
– Used in symbol resolving
• .strtab
– Save strings for symbol resolution
Static Linking
• Static link is responsible for combining several object files into final executable
$gcc hello.o -o hello.out
Two-pass Linking
• Two-pass Linking
• Space & Address Allocation– Fetch section length, attribute and position
– Collect symbol(define, reference) and put to a global table
• Symbol Resolution & Relocation– Modify relocation entry
Space & Address Allocation
• Define Symbols
– variables
– Functions
• The virtual address is allocated after linking
Symbol Table before Linking Symbol Table after Linking
Symbol Table
typedef struct elf64_sym {Elf64_Word st_name; /* Symbol name, index in string tbl */unsigned char st_info; /* Type and binding attributes */unsigned char st_other; /* No defined meaning, 0 */Elf64_Half st_shndx; /* Associated section index */Elf64_Addr st_value; /* Value of the symbol */Elf64_Xword st_size; /* Associated symbol size */
} Elf64_Sym;
Size: 24 bytes
Fist symbol
.strtab section at 0x000005700x570+0xe(offset) = 0x57e
This symbol is named shared
Symbol Resolution & Relocation
• Resolve symbol’s address in the final executable
– Address of external symbols are unknown before linking
– Before linking, the temporary location is put into object files
– Automatic patch the address with the correct one
Relocation Table
• Relocation table records the address of symbol to
patch
typedef struct elf64_rela {Elf64_Addr r_offset;Elf64_Xword r_info;Elf64_Sxword r_addend;
} Elf64_Rela;
r_offset r_info[1] r_info[2] r_addend
shared 14 00 00 00 00 00 00 00 0a 00 00 00 09 00 00 00 00 00 00 00 00 00 00 0
swap 21 00 00 00 00 00 00 00 02 00 00 00 0a 00 00 00 fc ff ff ff ff ff ff ff
0x0a -> R_X86_64_32/R_AMD64_320X02 -> R_X86_64_PC32/R_AMD64_PC32They have different way to patch address
Relocation Type and Patch Calculation
•
A - The addend value of the relocatable field.S - The value of the symbol P - The section offset or address of the storage unit being relocated, computed using r_offset.GOT - The address of the global offset tablehttps://docs.oracle.com/cd/E23824_01/html/819-0690/chapter6-54839.html
Program Execution
• After static linking, we have the executable file
• The loader is most important to make the program run
– Loading the executable into memory
– Dynamic resolving the symbols
Creation of Process
• Create a independent virtual AS – page directory(Linux)
• Read executable file header, create mapping between virtual AS and executable file – VMA, Virtual Memory Area
• Assign entry address to program register(PC)– Switch between kernel stack
and process stack
– CPU access attribute
Section to Segment Mapping
• Several sections are merge into the segment
– Depend on it’s permission
• Read
• Write
• Execution
Load Executable into Mem
• The program header section contains the information of segments
– Program load into memory in the unit of segments
– Readelf
– Program header tablein ELF file
Disadvantage of Static Linking
• Advantage
– Independent development
– Test individual modules
• Disadvantage
– Waste memory and disk space
• Every program has a copy of runtime library(printf, scanf, strlen, ...)
• Difficulty of updating module – Need to re-link and publish to user when a module is updated
Dynamic Linking
• Delay linking until execution – Load Time Relocation
• Example: – Program1.o, Program2.o, Lib.o– Execute Program1 → Load Program1.o – Program1 uses Lib → Load Lib.o– Execute Program2 → Load Program2.o – Program2 uses Lib → Lib.o has already been loaded into
physical memory – Advantage
• Save space • Easier to update modules
Lazy Binding
• Bind when the first time use the function(relocation, symbol searching)
• More efficient
– In most executions, only small amount of function called
– Not need to resolve all the symbols
Global Offset Table
• Divide into 2 part
– .got
• Store the reference address of global variables
– .got.plt
• Store the reference address of global functions
.GOT.PLT
• The first 3 items are fixed and have special purpose– address of .dynamic section
– link_map
– dl_runtime_resolve
• The following items are functions in the share libraries
.dynamic
.got
.got.plt
.data
Addr of .dynamic
link_map
dl_resolve
.
.
.call foo@plt
…
Lazy binding
.text
jmp *(bar@GOT)push indexjmp PLT0
foo@plt
.got.plt
printf
bar@plt+6
foo
…
push *(GOT + 4)jmp *(GOT + 8)
PLT0
36
Note: GOT here means .got.plt
.
.
.call foo@plt
…
.text
jmp *(bar@GOT)push indexjmp PLT0
foo@plt
.got.plt
printf
bar@plt+6
foo
…
push *(GOT + 4)jmp *(GOT + 8)
PLT0
Lazy binding
37
.
.
.call foo@plt
…
.text
jmp *(bar@GOT)
push indexjmp PLT0
foo@plt
.got.plt
printf
bar@plt+6
foo
…
push *(GOT + 4)jmp *(GOT + 8)
PLT0
因 bar還沒 call 過所以 bar在 .got.plt 中所存的值會是.plt中的下一行指令位置所以看起來會像沒有 jmp 過
Lazy binding
38
.
.
.call foo@plt
…
.text
jmp *(bar@GOT)push index
jmp PLT0
foo@plt
.got.plt
printf
bar@plt+6
foo
…
push *(GOT + 4)jmp *(GOT + 8)
PLT0
Lazy binding
39
.
.
.call foo@plt
…
.text
jmp *(bar@GOT)push indexjmp PLT0
foo@plt
.got.plt
printf
bar@plt+6
foo
…
push *(GOT + 4)jmp *(GOT + 8)
PLT0
Lazy binding
40
.
.
.call foo@plt
…
.text
jmp *(bar@GOT)push indexjmp PLT0
foo@plt
.got.plt
printf
bar@plt+6
foo
…
push *(GOT + 4)
jmp *(GOT + 8)
PLT0
push link_map
Lazy binding
41
.
.
.call foo@plt
…
.text
jmp *(bar@GOT)push indexjmp PLT0
foo@plt
.got.plt
printf
bar@plt+6
foo
…
push *(GOT + 4)jmp *(GOT + 8)
PLT0
jmp dl_runtime_resolve
dl_runtime_resolve (link_map,index)
Lazy binding
42
.
.
.call foo@plt
…
.text
.got.plt
printf
bar@plt+6
foo
…
..
..call _fix_up
..
..ret 0xc
dl_resolve
Lazy binding
43
.
.
.call foo@plt
…
.text
.got.plt
printf
foo@plt+6
bar
…
..
..call _fix_up
..
..ret 0xc
dl_resolve
foo 找到 bar 在 library 的位置後會填回 .got.plt
Lazy binding
44
.
.
.call foo@plt
…
Lazy binding
.text
.got.plt
printf
foo@plt+6
foo
…
..
..call _fix_up
..
..ret 0xc
dl_resolve
barreturn to bar
45
Program Memory Layout
• Flat memory model
– Default regions:
• stack
• heap
• mapping of executable file
• dynamic libraries
• Stack Frame(Activate Record)
– Return address, arguments
– Temporary variables
– Context
• Frame Pointer(ebp on i386)
• Stack Pointer(esp on i386)
Calling Convention
• Consistency between caller and callee •
• Argument passing order and method
– Stack, Register(eax for return value on i386) •
• Stack maintainer
– Keep consistency before and after function call
– Responsibility of caller or callee
– Name-mangling
– Default calling convention in C language is “cdecl”
LD_Preload
• Ordinarily the dynamic linker loads shared libs in whatever order it needs them
• $LD_PRELOAD is an environment variable containing a colon (or space) separated list of libraries that the dynamic linker loads before any others
LD_Preload
• Preloading a library means that its functions will be used before others of the same name in later libraries
• Allows functions to be overridden/replaced/ intercepted
• Program behaviour can be modified “non-invasively” – ie. no recompile/relink necessary – Especially useful for closed-source programs – And when the modifications don’t belong in the
program or the library
Implement Shared Lib
• Write our own function
• Compile into share library
gcc -Wall -fpic -shared -o libmylib.so mylibc.c
System Call
• User processes cannot perform privileged operations themselves
• Must request OS to do so on their behalf by issuing system calls
• System calls elevate privilege of user process
Ltrace
• Tracing system calls in Linux – strace command
• Output is printed for each system call as it is executed, including parameters and return codes
• ptrace() system call is used to implement strace – Also used by debuggers (breakpoint, singlestep, etc)– Maybe anti-debug
– How to solve?