lecture #8-9: linux advanced
TRANSCRIPT
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 1/25
High performance computing
for non-programmers
Lecture #8:Linux internals
Glib Ivashkevych
junior researcher, NSC KIPT
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 2/25
Linux boot
bird’s-eye view1 → BIOS (UEFI)
2 → GRUB1 & GRUB23 → Kernel image
4 →
init/systemd
5 → Functional system
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 3/25
Linux boot
BIOS stage
1→ power up, CPU resetone CPU becomes bootstrap, other - “application” processors
2→ CPU in real mode, POSTCPU behaves like Intel 8086, HW is tested and initialized (kinda)
3→ Search for boot device, MBRMBR - first sector (512 bytes) of boot drive, stage-1 BL + partition table.
To get MBR: # dd if=/dev/sda of=/tmp/mbr.bkp bs=512 count=1
bootstrap code
446 bytes
partition table
72 bytes 5
5 A A
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 4/25
Linux boot
GRUB1→ GRUB stage 1 BL from MBR
can do little (since it’s about 446 bytes)
2→ GRUB stage 1.5knows filesystems, can access by normal path
3→ GRUB stage 2presents menu, loads kernel image, maneuvers in unreal mode,since kernel image can be larger than 640K
Config files:/boot - they are here. /boot/grub2/grub.cfg is a main config file.
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 5/25
Linux boot
Kernel1→ Kernel image is loaded
from /boot/vmlinuz-*. bootstrap kernel code is <1Mb, CPU in real mode
2→ Kernel bootstrap codeinitializations and preparations to transition to protected mode
GDT & IDT are set here, A20 line is opened by kernel
image is decompressed, paging is set up and
3→ Kernel kernel is now PID 0, setups remaining CPUs, hardware is fully functional
kernel launches init (tries /sbin/init, /etc/init, /bin/init, /bin/sh)kernel thread waits for requests
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 6/25
Linux boot
init (systemd)1→ systemd reads config
/usr/lib/systemd/system/default.target
2→ systemd starts launching unitsunit can be: target, service, socket, etc.
unit config file specifies requirements and dependenciesif no unit config file - use file in /etc/init.d instead
systemd becomes a parent of any processwhich lost its direct parent (and what the hell does that
mean?! believe it or not, any process in Linux has a parent. except PID 0 and
PID 1).
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 7/25
Linux boot
processInstance of running program
Has ID called PID and parent
Can have command line args and has
access to environment
Has current working directory
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 8/25
Linux
birth of a processfork/exec pair of system calls
forkis called from parent processreturns twice (0 in child, PID of created process in parent)
exec
loads new program to memory
typically, never returns
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 9/25
Linux
process: CL args and env
argv[0] is a name of a program by convention
Depending on actual form of exec environment can be passed from parent ornot. If so, it is passed as a copy
Process can change its environment variables
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 10/25
Linux
process: memory layoutProcess has its own address space, which islinear and flat
Memory is virtual. Logical addresses inprogram do not correspond directly tophysical addresses
Memory is allocated in pages (4KB)
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 11/25
Linux
process: memory layout
text
initialized data
uninitialized data
heap
stack
0x00000000
text: program code, literals, constants
initialized: explicitly initialized data
uninitialized (bss): obvious, mainly
stack: per function call, contains local
variables, return values and where toreturn. One stack per thread
heap: dynamically allocated memory
args, env
v i r
t u a l a d d r e s s
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 12/25
Linux
process: memory paging
page 0 v i r t
u a l a d d r e s s
page 1
page 2
page 3
page 4
page 5
page 6
page frame 3
page frame 1
page frame 2
page frame 0
page frame 6
page frame 4
page frame 5
frame 0
frame 1
frame 2
frame 3
frame 4
frame 5
frame 6
page table
R A M
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 13/25
Linux
process infoVirtual filesystem /proc
/proc/<PID>/ - directory per process
/proc/<PID>/status - various info about process/proc/<PID>/cmdline
/proc/<PID>/cwd
/proc/<PID>/environ
/proc/<PID>/maps, /proc/<PID>/smaps
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 14/25
Linux
process executionCooperative vs. preemptive
multitasking
schedulerinterrupts
system callssignals
mechanismsof control
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 15/25
Linux
process schedulingCPU-bound processes
spend most of the time executing code
high throughput
vs.
I/O bound processesspend most the time waiting for I/O
low latency
conflict
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 16/25
Linux
process schedulingLinux Completely Fair Scheduler
Tries to emulate ideal multiprocessing
CPU time is divided into epochs
Process has nice value, which weight theirtimeslice
Processes with lowest CPU time are
scheduled
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 17/25
Linux
interruptsInterrupt: signal, that breaks current
CPU activity
Async: generated by hardware
Sync: generated by CPU itself(exceptionally conditions, etc.)
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 18/25
Linux
interruptsKernel should handle interrupts
peripheral device
interrupt
controller
CPUstop executionload kernel routine called ISR
(interrupt service routine)
critical actions (otherinterrupts disabled)
non-critical
deferred
kernel decides what to do:
schedule process, return topreviously running
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 19/25
Linux
system callsA way to utilize kernel services from
user-space (say, open file or fork)
Used via C standard library
Kernel enters the scene and performstask in kernel space
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 20/25
Linux
process deathProcess died→ Zombie
Zombie = entry in process table with exit code
Zombies retain little memory, but consumePIDs
Parent should use wait system call, or exit, sothat zombie child can be inherited by init
process
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 21/25
Linux
processes: tools to managetop
useful keys: -p <PID> -n -b -u <UID>
shortcuts:P (sort by CPU%)M (sort by memory%)k (send signal to process)c, z
iotop to check disk usageuseful keys: -o -k -b -t
shortcuts:← → r (choose sorting, reverse)
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 22/25
Linux
processes: tools to managepstree
display processes tree
useful keys: -p -s
ps
takes info from /proc
many different options
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 23/25
Linux
virtual filesystemSkipped: learn it yourself
Consider this:
Ch. 13 in Linux Kernel Development by Robert Love
or
Ch. 2 in How Linux Works by Brian Ward
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 24/25
Linux
The test of the machine is the satisfaction it gives you.There isn't any other test. If the machine producestranquility it's right. If it disturbs you it's wrong until either
the machine or your mind is changed.
Robert Pirsig,
in Zen and the Art of Motorcycle Maintenance
7/27/2019 Lecture #8-9: Linux advanced
http://slidepdf.com/reader/full/lecture-8-9-linux-advanced 25/25
Questions?