forking, w/ unified vm wrapup
DESCRIPTION
Forking, w/ Unified VM Wrapup. Vivek Pai / Kai Li Princeton University. Gedankenverification. Why is perfect LRU reasonable for the filesystem disk cache but not for the VM pages? How does the filesystem know what pages it has? - PowerPoint PPT PresentationTRANSCRIPT
Forking, w/ Unified VM Wrapup
Vivek Pai / Kai LiPrinceton University
2
GedankenverificationWhy is perfect LRU reasonable for the filesystem disk cache but not for the VM pages?How does the filesystem know what pages it has?What are the steps involved prior to a page being demand loaded in a VM system?What are the possible results of a TLB miss?
3
MechanicsLast quiz gradedMidterm on Thursday
In classClosed everything (no book, no notes, etc)Apply Occam’s razor when answering questionsSome short answer, some quiz-like questions, maybe some multiple choice or true/false
Don’t panic
4
Fear Is The Mind-killer“I must not fear.
Fear is the mind-killerFear is the little death that brings total obliterationI will face my fearI will permit it to pass over me and through meAnd when it has gone past I will turn the inner eye to see its pathWhere the fear has gone there will be nothingOnly I will remain” – Dune, Frank Herbert, specifically from the Bene Gesserit Litany Against Fear
5
The Big PictureWe’ve talked about single evictionsMost computers are multiprogrammed
Single eviction decision still neededNew concern – allocating resourcesHow to be “fair enough” and achieve good overall throughput
This is a competitive world – local and global resource allocation decisions
6
Imagine a Global LRUGlobal – across all processesIdea – when a page is needed, pick the oldest page in the systemProblems? Process mixes?
Interactive processesActive large-memory sweep processes
Mitigating damage?
7
Source of Disk AccessVM System
Main memory caches - full image on disk
FilesystemEven here, caching very useful
New competitive pressure/decisionsHow do we allocate memory to these two?How do we know we’re right?
8
Partitioning MemoryOriginally, specified by administrator
20% used as filesystem cache by defaultOn fileservers, admin would set to 80%Each subsystem owned pages, replaced them
Observation: they’re all basically pagesWhy not let them compete?Result: unified memory systems – file/VM
9
File Access Efficiencyread(fd, buf, size)
Buffer in process’s memoryData exists in two places – filesystem cache & process’s memoryKnown as “double buffering”
Various scenariosMany processes read same fileProcess wants only parts of a file, but doesn’t know which parts in advance
10
Result: Memory-Mapped Files
File
Process A
File
Process B
File
Process C
Map
Process A
Map
Process B
Map
Process C
File
11
Lazy Versus EagerEager: do things right away
read(fd, buf, size) – returns # bytes readBytes must be read before read completesWhat happens if size is big?
Lazy: do them as they’re neededmmap(…) – returns pointer to mappingMapping must exist before mmap completesWhen/how are bytes read?What happens if size is big?
12
Semantics: How Things Behave
What happens whenTwo process obtain data (read or mmap)One process modifies data
Two processes obtain data (read or mmap)A third process modifies dataThe two processes access the data
13
Being Too Smart…Assume a unified VM/File schemeYou’ve implemented perfect Global LRU
What happens on a filesystem “dump”?
14
Amdahl’s LawGene Amdahl (IBM, then Amdahl)Noticed the bottlenecks to speedupAssume speedup affects one componentNew time = (1-not affected) + affected/speedupIn other words, diminishing returns
15
NT x86 Virtual Address Space Layouts00000000
7FFFFFFF80000000
System cachePaged pool
Nonpaged pool
Kernel & execHAL
Boot drivers
Process page tablesHyperspace
Application codeGlobals
Per-thread stacksDLL code
3-GB user space
1-GB system space
BFFFFFFFC0000000
FFFFFFFF FFFFFFFF
C0000000
C0800000
16
Virtual Address Space in Win95 and Win9800000000
7FFFFFFF80000000
Operating system(Ring 0 components)
Shared, process-writable(DLLs, shared memory,
Win16 applications)
Win95 and Win98
User accessible
FFFFFFFF
C0000000
Unique per process(per application),user mode
Systemwideuser mode
Systemwidekernel mode
17
Details with VM ManagementCreate a process’s virtual address space
Allocate page table entries (reserve in NT)Allocate backing store space (commit in NT)Put related info into PCB
Destroy a virtual address spaceDeallocate all disk pages (decommit in NT)Deallocate all page table entries (release in NT)Deallocate all page frames
18
More Lazy Versus Eager IssuesAssume 1GB of swap spaceAssume 6 processes each dobuf = malloc(1024*1024*1024);
Should these operations proceed?What if they memset(buf, 0, 1024*1024)?What if they memset(buf, 0, 1024*1024*1024)?This happened in reality: IBM’s AIX OS
19
Page States (NT)
Active: Part of a working set and a PTE points to itTransition: I/O in progress (not in any working sets)Standby: Was in a working set, but removed.
A PTE points to it, not modified and invalid.Modified: Was in a working set, but removed.
A PTE points to it, modified and invalid.Modified no write: Same as modified but no write backFree: Free with non-zero contentZeroed: Free with zero contentBad: hardware errors
20
Working setreplacement
Page in or allocationDemandzero fault
Dynamics in NT VM
Processworking
set
Standbylist
Modifiedlist
Modifiedwriter
“Soft”faults
Freelist
Zerothread
Zerolist
Badlist
21
How To Launch a New Process?Obvious choice: “start process” system callBut not all processes start the same
“testprogram” versus “testprogram >& outfile” versus “testprogram arg1 arg2 >& outfile”
The “parent” process wants to specify various aspects of the child’s “environment”
Next step: add more parameters to specify environment
22
Can We Generalize?What happens as more information gets added to the process’s “environment” – more parameters? New system calls? This gets uglyWhat’s the most general way of setting up all of the environment?So, why not allow process setup at any point?
This is the exec( ) system call (and its variants)
23
But We Want a Parent and a ChildThe exec call “destroys” the current processSo, instead, destroy a copy of the process
The fork( ) call duplicates the current processBetter yet, don’t tightly couple fork and exec
This way, you can customize the child’s environment
So what does fork( ) entail?Making a copy of everything about the processOuch!
24
What Gets CopiedSo far, we’ve covered the following:
VM systemFile systemSignals
How do we go about copying this information?What parts are easy to copy, and what’s hard?What’s the common case with fork/exec?
What needs to get preserved in this scenario?
25
Shared MemoryHow to destroy a virtual address space?
Link all PTEsReference count
How to swap out/in?Link all PTEsOperation on all entries
How to pin/unpin?Link all PTEsReference count
.
.
.
.
.
.
...
.
.
.
Process 1
Process 2
w
...
w
Page table
Page table
Physicalpages
26
.
.
.
.
.
.
...
.
.
.
Copy-On-WriteChild’s virtual address space uses the same page mapping as parent’sMake all pages read-onlyMake child process readyOn a read, nothing happensOn a write, generates an access fault
map to a new page framecopy the page overrestart the instruction
Parent process
Child process
rr
...
rr
Page table
Page table
Physicalpages
27
Issues of Copy-On-WriteHow to destroy an address space
Same as shared memory case?
How to swap in/out?Same as shared memory
How to pin/unpinSame as shared memory