memory unix tables 1
TRANSCRIPT
Virtual Memory in Unix
Idan Shatz
Unix’s Method
� Ready?
The Big Picture
Main (Physical) Memory
128MB
0 Frame 0
Frame 1
Frame 24GB
A process
0 Page 0
Page 1
Page 2
Page 3
Page n
4GB
A process
0 Page 0
Page 1
Page 2
Page 3
Page n
Frame m
Swap Device
1GB
0 block 0
block 1
block 2
block p
Page out
Page in
The Unix Big Picture
Main (Physical) Memory
128MB
0 Frame 0
Frame 1
Frame 24GB
A process
0 Page 0
Page 1
Page 2
Page 3
Page n
4GB
A process
0 Page 0
Page 1
Page 2
Page 3
Page n
Frame m
Swap Device
1GB
0 block 0
block 1
block 2
block p
Page out
Page in
RegionTable
RegionTable Page
Table
PageTable
PageTable
PageTable
RegionTable
RegionTable Page
Table
PageTable
PageTable
PageTable
PFdataTable
PFdataTable
Swap-useTable
Swap-useTable
D.B.D.Table
D.B.D.Table
D.B.D.Table
D.B.D.Table
D.B.D.Table
D.B.D.Table
D.B.D.Table
D.B.D.Table
Tables
� Region table � Text, Data, Stack (per process)
� Page table� Virtual � Physical translation (per process)
� Disk Block Descriptor (DBD) table � Disk location of swapped pages (per process)
� PFData (Page Frame) table� Description of physical frame status (singleton)
� Swap in-use table� The same as PFData, but for the swap file (singleton)
Process Memory
0
pro
cess a
ddre
ss s
pace
Text
Data
Stack
static allocation
dynamic allocation
4GB
Region Table
0
pro
cess a
ddre
ss s
pace
Text
Data
Stack
static allocation
dynamic allocation
4GB
Text
Data
Stack
StartAddress
RegionSize
Permissions
Region table
Pages and Frames
0
pro
cess a
ddre
ss s
pace
Text
Data
Stack
static allocation
dynamic allocation
4GB
Main (Physical) Memory
16MB
0
Frame (4k)
Page
Virtual Memory
Page Table
Text
Data
Stack
StartAddress
RegionSize
Permissions
Page 0
Page 1
Page 2
…
Page N-2
Page N-1
PhysicalAddress
ValidFlag
Permissions Ref Mod COW Age
page table
region table
The Unix Big Picture
Main (Physical) Memory
128MB
0 Frame 0
Frame 1
Frame 24GB
A process
0 Page 0
Page 1
Page 2
Page 3
Page n
4GB
A process
0 Page 0
Page 1
Page 2
Page 3
Page n
Frame m
Swap Device
1GB
0 block 0
block 1
block 2
block p
Page out
Page in
RegionTable
RegionTable Page
Table
PageTable
PageTable
PageTable
RegionTable
RegionTable Page
Table
PageTable
PageTable
PageTable
Translation from Virtual to Physical address
Text
Data
Stack
StartAddress
RegionSize
Permissions
Page 0
Page 1
Page 2
…
Page N-2
Page N-1
PhysicalAddress
ValidFlag
Permissions Ref Mod COW Age
page table
region table
Virtual Address
Physical Address
Translation (Boooo)
Translation from Virtual to Physical address
Text
Data
Stack
StartAddress
RegionSize
Permissions
Page 0
Page 1
Page 2
…
Page N-2
Page N-1
PhysicalAddress
ValidFlag
Permissions Ref Mod COW Age
page table
region table
Virtual Address offset
Frame Number offset
=
Translation from Virtual to Physical address
Text
Data
Stack
StartAddress
RegionSize
Permissions
Page 0
Page 1
Page 2
…
Page N-2
Page N-1
PhysicalAddress
ValidFlag
Permissions Ref Mod COW Age
page table
region table
Virtual Address offset
Page Number offset
The Calculation:(Virtual – Start)/Page Size
Translation from Virtual to Physical address
Text
Data
Stack
StartAddress
RegionSize
Permissions
Page 0
Page 1
Page 2
…
Page N-2
Page N-1
PhysicalAddress
ValidFlag
Permissions Ref Mod COW Age
page table
region table
Virtual Address offset
Page Number offset
The Unix Big Picture
Main (Physical) Memory
128MB
0 Frame 0
Frame 1
Frame 24GB
A process
0 Page 0
Page 1
Page 2
Page 3
Page n
4GB
A process
0 Page 0
Page 1
Page 2
Page 3
Page n
Frame m
Swap Device
1GB
0 block 0
block 1
block 2
block p
Page out
Page in
RegionTable
RegionTable Page
Table
PageTable
PageTable
PageTable
RegionTable
RegionTable Page
Table
PageTable
PageTable
PageTable
PFdataTable
PFdataTable
Physical Memory & Pfdata table
Main (Physical) Memory
16MB
0
Frame (4k)
Pfdata table
SwapDevice
RefCount
BlockNumber
Swap/File
Status
The Unix Big Picture
Main (Physical) Memory
128MB
0 Frame 0
Frame 1
Frame 24GB
A process
0 Page 0
Page 1
Page 2
Page 3
Page n
4GB
A process
0 Page 0
Page 1
Page 2
Page 3
Page n
Frame m
Swap Device
1GB
0 block 0
block 1
block 2
block p
Page out
Page in
RegionTable
RegionTable Page
Table
PageTable
PageTable
PageTable
RegionTable
RegionTable Page
Table
PageTable
PageTable
PageTable
PFdataTable
PFdataTable
Swap-useTable
Swap-useTable
Swap Device & swap use
Swap device
1GB
0
Block (4k)
Swap use table
Ref Count
The Unix Big Picture
Main (Physical) Memory
128MB
0 Frame 0
Frame 1
Frame 24GB
A process
0 Page 0
Page 1
Page 2
Page 3
Page n
4GB
A process
0 Page 0
Page 1
Page 2
Page 3
Page n
Frame m
Swap Device
1GB
0 block 0
block 1
block 2
block p
Page out
Page in
RegionTable
RegionTable Page
Table
PageTable
PageTable
PageTable
RegionTable
RegionTable Page
Table
PageTable
PageTable
PageTable
PFdataTable
PFdataTable
Swap-useTable
Swap-useTable
D.B.D.Table
D.B.D.Table
D.B.D.Table
D.B.D.Table
D.B.D.Table
D.B.D.Table
D.B.D.Table
D.B.D.Table
Virtual & Swap connection
Text
Data
Stack
StartAddress
RegionSize
Permissions
Page 0
Page 1
Page 2
…
Page N-2
Page N-1
PhysicalAddress
ValidFlag
Permissions Ref Mod COW Age
page table
region table
Page 0
Page 1
Page 2
…
Page N-2
Page N-1
Disk block descriptor
table
SwapDevice
Fill 0/Demand fill
BlockNumber
Swap/File
Example
Swap Dev. 1
Block No. 2743Page No. 7941493K
Disk Block DescriptorPage Table EntryVirtual Address
Block No. 2743
Swap Dev. 1
Ref Cnt 1
Page Frame 794
Red Cnt 1
Swap Use
Swap Dev. Block 2743Physical Page 794
Agenda
� Page Faults
� Ref, Mod, COW and Age
� Page In/Out Method
� Pager Process
� Page-Stealer Process
� System Calls
� Exec
� Fork
Page Faults
� 3 Types of page Faults
� Page Faults
� Protection Fault
� Segmentation Fault
Page Faults
� 3 Types of page Faults
� Page Faults
� Happens when valid=0.
� Protection Fault
� Happens on permission violation
� Segmentation Fault
� Happens when virtual address is illegal.
Page Faults
� 3 Types of page Faults
� Page Faults
� Happens when valid=0.
� Causes pages in only (How?)
� Protection Fault
� Happens on permission violation
� Good Example: Writing to a read-only page
� Segmentation Fault
� Happens when virtual address is illegal.
Ref & Mod
Page 0
Page 1
Page 2
…
Page N-2
Page N-1
PhysicalAddress
ValidFlag
Permissions Ref Mod COW Age
page table
Updated by CPU:• Ref – set to 1 when the processaccesses a page.
• Mod – set to 1 when the processmodifies a page
COW & Age
Page 0
Page 1
Page 2
…
Page N-2
Page N-1
PhysicalAddress
ValidFlag
Permissions Ref Mod COW Age
page table
The other two, updated by O.S.• COW – Copy on Write Flag
The page will be copiedon first write (Fork)
• Age – Page Age (a number)=0 – to warm pages>0 – to cold pages
The Unix Big Picture
Main (Physical) Memory
128MB
0 Frame 0
Frame 1
Frame 24GB
A process
0 Page 0
Page 1
Page 2
Page 3
Page n
4GB
A process
0 Page 0
Page 1
Page 2
Page 3
Page n
Frame m
Swap Device
1GB
0 block 0
block 1
block 2
block p
Page out
Page in
RegionTable
RegionTable Page
Table
PageTable
PageTable
PageTable
RegionTable
RegionTable Page
Table
PageTable
PageTable
PageTable
PFdataTable
PFdataTable
Swap-useTable
Swap-useTable
D.B.D.Table
D.B.D.Table
D.B.D.Table
D.B.D.Table
D.B.D.Table
D.B.D.Table
D.B.D.Table
D.B.D.Table
PagerPager
Page-stealerPage-stealer
Pager
� Loads Pages into main memory.
� Usually, the Pager process is asleep.
� Wakes up on a Page Fault (Type 1).
Pager
� The Page In process:
1. Look at the D.B.D table to find the page at the swap.
2. Look at the Pfdata table to check if the page is found at the main memory.
a. Skip to 4
3. Load the page from swap to main memory
4. Update tables.
Page 0
Page 1
Page 2
…
Page N-2
Page N-1
Disk block descriptor
table
When Valid=0
PhysicalAddress
ValidFlag
. . . .
Page 0
Page 1
Page 2
…
Page N-2
Page N-1
page tableText
Data
Stack
region table
Virtual Address offset
SwapDevice
Fill 0/Demand fill
BlockNumber
Swap/File
Find the swap block number from the disk block descriptor table
When Valid=0
Main (Physical) Memory
16MB
0
Frame (4k)
Pfdata table
SwapDevice
RefCount
BlockNumber
Swap/File
Status
Before reading the block from the swap,Unix tries to find the page in the main memory.Pfdata has a hash for speeding up the search process
Page Stealer
� The Bad Guy �
� Wakes up when the number of free frames passes the low-water mark of free frames
� Start to free frames till the high-water mark
High
Low
Which page to throw?
� Remember the Locality Rule?
� Throw cold pages
� The Aging process
� Happens every second
� Update Age field
� If (REF = =0) Age ++;
� Else Age=0;
� Turns off REF field
Exec
� Create a new virtual memory
� New Region table
� New Page Tables & D.B.D
Fork
� Duplicate the virtual memory� Text – can be shared because it is read-only
� Data & Stack – need to be duplicated
� The Problem:� Forking in order to do exec…
� Duplicate on Demand� Share while you can…� Make a copy only when needed!
Copy on Demand
� A page at the data region can be shared until one of the processes modifies it.
� Therefore, the O.S. needs to trace the first write to the page
� Make the page Read-only.
� Turn on the COW flag.
� When we have Protection Fault
� Check if the COW is on, if so, make a duplicate.
Fork
� Text Region� Share the page table
� Data & Stack� Copy the page table.
� To all valid pages: Read-only & COW
� Protection Fault� If COW is on
� copy the page
� Turn off COW.
� Change back to Read-Write permissions.