21scs147l17replacementpolicy4-7[1]
TRANSCRIPT
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
1/60
Cache memory Replacement
Policy, Virtual Memory
Prof. Sin-Min Lee
Department of Computer Science
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
2/60
Associative Cache
Access orderA0 B0 C2 A0 D1 B0 E4 F5 A0
C2 D1 V0 G3 C2 H7 I6 A0 B0
Tc = 10 ns
Tp = 60 ns
FIFO
h = 0.389
TM = 40.56 ns
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
3/60
FIFO
MEMORY
FIFO is similar to LRU except that because FIFO method doesn't move
the most recently used block up in the queue, it ends up discarding ablock that was used recently instead of the least recently used. In thisexample, when the memory is full A is discarded though it was recentlyused.
Hit ratio: 7/18
A is removed from memory, although
it was not the least recently used.
A B C A D B E F A C D B G C H I A B
0 A B C C D D E F F F F F G G H I A B
1 A B B C C D E E E E E F F G H I A
2 A A B B C D D D D D E E F G H I
3 A A B C C C C C D D E F G H
4 A B B B B B C C D E F G
5 A A A A A B B C D E F6 A A B C D E
7 A B C D
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
4/60
LRU
A B C A D B E F A C D B G C H I A B
0 A B C A D B E F A C D B G C H I A B
1 A B C A D B E F A C D B G C H I A
2 A B C A D B E F A C D B G C H I
3 B C A D B E F A C D B G C H
4 C A D B E F A A D B G C
5 C C D B E F F A D B G
6 E E F A D D
7 E F F F
MEMORY
MEMORY FULL
E is LRU and isremoved from memory
LRU (Least Recently Used) is a method that keeps track of how oftena data block is used.
Hit ratio: 9/18
The bottom block is the LeastRecently Used, which gets
replaced (discarded) in favor of thenew one, when the memory is full.
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
5/60
Direct-Mapped Cache
Access orderA0 B0 C2 A0 D1 B0 E4 F5 A0
C2 D1 V0 G3 C2 H7 I6 A0 B0
Tc = 10 ns
Tp = 60 ns
h = 0.167
TM = 50.67 ns
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
6/60
2-Way Set Associative Cache
Access orderA0 B0 C2 A0 D1 B0 E4 F5 A0C2 D1 V0 G3 C2 H7 I6 A0 B0
Tc = 10 ns
Tp = 60 ns
LRU
h = 0.31389
TM = 40.56 ns
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
7/60
Associative Cache
(FIFO Replacement Policy)
Data A B C A D B E F A C D B G C H I A B
C
A
C
HE
A A A A A A A A A A A A A A A I I I
B B B B B B B B B B B B B B B A A
C C C C C C C C C C C C C C C B
D D D D D D D D D D D D D D
E E E E E E E E E E E E
F F F F F F F F F F F
G G G G G G
H H H H
Hit? * * * * * * *
Hit ratio = 7/18
A0 B0 C2A0 D1 B0 E4 F5A0 C2 D1 B0 G3 C2 H7 I6A0 B0
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
8/60
Two-way set associative cache
(LRU Replacement Policy)
Hit ratio = 7/18
A0 B0 C2A0 D1 B0 E4 F5A0 C2 D1 B0 G3 C2 H7 I6A0 B0
Data A B C A D B E F A C D B G C H I A B
C
A
C
H
E
0 A-0 A-1 A-1 A-0 A-0 A-1 E-0 E-0 E-1 E-1 E-1 B-0 B-0 B-0 B-0 B-0 B-1 B-0
0 B-0 B-0 B-1 B-1 B-0 B-1 B-1 A-0 A-0 A-0 A-1 A-1 A-1 A-1 A-1 A-0 A-1
1 D-0 D-0 D-0 D-1 D-1 D-1 D-0 D-0 D-0 D-0 D-0 D-0 D-0 D-0
1 F-0 F-0 F-0 F-1 F-1 F-1 F-1 F-1 F-1 F-1 F-1
2 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-1 C-1 C-12 I-0 I-0 I-0
3 G-0 G-0 G-1 G-1 G-1 G-1
3 H-0 H-0 H-0 H-0
Hit? * * * * * * *
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
9/60
Associative Cache with 2 byte line size (FIFO
Replacement Policy)
Hit ratio = 11/18
A0 B0 C2A0 D1 B0 E4 F5A0 C2 D1 B0 G3 C2 H7 I6A0 B0
A and J; B and D; C and G; E and F; and I and H
Data A B C A D B E F A C D B G C H I A B
C
A
C
H
E
A A A A A A A A A A A A A A I I I I
J J J J J J J J J J J J J J H H H H
B B B B B B B B B B B B B B B A A
D D D D D D D D D D D D D D D J J
C C C C C C C C C C C C C C C B
G G G G G G G G G G G G G G G D
E E E E E E E E E E E E
F F F F F F F F F F F F
Hit? * * * * * * * * * * *
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
10/60
Direct-mapped Cache
with line size of 2 bytes
Hit ratio 7/18
Data A B C A D B E F A C D B G C H I A B
C
A
C
H
E
0 A B B A B B B B A A B B B B B B A B
1 J D D J D D D D J J D D D D D D J D
2 C C C C C C C C C C C C C C C C
3 G G G G G G G G G G G G G G G G
4 E E E E E E E E E E E E
5 F F F F F F F F F F F F
6 I I I I
7 H H H H
Hit? * * * * * * *
A0 B0 C2A0 D1 B0 E4 F5A0 C2 D1 B0 G3 C2 H7 I6A0 B0
A and J; B and D; C and G; E and F; and I and H
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
11/60
Two-way set Associative Cache
with line size of 2 bytes
Hit ratio = 12/18
Data A B C A D B E F A C D B G C H I A B
C
A
C
H
E
0 A-0 A-1 A-1 A-0 A-1 A-1 E-0 E-0 E-1 B-0 B-0 B-0 B-0 B-0 B-0 B-0 B-1 B-0
1 J-0 J-1 J-1 J-0 J-1 J-1 F-0 F-0 F-1 D-0 D-0 D-0 D-0 D-0 D-0 D-0 D-1 D-0
0 B-0 B-0 B-1 B-0 B-0 B-1 B-1 A-0 A-0 A-1 A-1 A-1 A-1 A-1 A-1 A-0 A-1
1 D-0 D-0 D-1 D-0 D-0 D-1 D-1 J-0 J-0 J-1 J-1 J-1 J-1 J-1 J-1 J-0 J-1
2 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-1 C-1 C-1 C-1
3 G-0 G-0 G-0 G-0 G-0 G-0 G-0 G-0 G-0 G-0 G-0 G-0 G-1 G-1 G-1 G-1
2 I-0 I-0 I-0 I-0
3 H-0 H-0 H-0 H-0
Hit? * * * * * * * * * * * *
A0 B0 C2A0 D1 B0 E4 F5A0 C2 D1 B0 G3 C2 H7 I6A0 B0
A and J; B and D; C and G; E and F; and I and H
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
12/60
Page Replacement - FIFO
FIFO is simple to implement When page in, place page id on end of list
Evict page at head of list
Might be good? Page to be evicted has been inmemory the longest time
But?
Maybe it is being used We just dont know
FIFO suffers from Beladys Anomaly fault ratemay increase when there is more physical
memory!
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
13/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
14/60
Parkinson's law : "Programs expand to fill the memory available to
hold them"
Idea : Manage the storage available efficiently between the available
programs.
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
15/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
16/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
17/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
18/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
19/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
20/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
21/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
22/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
23/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
24/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
25/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
26/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
27/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
28/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
29/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
30/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
31/60
Before VM
Programmers tried to shrink programs to fit tiny
memories
Result: Small
Inefficient Algorithms
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
32/60
Solution to Memory Constraints
Use a secondary memory such as disk
Divide disk into pieces that fit memory (RAM)
Called Virtual Memory
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
33/60
Implementations of VM
Paging
Disk broken up into regular sized pages
Segmentation Disk broken up into variable sized segments
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
34/60
Memory Issues
Idea: Separate concepts of
address space Disk
memory locations RAM
Example:
Address Field = 216 = 65536 memory cells
Memory Size = 4096 memory cells
How can we fit
the Address Space
into Main Memory?
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
35/60
Paging
Break memories into Pages
NOTE: normally Main Memory has thousands of
pages
page
pagepage
1 page = 4096 bytes
New Issue: How to manage addressing?
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
36/60
Address MappingMapping 2ndary Memory addresses to
Main Memory addresses
page
page page
1 page = 4096 bytes
physical addressirtual address
Add M i
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
37/60
Address Mapping
Mapping 2ndary Memory (program/virtual)
addresses to Main Memory (physical) addresses
pagepage page
1 page = 4096 bytes
physical addressused by hardwarevirtual addressused by program
4095 8191
0 4096
virtualphysical
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
38/60
Paging
page
pagepage
4095 8191
04096
virtualphysical
0
4095 /04095/0
Illusion that Main Memory isLarge
Contiguous
Linear
Size(MM) = Size(2ndry M)
Transparent to Programmer
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
39/60
Paging Implementation
Virtual Address Space (Program) & Physical
Address Space (MM)
Broken up into equal pages(just like cache & MM!!)
Page size Always a power of 2
Common Size:512 to 64K bytes
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
40/60
Paging Implementation
Page Frames
Page Tables
Programs use Virtual Addresses
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
41/60
Memory Mapping
Note: 2ndry Mem = 64K; Main Mem = 32K
Page Frame:
home ofVM pages in MMPage Table:
home ofmappings for VM pages
Page # Page Frame #
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
42/60
Memory Mapping
Memory Management Unit (MMU):Device that performs virtual-to-physical mapping
MMU
MMU
15-bitPhysicalAddress
32-bit VMAddress
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
43/60
Memory Management Unit
32-bit Virtual Address
Broken into 2 portions
20-bit 12-bit
Virtual page # offset in page
(since our pages are 4KB)
How to determine if page is in
MM?Present/Absent Bit
in Page Table Entry
MMU
D d P i
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
44/60
Demand Paging
Possible Mapping
of pages
Page Fault:
Requested page isnot in MM
Demand Paging:
Page is demanded
by program
Page is loaded into
MM
D d P i
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
45/60
Demand Paging
Possible Mapping
of pages
Page Fault:
Requested page isnot in MM
Demand Paging:
Page is demanded
by program
Page is loaded into
MM
But What to bring in for a program on start up?
W ki S t
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
46/60
Working Set
Set of pages used by a process
Each process has a unique memory map
Importance in regards to a multi-tasked OS
At time t, there is a set of all
k recently usedpages
References tend to cluster on a small number ofpages
Put this set to Work!!! Store & Load it during Process Switching
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
47/60
Page Replacement Policy
Working Set:
Set of pages used actively & heavily
Kept in memory to reduce Page Faults
Set is found/maintained dynamically by OS
Replacement: OS tries to predict which
page would have least impact on the
running program
Common Replacement Schemes:Least Recently Used (LRU)
First-In-First-Out (FIFO)
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
48/60
Replacement Policy
Placement Policy
Which page is replaced?
Page removed should be the page least likely tobe referenced in the near future
Most policies predict the future behavior on the
basis of past behavior
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
49/60
Basic Replacement Algorithms
Least Recently Used (LRU)
Replaces the page that has not been referenced
for the longest timeBy the principle of locality, this should be the
page least likely to be referenced in the near
future
Each page could be tagged with the time of last
reference. This would require a great deal of
overhead.
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
50/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
51/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
52/60
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
53/60
SRAM DRAM
DRAMs use only
one transistor, plusa capacitor.
DRAMs are smallerand less expensive
because SRAMs aremade from four to sixtransistors (flip flops)
per bit.
SRAMs don't requireexternal refresh
circuitry or other workin order for them tokeep their data intact.
SRAM is faster thanDRAM
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
54/60
It has been discovered that for about 90%ofthe time that our programs execute only 10%of our code is used!
This is known as the Locality PrincipleTemporal Locality
When a program asks for a location in memory , it willlikely ask for that samelocation again very soonthereafter
Spatial Locality When a program asks for a memory location at a memory
address (lets say 1000) It will likely need a nearbylocation1001,1002,1003,10004 etc.
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
55/60
fastest possible access (usually 1 CPU cycle)Registers
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
56/60
We established that the Locality principlestates that only a small amount of Memory isneeded for most of the programs lifetime
We now have a Memory Hierarchy that placesvery fast yet expensive RAM near the CPU andlargerslowercheaper RAM further away
The trick is to keep the data that the CPUwants in the small expensive fast memoryclose to the CPU and how do we do
that???
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
57/60
Hardware and the Operating System are
responsible for moving data throughout theMemory Hierarchy when the CPU needs it.
Modern programming languages mainly assume
two levels of memory, main memory and diskstorage.
Programmers are responsible for moving data
between disk and memory through file I/O.
Optimizing compilers are responsible for
generating code that, when executed, will cause
the hardware to use caches and registers
efficiently.
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
58/60
A computer program or a hardware-maintainedstructure that is designed to manage a cache ofinformation
When the smaller cache is full, the algorithmmust choose which items to discard to makeroom for the new data
The "hit rate" of a cache describes how often a
searched-for item is actually found in the cache The "latency" of a cache describes how long
after requesting a desired item the cache canreturn that item
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
59/60
Each replacement strategy is a compromise between hit rate andlatency.
Direct Mapped Cache The direct mapped cache is the simplest form of cache and the easiest to check for a
hit.
Unfortunately, the direct mapped cache also has the worst performance, becauseagain there is only one place that any address can be stored.
Fully Associative Cache The fully associative cache has the best hit ratio because any line in the cache can
hold any address that needs to be cached.
However, this cache suffers from problems involving searching the cache
A replacement algorithm is used usually some form of a LRU "least recently used"algorithm
N-Way Set Associative Cache The set associative cache is a good compromise between the direct mapped and set
associative caches.
-
7/27/2019 21SCS147L17ReplacementPolicy4-7[1]
60/60
Virtual Memory is basically the extension ofphysical main memory (RAM) into a lowercost portion of our Memory Hierarchy (letssay Hard Disk)
A form of the Overlay approach, managed bythe OS, called Paging is used to swap pages
of memory back and forth between the Diskand Physical Ram.
Hard Disks are huge, but to you remember howslow they are??? Millions of times slower that