multilevel caches microprocessors are getting faster and including a small high speed cache on the...
TRANSCRIPT
Computer Architecture CSE 3322
Lecture 23
Multilevel Caches
• Microprocessors are getting faster and including a smallhigh speed cache on the same chip
Multilevel Caches
• Microprocessors are getting faster and including a smallhigh speed cache on the same chip
• A Second level cache is off chip that is larger and slowerthat interfaces with the Main Memory
Multilevel Caches
• Microprocessors are getting faster and including a smallhigh speed cache on the same chip
• A Second level cache is off chip that is larger and slowerthat interfaces with the Main Memory
• Same cache techniques can be used
Multilevel Caches
• Microprocessors are getting faster and including a smallhigh speed cache on the same chip
• A Second level cache is off chip that is larger and slowerthat interfaces with the Main Memory
• Same cache techniques can be used
Ave Memory Access Time = Hit time(L1)+ Miss rate (L1) * Miss Penalty( L1)
Multilevel Caches
• Microprocessors are getting faster and including a smallhigh speed cache on the same chip
• A Second level cache is off chip that is larger and slowerthat interfaces with the Main Memory
• Same cache techniques can be used
Ave Memory Access Time = Hit time(L1)+ Miss rate (L1) * Miss Penalty( L1)
Miss penalty (L1) = Hit time (L2) + Miss rate (L2) * Miss Penalty( L2)
Multilevel Caches
• Microprocessors are getting faster and including a smallhigh speed cache on the same chip
• A Second level cache is off chip that is larger and slowerthat interfaces with the Main Memory
• Same cache techniques can be used
Ave Memory Access Time = Hit time(L1)+ Miss rate (L1) * Miss Penalty( L1)
Miss penalty (L1) = Hit time (L2) + Miss rate (L2) * Miss Penalty( L2)
Ave Memory Access Time = Hit time(L1)+ Miss rate (L1) * Hit time (L2)+ Miss rate (L1)*Miss rate (L2) * Miss Penalty( L2)
Ave Memory Access Time = Hit time(L1)
+ Miss rate (L1) * Hit time (L2)
+ Miss rate (L1)*Miss rate (L2) * Miss Penalty( L2)
Ex: Hit time (L1) = 1 clock cycleMiss rate(L1) = 10%Hit time ( L2) = 10 Clock CyclesMiss rate (L2) = 20%Miss penalty (L2) = 100 clock cyles
If no secondary cacheAve Memory Access = 1 + 0.1 * 100 = 11 clock cycles
Ave Memory Access Time = Hit time(L1)
+ Miss rate (L1) * Hit time (L2)
+ Miss rate (L1)*Miss rate (L2) * Miss Penalty( L2)
Ex: Hit time (L1) = 1 clock cycleMiss rate(L1) = 10%Hit time ( L2) = 10 Clock CyclesMiss rate (L2) = 20%Miss penalty (L2) = 100 clock cyles
If no secondary cacheAve Memory Access = 1 + 0.1 * 100 = 11 clock cycles
With secondary cacheAve Memory Access = 1 + 0.1*10 + 0.1*0.2*100
1 + 1 + 2 = 4 clock cycles
Characteristics of programs that makes efficient use of cache memory
Characteristics of programs that makes efficient use of cache memory
• Code has tight loops with lots of reuse
Characteristics of programs that makes efficient use of cache memory
• Code has tight loops with lots of reuse
• Code minimizes jumps and branches to far away
Characteristics of programs that makes efficient use of cache memory
• Code has tight loops with lots of reuse
• Code minimizes jumps and branches to far away
• Data marches through arrays
Processor
Memory Hierarchy
Cache Memory
Main Memory
Transfer in 1 clockcycle
Transfer > 10 clockcycles
SRAM
DRAM
Processor
Virtual Memory Hierarchy
Cache Memory
Main Memory
Transfer in 10 + clock cycles
Transfer > 100,000 clock cycles
DRAM
HardDisk
Virtual MemoryThe Illusion of Unlimited Amount of Memory
1. Invented to allow programmers to use the full addressingcapability of the processor. Not limited by physical RAM
Virtual MemoryThe Illusion of Unlimited Amount of Memory
1. Invented to allow programmers to use the full addressingcapability of the processor. Not limited by physical RAM
2. Enabled Multiprocessing. Computer running multipleprocesses ( programs and environments) each with it’sown address space. ( Time Sharing )
Virtual MemoryThe Illusion of Unlimited Amount of Memory
1. Invented to allow programmers to use the full addressingcapability of the processor. Not limited by physical RAM
2. Enabled Multiprocessing. Computer running multipleprocesses ( programs and environments) each with it’sown address space. ( Time Sharing )
3. Provides a structure for protection. Pages can have restricted use by assigned processes. Types and Levels
of access.
Virtual MemoryThe Illusion of Unlimited Amount of Memory
1. Invented to allow programmers to use the full addressingcapability of the processor. Not limited by physical RAM
2. Enabled Multiprocessing. Computer running multipleprocesses ( programs and environments) each with it’sown address space. ( Time Sharing )
3. Provides a structure for protection. Pages can have restricted use by assigned processes. Types and Levels
of access.
4. Provides simple relocation by mapping to pages inphysical memory
Virtual Address – 4 GB
31 30 29 . . . 14 13 12 11 10 9 8 . . . 2 1 0
Virtual page number Page offset
28 27 26 . . . 14 13 12 11 10 9 8 . . . 2 1 0
Physical page number Page offset
Page = 212 = 4KBTranslation
Physical Address – 512 MB
Virtual Address
Virtual Page No. Page Offset
PageTable Main
Memory
Hard Disk
PhysicalAddress
Page = Block
What is the miss penalty?Called Page Fault
1. Where can a page ( block) be placed in Main Memory?
1. Where can a page ( block) be placed in Main Memory?
• The miss penalty is very high, so minimize miss rate
1. Where can a page ( block) be placed in Main Memory?
• The miss penalty is very high, so minimize miss rate
• The long access time of the hard disk enables the OS to control placing the pages in Main Memory
1. Where can a page ( block) be placed in Main Memory?
• The miss penalty is very high, so minimize miss rate
• The long access time of the hard disk enables the OS to control placing the pages in Main Memory
• The lowest miss rate results from locating pagesanywhere in Main Memory ( Fully Associative )
2. How is a page ( block) found if it is in Main Memory?
2. How is a page ( block) found if it is in Main Memory?
• A page is located by the page table that contains thephysical address of the page in Main Memory inthe entry addressed by the virtual page number.( The index is mapped by the software using the page table and no tag is required.)
2. How is a page ( block) found if it is in Main Memory?
• A page is located by the page table that contains thephysical address of the page in Main Memory inthe entry addressed by the virtual page number.( The index is mapped by the software using the page table and no tag is required.)
• The page table has an entry for every virtual pagenumber which can be 1 M entries. So, the page table is typically stored in Main Memory.
2. How is a page ( block) found if it is in Main Memory?
• A page is located by the page table that contains thephysical address of the page in Main Memory inthe entry addressed by the virtual page number.( The index is mapped by the software using the page table and no tag is required.)
• The page table has an entry for every virtual pagenumber which can be 1 M entries. So, the page table is typically stored in Main Memory.
• Accessing Main Memory twice each time takes to long,so another cache must be added.
3. Which page ( block) should be replaced on a VirtualMemory Miss ( Page Fault )
3. Which page ( block) should be replaced on a VirtualMemory Miss ( Page Fault )
• Under OS control, so clever algorithms are feasible.
3. Which page ( block) should be replaced on a VirtualMemory Miss ( Page Fault )
• Under OS control, so clever algorithms are feasible.
• Usually try to approximate least recently used (LRU)replacement strategy.
3. Which page ( block) should be replaced on a VirtualMemory Miss ( Page Fault )
• Under OS control, so clever algorithms are feasible.
• Usually try to approximate least recently used (LRU)replacement strategy.
• May keep a use bit, which is set whenever a page isaccessed to help estimate LRU.
4. What happens on a Write?
4. What happens on a Write?
• Write Through is not feasible due to the very slowspeed of the Hard Disk
4. What happens on a Write?
• Write Through is not feasible due to the very slowspeed of the Hard Disk
• The write strategy is always Write Back
4. What happens on a Write?
• Write Through is not feasible due to the very slowspeed of the Hard Disk
• The write strategy is always Write Back
• Write Back# The word is written only to the page in Main
Memory (cache).# The modified page is written to the Hard Disk only when it is replaced. A Dirty Bit
indicates if a page has been altered.
Selecting a Page Size
• Reasons Pages should be larger:1. Minimize miss rate, but too large can get
no improvement or even increase.2. Reduces the size of the page table3. Transfers to / from Hard Disk are more efficient
( also over networks)
Selecting a Page Size
• Reasons Pages should be larger:1. Minimize miss rate, but too large can get
no improvement or even increase.2. Reduces the size of the page table3. Transfers to / from Hard Disk are more efficient
( also over networks)
• Reasons Pages should be smaller:1. Wastes space because each process has several
primary pages required.2. Start up time is longer for small processes.
Selecting a Page Size
• Reasons Pages should be larger:1. Minimize miss rate, but too large can get
no improvement or even increase.2. Reduces the size of the page table3. Transfers to / from Hard Disk are more efficient
( also over networks)
• Reasons Pages should be smaller:1. Wastes space because each process has several
primary pages required.2. Start up time is longer for small processes.
• Page size is 4 KB to 64 KB
Virtual Address
Virtual Page No. Page Offset
PageTable Main
Memory
Hard Disk
PhysicalAddress
Page = Block
Physical Page Address
Valid Use
Dirty
Page TableMaps Virtual PageNo. to Disk Addr
The Page Table is• Large• Usually in DRAM Main Memory• Too Slow
The Page Table is• Large• Usually in DRAM Main Memory• Too Slow
So, provide a cache for the Page Table
TLB – translation-lookaside buffer
TLB – translation-lookaside buffer
Virtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Physical Page Addr
Tag
Valid, Dirty, Use
TLB – translation-lookaside buffer
Virtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Page Table Register
+
PhysicalPage Addr
Physical Page Addr
Tag
Valid, Dirty, Use
TLB – translation-lookaside bufferVirtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Page Table Register
+
PhysicalPage Addr
Physical Page Addr
Tag
Valid, Dirty, Use
TLB – translation-lookaside bufferVirtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Page Table Register
+
PhysicalPage Addr
Physical Page Addr
Tag
Valid, Dirty, Use
Hit
TLB – translation-lookaside bufferVirtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Page Table Register
+
PhysicalPage Addr
Physical Page Addr
Tag
Valid, Dirty, Use
Hit• Read Main Set Use Bit TLB
TLB – translation-lookaside bufferVirtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Page Table Register
+
PhysicalPage Addr
Physical Page Addr
Tag
Valid, Dirty, Use
Hit• Write Main Set Dirty Bit TLB
TLB – translation-lookaside bufferVirtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Page Table Register
+
PhysicalPage Addr
Physical Page Addr
Tag
Valid, Dirty, Use
Miss
TLB – translation-lookaside bufferVirtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Page Table Register
+
PhysicalPage Addr
Physical Page Addr
Tag
Valid, Dirty, Use
MissTLB Miss or Page Fault
TLB Miss –
TLB Miss –
1. Select entry to be replaced (LRU or Random)
TLB Miss –
1. Select entry to be replaced (LRU or Random)2. Write Back Use and Dirty Bits to Page Table
replaced entry
TLB Miss –
1. Select entry to be replaced (LRU or Random)2. Write Back Use and Dirty Bits to Page Table
replaced entry3. Access requested entry in Page Table
If Page is in Main Memory ( Valid), load thetranslation from the Page Table to TLB andtry again
TLB Miss – 1. Select entry to be replaced (LRU or Random)2. Write Back Use and Dirty Bits to Page Table
replaced entry3. Access requested entry in Page Table
If Page is in Main Memory ( Valid), load thetranslation from the Page Table to TLB andtry again
If Page is not in Main Memory ( Valid), then it isa Page Fault :
Write Back Replaced Page to Disk if DirtyMove page from Disk to Main MemoryUpdate Page Table & TLB and try again
Virtual Address
31 30 29 . . . 14 13 12 11 10 9 . . . . 2 1 0Virtual Page Number Page Offset
==
==
Tag Physical Page Number20
TLB
Hit
Valid
Dirty
To Main Memory
2012
Virtual Address
31 30 29 . . . 14 13 12 11 10 9 . . . . 2 1 0Virtual Page Number Page Offset
==
==
Tag Physical Page Number20
TLB
Hit
Valid
Dirty
To Main Memory
2012
Design GoalsHit Time – 1 clock cycleTLB Miss Penalty – 10 - 20 clock cyclesTLB Miss Rate - < 1%
Virtual Address
31 30 29 . . . 14 13 12 11 10 9 . . . . 2 1 0Virtual Page Number Page Offset
==
==
Tag Physical Page Number20
TLB
Hit
Valid
Dirty
Tag - m Index-k
20 12Block Offset
Byte Offset
Main Memory Cache
PhysicalAddress
TLB – translation-lookaside buffer
Virtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Physical Page Addr
Tag
Valid, Dirty, Use
Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address
TLB – translation-lookaside buffer
Virtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Physical Page Addr
Tag
Valid, Dirty, Use
Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address
Size of page table assuming no disk address and 4 bits V,P,D,U?
TLB – translation-lookaside buffer
Virtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Physical Page Addr
Tag
Valid, Dirty, Use
Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address
Size of page table assuming no disk address and 4 bits V,P,D,U?
14
TLB – translation-lookaside buffer
Virtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Physical Page Addr
Tag
Valid, Dirty, Use
Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address
Size of page table assuming no disk address and 4 bits V,P,D,U?
1426
TLB – translation-lookaside buffer
Virtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Physical Page Addr
Tag
Valid, Dirty, Use
Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address
Size of page table assuming no disk address and 4 bits V,P,D,U?
1426
262
TLB – translation-lookaside buffer
Virtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Physical Page Addr
Tag
Valid, Dirty, Use
Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address
Size of page table assuming no disk address and 4 bits V,P,D,U?
1426
26222
TLB – translation-lookaside buffer
Virtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Physical Page Addr
Tag
Valid, Dirty, Use
Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address
Size of page table assuming no disk address and 4 bits V,P,D,U?
1426
26222
x 26
TLB – translation-lookaside buffer
Virtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Physical Page Addr
Tag
Valid, Dirty, Use
Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address
TLB 2-way Associative with 256 entries. Index and Tag?
1426
26222
x 26
TLB – translation-lookaside buffer
Virtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Physical Page Addr
Tag
Valid, Dirty, Use
Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address
TLB 2-way Associative with 256 entries. Index and Tag?
1426
26222
x 26
Tag Index
TLB – translation-lookaside buffer
Virtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Physical Page Addr
Tag
Valid, Dirty, Use
Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address
TLB 2-way Associative with 256 entries. Index and Tag?
1426
26222
x 26
Tag Index-8
TLB – translation-lookaside buffer
Virtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Physical Page Addr
Tag
Valid, Dirty, Use
Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address
TLB 2-way Associative with 256 entries. Index and Tag?
1426
26222
x 26
Tag-18 Index-8
TLB – translation-lookaside buffer
Virtual Page No. Page Offset
PageTable
MainMemory
Hard Disk
TLB
Physical Page Addr
Tag
Valid, Dirty, Use
Ex: - 40 bit virtual byte address - 16KB pages - 36 bit physical address
TLB 2-way Associative with 256 entries. Index and Tag?
1426
26222
x 26
Tag-18 Index-8
4 18
88
CPU
TLB
On Chip Cache
Secondary Cache
DRAM Main Memory
Hard Disk
Virtual Address 32-64 bits4KB pages
Microprocessors
CPU
TLB
On Chip Cache
Secondary Cache
DRAM Main Memory
Hard Disk
Virtual Address 32-64 bits4KB pagesTLB Split Data and Instruction 4-Way Set Associative
64 – 128 entries
Microprocessors
CPU
TLB
On Chip Cache
Secondary Cache
DRAM Main Memory
Hard Disk
Virtual Address 32-64 bits4KB pagesTLB Split Data and Instruction 4-Way Set Associative
64 – 128 entriesOn Chip Cache Split Data and Instruction 8KB – 16KB each 4-Way Associative 32 Bytes / Block
Microprocessors
Implementing Protection with Virtual Memory
• Add protection bits to the TLB / Page TableRead / Write AccessUser(s) / Supervisor
Implementing Protection with Virtual Memory
• Add protection bits to the TLB / Page TableRead / Write AccessUser(s) / Supervisor
• CPU supplies Read / Write and User / Supervisorsignals for each access
Implementing Protection with Virtual Memory
• Add protection bits to the TLB / Page TableRead / Write AccessUser(s) / Supervisor
• CPU supplies Read / Write and User / Supervisorsignals for each access
• Comparisons can be made in the TLB. Non-comparecan cause an exception
Implementing Protection with Virtual Memory
• Add protection bits to the TLB / Page TableRead / Write AccessUser(s) / Supervisor
• CPU supplies Read / Write and User / Supervisorsignals for each access
• Comparisons can be made in the TLB. Non-comparecan cause an exception
• User programs cannot modify the protectionbits in Page Table / TLB