dynamic verification of cache coherence protocols
DESCRIPTION
Dynamic Verification of Cache Coherence Protocols. Jason F. Cantin Mikko H. Lipasti James E. Smith. Introduction. Multiprocessors are used for a variety commercial and mission-critical tasks Reliability is a growing concern Coherence is a fundamental feature of shared-memory MPs - PowerPoint PPT PresentationTRANSCRIPT
Dynamic Verification of Cache Dynamic Verification of Cache Coherence ProtocolsCoherence Protocols
Jason F. Cantin
Mikko H. Lipasti
James E. Smith
6/30/2001 Workshop on Memory Performance
Issues
IntroductionIntroduction
Multiprocessors are used for a variety commercial and mission-critical tasks
Reliability is a growing concernCoherence is a fundamental feature of
shared-memory MPs• High design complexity• Relatively low interconnect reliability
6/30/2001 Workshop on Memory Performance
Issues
Introduction:Introduction:Cache Coherence ProtocolsCache Coherence ProtocolsNotoriously difficult to design and verify
Often conceptually simple, but with complex implementations for efficiency and handling special cases
Multiple finite state machines operating concurrently
6/30/2001 Workshop on Memory Performance
Issues
Introduction:Introduction:A Simple ExampleA Simple ExampleMSI Protocol
“Architected” State Invalid / Not Present Shared (readable) Modified (read/write)
I S
M
Bus_RdX, Replace
Read
Write
Bus_RdX,Replace
Write
Bus_Rd
6/30/2001 Workshop on Memory Performance
Issues
M
S
PendRdI
I
PendWBM
PendRdX
S
PendRdX
I
Bus_RdX, Replace
ReadBus_Av
Replace
Bus_Av
Bus_Av
Bus_Av
Bus_Rd
Write
Bus_RdX
Write
Bus_RdX
Introduction:Introduction:Simple Example with a BusSimple Example with a Bus
MSI Protocol “Implementation” State
Transient states for pending operations
Arcs to satisfy requests while operations pending
6/30/2001 Workshop on Memory Performance
Issues
ProblemProblem
In practice, implementations can have dozens of states• Atomic memory operations• Split transaction buses• Protocol optimizations
Complexity grows exponentially with added states• Random testing: Low Coverage• Exhaustive testing: Too time consuming
6/30/2001 Workshop on Memory Performance
Issues
Dynamic VerificationDynamic Verification
Check the implementation at runtime• It is easier to check a computation than to
do the actual computation, provided there is a delay between the computation and the check (Rotenberg, AR-SMT)
• Simplified version of a processor implementation can be used for online verification (Austin, DIVA)
6/30/2001 Workshop on Memory Performance
Issues
Dynamic Verification of Cache Dynamic Verification of Cache CoherenceCoherence
A distributed form of dynamic verification for multiprocessor memory systems• Simplified version of protocol added to each node
• Maintains architected state
• Check completed transitions and actions against simple protocol
• Additional messages (assertions) sent between nodes to ensure coherence
6/30/2001 Workshop on Memory Performance
Issues
Conceptual View for Superscalar Conceptual View for Superscalar Processors (DIVA)Processors (DIVA) Single, centralized
check processor Receives
instructions serially in program order from implementation
Physical registersComplex ExecutionProcessor
CheckProcessor
Arch.registers
Prediction Tables
Committed results
R.O.B.
Arch.registers
6/30/2001 Workshop on Memory Performance
Issues
Conceptual View for CoherenceConceptual View for Coherence
Distributed checking hardware
Transitions received in parallel, in completion order
Shared Logical Bus
Shared Validation Bus
Implementation Protocol
Simple Protocol
Completed Transitions
6/30/2001 Workshop on Memory Performance
Issues
High Level OrganizationHigh Level Organization
Cache
Controller
P
Shared logical bus(addresses, data, control)
Memory
DV-CCChecker
Validation bus (assertions to be checked)
6/30/2001 Workshop on Memory Performance
Issues
BenefitsBenefits
Detects hardware faults• Redundant computation• Including intermittent network failures
Detects design mistakes• Checker is simple and easy to verify
6/30/2001 Workshop on Memory Performance
Issues
DrawbacksDrawbacks
Time is required for checking, but…• May be overlapped with other activities• Simple protocol requires fewer transitions
Assertions consume bandwidth• May need second bus / network
Additional hardware• But not much
6/30/2001 Workshop on Memory Performance
Issues
DV for coherence in an SMPDV for coherence in an SMP
Architected state stored in a second tag array
Transactions sent to the checker when architected state changes• Address• Initial State and Final States• Input (Request, Snoop Responses, etc)• Action (Send Data, Respond Shared, etc)
6/30/2001 Workshop on Memory Performance
Issues
DV for coherence in an SMP (2)DV for coherence in an SMP (2)
Checker compares the initial state of a transition against the architected state
Final state and action recomputed and compared to implementation’s result
Assertions broadcast to other nodes to check coherence and confirm completion of transactions
Watchdog timer detects deadlock, livelock, and other omission failures
6/30/2001 Workshop on Memory Performance
Issues
Init. stateFinal state Input ActionAddress
NextState Logic
=?
Arch.Tag
State
Action Logic
=?=?
Error Detection / Diagnosis
Transition From Implementation Protocol
UpdateTags
Validation Bus
AsstSendBuffer
Checking a State TransitionChecking a State Transition
Watchdogtimer
6/30/2001 Workshop on Memory Performance
Issues
Arch.Tag
State
Error Detection / Diagnosis
Watchdogtimer
Validation Bus
AssertRecv
Buffer
OK
Address
Remote State
Checking an AssertionChecking an Assertion
6/30/2001 Workshop on Memory Performance
Issues
When to Broadcast AssertionsWhen to Broadcast Assertions
For MSI:1. I S (readable copy loaded)2. I M (writeable copy loaded)3. S M (upgrade)4. M I (writeback)
Note: The MS transition results from remote reads, and doesn’t require an extra assertion. Replacements (SI) are not considered here.
6/30/2001 Workshop on Memory Performance
Issues
Preliminary Data (4-way SMP)Preliminary Data (4-way SMP)
Most memory references do not change cache state (checker need not have high bandwidth)
MSI Stable State Changes/Reference
0.54%1.19%
7.17%
1.71%2.50%
0%
2%
4%
6%
8%
barnes ocean db2 specweb tpc-w
Perc
enta
ge o
f Re
fere
nces
6/30/2001 Workshop on Memory Performance
Issues
Preliminary Data (4-way SMP)Preliminary Data (4-way SMP)
0%
20%
40%
60%
80%
100%
Ext
ra M
ess
ag
es I-->M, S-->M
I-->S, I-->M
I-->S, I-->M, S-->M
All Transactions
6/30/2001 Workshop on Memory Performance
Issues
Future WorkFuture Work
Performance impact for a real SMP protocol implementation• In progress
Directory-based protocolsDynamically verifying memory modelsRecovery
• Can stall to avoid error propagation• Can write checkpoints periodically
6/30/2001 Workshop on Memory Performance
Issues
In SummaryIn Summary
Dynamic verification can be applied to multiprocessor systems (in a distributed manner)
Improves fault-tolerance, and design verification may be relaxed
More to come