Vinodh Cuppu and Bruce Jacob, University of Maryland

Concurrency, Latency, or System Overhead: which Has the Largest Impact on Uniprocessor DRAM-System Performance?

Richard Wells

ECE 7810

April 21, 2009

The University of Utah

Reservations

The paper is old Presented at ISCA 2001 Only considers uniprocessor systems

They draw some conclusions that while valid are focused on their research goals

Papers relating to our groups project are not prevalent in recent years, except one already presented at the architecture reading club.

The University of Utah

Overview

Investigate DRAM system organization parameters to determine bottleneck

Determine synergy or antagonism between groups of parameters

Empirically determine the optimal DRAM system configuration

The University of Utah

Methodologies to increase system performance

Concurrent transactions

Reducing latency

Reduce system overhead

The University of Utah

Previous approaches to reduce memory system overhead

DRAM Component Increase bandwidth

Current “tack” taken by the PC industry Reduce DRAM latency

ESDRAM SRAM cache for the full row buffer Allows precharge to begin immediately after

access FCRAM

Subdivide internal bank by activating only a portion of each wordline

The University of Utah

Previous approaches to reduce memory system overhead (cont.)

Reduce capacitance on word access to 30 ns (2001)

MoSys Subdivides storage into a large number of very

small banks Reduces latency of DRAM core to nearly that of

SRAM VCDRAM

Set-associative SRAM buffer that holds a number of sub-pages

The University of Utah

The Jump

DRAM oriented approaches do reduce application execution time

Because zero latency DRAM doesn’t reduce the overhead of memory system to zero, bus transactions are considered

Other factors considered Turnaround time Queuing delays Inefficiencies due to asymmetric read/write requests Multiprocessor - Arbitration and Cache coherence would

add to overhead

The University of Utah

CPU – DRAM Channel

Access reordering (cited Impulse group here at the U) Compacts sparse data into densely-packed bus

transactions Reduces the number of bus transactions Possibly reduces duration of bus transaction

The University of Utah

Increasing concurrency

Different banks on the same channel Independent channels to different banks Pipelined requests Split-transaction bus

The University of Utah

Decreasing channel latency

Due to channel contention Back to back read requests Read arriving during precharge Narrow channels Large data burst size

The University of Utah

Addressing System Overhead

Bus turnaround time Dead cycles due to asymmetric read/write

shapes Queuing overhead Coalescing queued requests Dynamic re-prioritization of requests

The University of Utah

Timing Assumptions

10 ns address 70 ns until burst starts on a read 40 ns until a write can start

The University of Utah

Split Transaction Bus Assumptions

Overlapping Supported Back-to-back reads Back-to-back read/write pairs

The University of Utah

Burst Ordering, Coalescing

Critical-burst first, non-critical burst second, writes last

Coalesce writes followed by reads

The University of Utah

Bit Addressing & Page Policy

Bit assignments chosen to exploit page mode and maximize degree of memory concurrency Most significant bits identify the smallest-scale

component in the system Least significant bits identify the largest-scale

component in the system Allows sequential addresses to be stripped

across channels maximizing concurrency Close-page auto-precharge policy

The University of Utah

Simulation Environment

SimpleScalar (used in 6810) 2 GHz clock L1 caches 64Kb/64Kb, 2-way set associative L2 cache unified 1Mb, 4-way set associative, 10

cycle access time Lock-up free cache using miss status holding

register (MSHR)

The University of Utah

Timing Calculations

CPU + DRAM determined by running a second simulation with perfect primary memory (available on next cycle)

The University of Utah

Results – Degrees of Freedom

Bus Speed: 800 MHz Bus width: 1, 2, 4, 8 bytes Channels: 1, 2, 4 Banks/Channel: 1, 2, 4, 8 Queue Size: infinite, 0, 1, 2, 8, 16,

32 Turnaround: 0, 1 cycles R/W shapes: symmetric, asymmetric

The University of Utah

Results – Execution Times

Assumes infinite request queue System parameters can lead to widely varying CPI

The University of Utah

Results – Turnaround and Banks

Turnaround only accounts for 5% of system related overhead

Banks/Channel accounts for 1.2x – 2x variation – shows concurrency is important

Latency accounts for over about 50% of CPI

The University of Utah

Results – Burst Length vs. BW

Accounts for 10-30% of execution time Wider channels have optimal performance with larger bursts Narrow channels have optimal performance with smaller bursts

The University of Utah

Results - Concurrency

The University of Utah

Results – Concurrency (Cont.)

Increasing the number of banks typically increases performance, but not always much

Many narrow channels is risky because application might not have much inherent concurrency

Optimal 1 channel x 4 bytes x 64 byte burst, 2 channel x 2 bytes x 64 byte burst, 1 channel x 4 bytes x 128 byte burst

Performance varies depending on the concurrency of the benchmark

The University of Utah

Results – Concurrency (Cont.)

“We find that, in a uniprocessor setting, concurrency is very important, but it is not more important than latency. . . . However, we find that if, in an attempt to increase support for concurrent transactions, one interleaves very small bursts or fragments the DRAM bus into multiple channels, one does so at the expense of latency, and this expense is too great for the levels of concurrency being produced.”

The University of Utah

Results – Request Queue Size

The University of Utah

Results – Request Queue Size

How queuing benefits system performance Sub-blocks of different read requests can be interleaved Writes can be buffered until read-burst traffic has died

down Read and write requests may be coalesced

Applications with significant write activity see more benefit from queuing Bzip has many more writes than GCC

Anomalies attributed to requests with temporal locality go to the same bank. With a small queue they are delayed.

The University of Utah

Conclusions

Tuning system level parameters can improve the memory system performance by 40% Bus turnaround – 5-10% Banks – 1.2x – 2x Burst length vs. bandwidth – 10%-30% Concurrency

Smaller bursts to allow for interleaving is not a good idea because it limits concurrency

The University of Utah

Our Project

To evaluate the effect of mat array size on power and latency of the DRAM chips.

Simulators Cacti DRAMSim Simics

Predicted Results Positive

Decreased memory latency Decreased power profile DIMM parallelism increase

Negative Decreased row buffer hit rates Decreased memory capacity (for same chip area) Increase the important cost/bit metric

The University of Utah

How project relates to the paper

Trying to decrease the memory system bottlenecks Although we have evaluated bottlenecks

differently Jacob indirectly showed the importance of

minimizing DRAM latency DRAM latency was largest portion of CPI so

Amdahl’s law would justify reducing latency Both our solutions could work together

synergistically

The University of Utah

Additional thoughts

The current path of DRAM innovation has limitations

DRAM chips and DIMMs need to undergo fundamental changes, of which this could be a step

Helps power efficiency Can balance with cost effectiveness Partially addresses the memory gap

The University of Utah

Questions

Questions?