flin: enabling fairness and enhancing performance in modern nvme solid state...

FLIN: Enabling Fairness and Enhancing Performancein Modern NVMe Solid State Drives

Arash Tavakkol† Mohammad Sadrosadati† Saugata Ghose‡ Jeremie S. Kim‡† Yixin Luo‡Yaohua Wang†§ Nika Mansouri Ghiasi† Lois Orosa†∗ Juan Gómez-Luna† Onur Mutlu†‡

†ETH Zürich ‡Carnegie Mellon University §NUDT ∗Unicamp

Modern solid-state drives (SSDs) use new host–interface pro-tocols, such as NVMe, to provide applications with fast accessto storage. These new protocols make use of a concept knownas the multi-queue SSD (MQ-SSD), where the SSD has directaccess to the application-level I/O request queues. This removesmost of the OS software stack that was used in older protocolsto control how and when the I/O requests were dispatched tostorage devices. Unfortunately, while the elimination of the OSsoftware stack leads to a signi�cant performance improvement,we show in this paper that it introduces a new problem: unfair-ness. This is because the elimination of the OS software stackeliminates the mechanisms that were used to provide fairnessamong applications in older SSDs.

To study application-level unfairness, we perform experimentsusing four real state-of-the-art MQ-SSDs. We demonstrate thatthe lack of fair scheduling mechanisms leads to high unfairnessamong concurrently-executing applications due to the interfer-ence among them. For instance, when one of these applicationsissues manymore I/O requests than others, the other applicationsare slowed down signi�cantly. We perform a comprehensiveanalysis of interference in real MQ-SSDs, and �nd four majorinterference sources: (1) the intensity of requests sent by eachapplication, (2) di�erences in request access patterns, (3) theratio of reads to writes, and (4) garbage collection.To alleviate unfairness in MQ-SSDs, we propose the Flash-

Level INterference-aware scheduler (FLIN). FLIN is a lightweightI/O request scheduling mechanism that provides fairness amongrequests from di�erent applications. FLIN uses a three-stagescheduling algorithm that protects against all four major sourcesof interference, while respecting the application-level prioritiesassigned by the host. FLIN is implemented fully within theSSD controller �rmware, requiring no new hardware, and hasnegligible (<0.06%) storage cost. Compared to a state-of-the-artI/O scheduler, FLIN improves the fairness and performance of awide range of enterprise and datacenter storage workloads, withan average improvement of 70% and 47%, respectively.

1. IntroductionSolid-state drives (SSDs) are widely used as a storage

medium today due to their high throughput, low responsetime, and low power consumption compared to conventionalhard disk drives (HDDs). As more SSDs are deployed in data-centers and enterprise platforms, there has been a continuedneed to improve SSD performance. One area where SSD man-ufacturers have innovated on SSD performance is the host–interface protocol, which coordinates communication betweenapplications and the SSD. SSDs initially adopted existing host–interface protocols (e.g., SATA [88]) that were originally de-signed for lower-performance HDDs. As the performance ofthe underlying storage technology (e.g., NAND �ash memory)used by the SSD increased, these host–interface protocolsbecame a signi�cant performance bottleneck [114], mainlybecause these protocols rely on the OS to manage I/O requestsand data transfers between the host system and the SSD.

To overcome this bottleneck, modern enterprise SSDs (e.g.,[30–33,65,66,87,103,104,110,111]) use new high-performanceprotocols, such as the Non-Volatile Memory Express (NVMe)protocol [21, 83]. These new protocols make use of the multi-queue SSD (MQ-SSD) [8,48,101] concept, where multiple host-side I/O request queues (in software) are directly exposed tothe SSD controller. There are two bene�ts to directly expos-ing the request queues to the SSD controller: (1) there is nolonger any need for the OS software stack to manage the I/Orequests; and (2) the SSD can make more e�ective I/O requestscheduling decisions than the OS, since the SSD knows ex-actly which of its internal resources are available or are busyserving other requests. Thus, the protocols eliminate the OSsoftware stack, enabling MQ-SSDs to provide signi�cantlyhigher performance than traditional SSDs [101, 114].

Unfortunately, eliminating the OS software stack also elim-inates critical mechanisms that were previously implementedas part of the stack, such as fairness control [6, 8, 35, 55, 56, 84,89, 106, 118]. Fairness control mechanisms work to equalizethe e�ects of interference across applications when multipleapplications concurrently access a shared device. Fairness isa critical requirement in multiprogrammed computers andmulti-tenant cloud environments, where multiple I/O �ows(i.e., series of I/O requests) from di�erent applications concur-rently access a single, shared SSD [36, 43, 84, 91, 94, 101].

For older host–interface protocols, the OS software stackprovides fairness by limiting the number of requests that eachapplication can dispatch from the host to the SSD. Since fair-ness was handled by the OS software stack, the vast majorityof state-of-the-art SSD I/O request schedulers did not con-sider request fairness [20, 36, 37, 40, 77, 90, 112]. Surprisingly,even though new host–interface protocols, such as NVMe, donot use the OS software stack, modern MQ-SSDs still do notcontain any fairness mechanisms. To demonstrate this, weperform experiments using four real state-of-the-art enter-prise MQ-SSDs. We make two major �ndings. First, when twoapplications share the same SSD, one of the applications typi-cally slows down (i.e., it takes more time to execute comparedto if it were accessing the SSD alone) signi�cantly. When werun a representative workload on our four SSDs, we observethat such slowdowns range from 2x to 106x. Second, with theremoval of the fairness control mechanisms that were in thesoftware stack, an application that makes a large number ofI/O requests to an MQ-SSD can starve requests from otherapplications, which can hurt overall system performance andlead to denial-of-service issues [8, 84, 101]. Therefore, we con-clude that there is a pressing need to introduce fairness controlmechanisms within modern SSD I/O request schedulers.

In order to understand how to control fairness in a modernSSD, we experimentally analyze the sources of interferenceamong I/O �ows from di�erent applications in an SSD usingthe detailed MQSim simulator [101]. Our experimental resultsenable us to identify four major types of interference:1. I/O Intensity: The SSD controller breaks down each I/O

request into multiple page-size (e.g., 4 kB) transactions. An

1

I/O �ow with a greater �ow intensity (i.e., the rate at whichthe �ow generates transactions) causes a �ow with a lower�ow intensity to slow down, by as much as 49x in ourexperiments (Section 3.1).

2. Access Pattern: The SSD contains per-chip transactionqueues that expose the high internal parallelism that isavailable. Some I/O �ows exploit this parallelism by in-serting transactions that can be performed concurrentlyinto each per-chip queue. Such �ows slow down by asmuch as 2.4x when run together with an I/O �ow that doesnot exploit parallelism. This is because an I/O �ow thatdoes not exploit parallelism inserts transactions into onlya small number of per-chip queues, which prevents thehighly-parallel �ow’s transactions from completing at thesame time (Section 3.2).

3. Read/Write Ratio: Because SSD writes are typically 10–40xslower than reads [62–64,67,92], a �ow with a greater frac-tion of write requests unfairly slows down a concurrently-running �ow with a smaller fraction of write requests, i.e.,a �ow that is more read-intensive (Section 3.3).

4. Garbage Collection: SSDs perform garbage collection(GC) [10,13,16,23], where invalidated pages within the SSDare reclaimed (i.e., erased) some time after the invalidation.When an I/O �ow requires frequent GC operations, theseoperations can block the I/O requests of a second �ow thatperforms GC only infrequently, slowing down the second�ow by as much as 3.5x in our experiments (Section 3.4).To address the lack of OS-level fairness control mechanisms,

prior work [36] implements a fair I/O request scheduling pol-icy in the SSD controller that is similar to the software-basedpolicies previously used in the OS [120]. Unfortunately, thisapproach has two shortcomings: (1) it cannot control twotypes of interference, access pattern interference or GC inter-ference; and (2) it signi�cantly decreases the responsivenessand overall throughput of an MQ-SSD [89]. Our goal inthis work is to design an e�cient I/O request scheduler fora modern SSD that (1) provides fairness among I/O �ows bymitigating all four types of interference within an SSD, and(2) maximizes the performance and throughput of the SSD.

To this end, we propose the Flash-Level INterference-awarescheduler (FLIN) for modern SSDs. FLIN considers the prior-ity class of each �ow (which is determined by the OS), andcarefully reorders transactions within the SSD controller tobalance the slowdowns incurred by �ows belonging to thesame priority class. FLIN uses three stages to schedule re-quests. The �rst stage, Intensity- and Parallelism-aware QueueInsert, reorders transactions to mitigate the impact of di�erentI/O intensities and di�erent access patterns. The second stage,Flow-level Priority Arbitration, prioritizes transactions basedon the priority class of their corresponding �ow. The thirdstage, Read/Write Wait-Balancing, balances the overall stalltime of read and write requests, and schedules GC activitiesto distribute the GC overhead proportionally across di�erent�ows. FLIN is fully implementable in the �rmware of a mod-ern SSD. As such, it requires no additional hardware, and inthe worst case consumes less than 0.06% of the DRAM storagespace that is already available within the SSD controller.

Our evaluations show that FLIN signi�cantly improves fair-ness and performance across a wide variety of workloads andMQ-SSD con�gurations. Compared to a scheduler that uses astate-of-the-art scheduling policy [37] and fairness controlssimilar to those previously used in the OS [36], FLIN improvesfairness by 70%, and performance by 47%, on average, across40 MQ-SSD workloads that contain various combinations of

concurrently-running I/O �ows with di�erent characteristics.We show that FLIN improves both fairness and performanceacross all of our tested MQ-SSD con�gurations.

In this paper, we make the following major contributions:• We provide an in-depth experimental analysis of unfairness

in state-of-the-art multi-queue SSDs. We identify four majorsources of interference that contribute to unfairness.

• We propose FLIN, a new I/O request scheduler for mod-ern SSDs that e�ectively mitigates interference amongconcurrently-running I/O �ows to provide both high fair-ness and high performance.

• We comprehensively evaluate FLIN using a wide variety ofstorage workloads consisting of concurrently-running I/O�ows, and demonstrate that FLIN signi�cantly improvesfairness and performance over a state-of-the-art I/O requestscheduler across a variety of MQ-SSD con�gurations.

2. Background and MotivationWe �rst provide a brief background on multi-queue SSD

(MQ-SSD) device organization and I/O request management(Section 2.1). We then motivate the need for a fair MQ-SSDI/O scheduler (Section 2.2).

2.1. SSD OrganizationFigure 1 shows the internal organization of an MQ-SSD. The

components within an SSD are split into two groups: (1) theback end, which consists of data storage units; and (2) the frontend, which consists of control and communication units. Thefront end and back end work together to service the multiplepage-size I/O transactions that constitute an I/O request.

Host DRAMHost DRAM HIL

Device-levelRequest Queues

FTL

Flash Management

Data

WRQ

RDQ

Front end

Chip 0 Chip 1

Back end

GC-WRQ

GC-RDQ

Channel0

Chip 3 Queue

MQ

-SSD

Ho

st-S

ide

I/O

Req

ue

st Q

ueu

es

i

DRAM

Chip 0 Queue

Chip 2 Queue

Chip 1 Queue

FCCFCC

Chip 2 Chip 3Channel1

FCCFCC

Address Translation

Address Translation

Transaction Scheduling Unit (TSU)

Transaction Scheduling Unit (TSU)

Die

0

Plane0Plane1

Die

0

Plane0Plane1

Die

1

Plane0Plane1

Die

1

Plane0Plane1

Mu

ltiplexe

d

Interface

Mu

ltiplexe

d

Interface

Bu

s Interface

Bu

s Interface

Mu

ltiplexe

d

Interface

Bu

s Interface

Microprocessor

Request i, Page 1

Request i, Page M

Request i, Page 1

Request i, Page M

Figure 1: Internal organization of an MQ-SSD.

The back end contains multiple channels (i.e., request buses),where the channels can service I/O transactions in parallel.Each channel is connected to one or more chips of the underly-ing memory (e.g., NAND �ash memory). Each chip consists ofone or more dies, where each die contains one or more planes.Each plane can service an I/O transaction concurrently withthe other planes. In all, this provides four levels of parallelismfor servicing I/O transactions (channel, chip, die, and plane).

The front end manages the back end resources and issuesI/O transactions to the back end channels. There are threemajor components within the front end [37, 61, 101]: (1) thehost–interface logic (HIL), which implements the protocol usedto communicate with the host and fetches I/O requests fromthe host-side request queues in a round-robin manner [83];(2) the �ash translation layer (FTL), which manages SSD re-sources and processes I/O requests [10,13] using an embeddedprocessor and DRAM; and (3) the �ash channel controllers(FCCs), which are used to communicate between the otherfront end modules and the back end chips.

There are seven major steps in processing an I/O request.(1) The host generates the request and inserts the request intoa dedicated in-DRAM I/O queue on the host side, where each

2

queue corresponds to a single I/O �ow.1 (2) The HIL in theMQ-SSD fetches the request from the host-side queue andinserts it into a device-level queue within the SSD. (3) The HILparses the request and breaks it down into multiple transac-tions. (4) The address translation module in the FTL performsa logical-to-physical address translation for each transaction.(5) The transaction scheduling unit (TSU) [101, 102] in the FTLschedules each transaction for servicing [20, 102], and placesthe transaction into a chip-level software queue. (6) The TSUselects a transaction from the chip-level queue and dispatchesit to the corresponding FCC. (7) The FCC issues a commandto the target back end chip to execute the transaction.

In addition to processing I/O requests, the SSD controllermust perform maintenance operations. One such operationis garbage collection (GC) [10, 13, 16, 23]. Due to circuit-levellimitations, many of the underlying memories used in SSDsmust perform writes to a logical page out of place (i.e., the datais written to a new physical page). For example, in NAND �ashmemory, before new data can be written to a physical page,the page must �rst be erased. The erase operation can only beperformed at the granularity of a �ash block, which containshundreds of pages [10, 13]. Since many other pages in a blockmay still contain valid data at the time of the update, theblock containing the page that is being updated is typically noterased at that time, and the page is simply marked as invalid.Periodically, the GC procedure (1) identi�es candidate blocksthat are ready to be erased (i.e., blocks where most of the pagesare marked as invalid), (2) generates read/write transactionsto move any remaining valid data in each candidate block toanother location; and (3) generates an erase transaction foreach block.

The TSU is responsible for scheduling both I/O transactionsand GC transactions. For I/O transactions, it maintains aread queue (RDQ in Figure 1) and a write queue (WRQ) inDRAM for each back end chip. GC transactions are kept in aseparate set of read (GC-RDQ) and write (GC-WRQ) queuesin DRAM. The TSU typically employs a �rst-come �rst-servetransaction scheduling policy, and typically prioritizes I/Otransactions over GC transactions [51, 117]. To better exploitthe parallelism available in the back end, some state-of-the-artTSUs partially reorder the transactions within a queue [37,40].2.2. The Need for Fairness Control in an SSD

When multiple I/O �ows are serviced concurrently, the be-havior of the requests of one �ow can negatively interfere withthe requests of another �ow. In order to understand the impactof such inter-�ow interference, and how the scheduler a�ectsthis type of interference, we would like to quantify how fairlythe requests of di�erent �ows are being treated. We say thatan I/O scheduler is completely fair if concurrently-running�ows with equal priority experience the same slowdown dueto interference [19, 22, 70, 73, 84]. The slowdown Si of �ow ican be calculated as:

Si = RT sharedi /RTalone

i (1)where RT shared

i is the average response time of �ow i when itruns concurrently with other �ows, and RTalone

i is the same�ow’s average response time when it runs alone. The response

1In this work, we use the term �ow to indicate a series of I/O requeststhat (1) are generated by a single user process; or (2) originate from thesame host-side I/O queue (e.g., for systems running virtual machines, wherefairness should be enforced at the VM level [36], and where each VM can beassigned to a dedicated I/O queue). To simplify explanations without loss ofgenerality, we associate each �ow with a dedicated host-side I/O queue inthis work.

time of each request is measured from the time the requestis generated on the host side to the time the host receives aresponse from the SSD. We de�ne fairness (F ) [22] as the ratioof the maximum and minimum slowdowns experienced byany �ow in the system:

F = MINi{Si}/MAX

i{Si} (2)

As de�ned, 0 < F ≤ 1. A lower F value indicates a greaterdi�erence between the �ow that is slowed down the most andthe �ow that is slowed down the least, which is more unfair tothe �ow that is slowed down the most. In other words, highervalues of F are desirable.

We conduct a large set of real system experiments to studythe fairness of scheduling policies implemented in four realMQ-SSDs [31, 33, 65, 111] (which we call SSD-A through SSD-D), using workloads that capture a wide range of I/O �ow be-havior (see Section 6). Figure 2 shows the slowdowns and fair-ness for a representative workload where a moderate-intensityread-dominant �ow (generated by the tpce application; seeTable 2) runs concurrently with a high-intensity �ow withan even mix of reads and writes (generated by the tpcc ap-plication; see Table 2), as measured over a 60 s period. Wemake two key observations from the �gure. First, the amountof interference between the two �ows �uctuates greatly overthe period, which leads to high variability in the slowdownof each �ow. Across the four SSDs, tpce experiences slow-downs as high as 2.8x, 69x, 661x, and 11x, respectively, whileits average slowdowns are 2x, 4.4x, 106x, and 5.3x. Second, theSSDs that we study do not provide any fairness guarantees.We observe that tpcc does not experience high slowdownsthroughout the entire period. We conclude that modern MQ-SSDs focus on providing high performance at the expense oflarge amounts of unfairness, because they lack mechanismsto mitigate interference among �ows.

12345

Slo

wdo

wn

tpce tpcc

0.000.250.500.751.00

0 10 20 30 40 50 60

Fai

rnes

s

Time (s)

(a) SSD-A

15304560

Slo

wdo

wn

tpce tpcc

0.000.250.500.751.00

0 10 20 30 40 50 60

Fai

rnes

s

Time (s)(b) SSD-B

150300450600

Slo

wdo

wn tpce tpcc

0.000.250.500.751.00

0 10 20 30 40 50 60

Fai

rnes

s

Time (s)(c) SSD-C

369

12

Slo

wdo

wn tpce tpcc

0.000.250.500.751.00

0 10 20 30 40 50 60

Fai

rnes

s

Time (s)(d) SSD-D

Figure 2: Slowdown and fairness of �ows in a representativeworkload on four real MQ-SSDs.

3. An Analysis of Unfairness in MQ-SSDsAs Section 2.2 shows, real MQ-SSDs do not provide fair-

ness across multiple queues. This is because state-of-the-arttransaction scheduling units (TSUs) [20, 37, 40, 77, 90, 112] aredesigned to maximize the overall I/O request throughput ofthe entire SSD, and thus they do not take into account thethroughput of each �ow. Our goal in this work is to design ahigh-performance TSU for MQ-SSDs that delivers high SSDthroughput while providing fairness for each �ow.

In order to meet this goal, we must (1) identify the runtimecharacteristics of each I/O �ow, and (2) understand how these

3

characteristics lead to unfairness. An I/O �ow is made up of astream of I/O requests, where each request is de�ned by itsarrival time, starting address, size of the data that it requires,and access type (i.e., read or write) [42, 79, 119]. Based on therequests that make up a particular �ow, we de�ne four keycharacteristics of the �ow: (1) I/O intensity, (2) access pattern,(3) read/write ratio of the requests in the �ow, and (4) garbagecollection (GC) demands of the �ow. In this section, we studyhow di�erences in each of these characteristics can lead tounfairness, using the methodology described in Section 6.3.1. Flows with Di�erent I/O Intensities

The I/O intensity of a �ow refers to the rate at which the�ow generates transactions. This is dependent on the arrivaltime and the size of the I/O requests (as larger I/O requestsgenerate a greater number of page-sized transactions). Thereare three factors that contribute to the amount of time requiredto service each transaction, which is known as the transactionturnaround time: (1) the wait time at the chip-level queue,(2) the transfer time between the front end and the back end,and (3) the time required to execute the transaction in thememory chip. As the intensity of a �ow increases, the transfertime and execution time do not change, but the wait time atthe chip-level queue grows.

Figure 3 shows how �ow intensity a�ects chip-level queuelength and average transaction turnaround time. We run a100%-read synthetic �ow where we vary the I/O intensity from16 MB/s to 1024 MB/s. Figures 3a and 3b show, respectively,the average length of the chip-level queues and the breakdownof the transaction turnaround time with respect to the I/Ointensity. We make three conclusions from the �gures. First,the lengths of the chip-level queues increase proportionatelywith the I/O intensity of the running �ow. Second, a longerqueue leads to a longer wait time for transactions. Third, asthe �ow intensity increases, the contribution of the queue waittime to the transaction turnaround time increases, dominatingthe other two factors (shown in the �gure).

1

10

102

103

104

105

106

16 32 64 128 256 512 1024

Nor

mal

ized

Chi

p-le

vel

Que

ue L

engt

h

Flow I/O Intensity (MB/s)

0

40

80

120

160

200

240

16 32 64 128 256 512 1024Avg

. Fla

sh T

rans

actio

nT

urna

roun

d T

ime

(μs)


Flash OperationBus Transfer

Chip Queue Waiting

3678

(a) Chip-level queue length

1

10

102

103

104

105

106

16 32 64 128 256 512 1024

Nor

mal

ized

Chi

p-le

vel

Que

ue L

engt

h


0

40

80

120

160

200

240

16 32 64 128 256 512 1024Avg

. Fla

sh T

rans

actio

nT

urna

roun

d T

ime

(μs)


Flash OperationBus Transfer

Chip Queue Waiting

3678

(b) Transaction turnaround time

Figure 3: E�ect of �ow I/O intensity.When two �ows with di�erent I/O intensities execute con-

currently, each �ow experiences an increase in the averagelength of the chip-level queues, which can more adverselya�ect the �ow with a low I/O intensity than the �ow with ahigh I/O intensity. To characterize the e�ect, we execute alow-intensity 16 MB/s synthetic �ow, referred to as the base�ow, together with a synthetic �ow of varying I/O intensities,referred to as the interfering �ow. The I/O requests of the two�ows are purely reads. Figure 4 depicts the slowdown and fair-ness results. The x-axis shows the intensity of the interfering�ow. We observe that even when the intensity of the inter-fering �ow exceeds 256 MB/s, the slowdown of the interfering�ow is very small, while the base �ow is slowed down by asmuch as 49x. The unfair slowdown of the base �ow is due to adrastic increase in the average length of the chip-level queues

compared to when the base �ow runs alone. We conclude thatthe average response time of a low-intensity �ow substantiallyincreases due to interference from a high-intensity �ow.3.2. Flows with Di�erent Access Patterns

The access pattern of a �ow determines how its transactionsare distributed across the chip-level queues. Some �ows cantake advantage of the parallelism available in the back endof the SSD, by distributing their transactions across all ofthe chip-level queues equally. Other �ows do not bene�t asmuch from the back-end parallelism, as their transactions aredistributed unevenly across the chip-level queues. The accesspattern depends on the starting addresses and sizes of eachrequest in the �ow.

To analyze the interference between concurrent �ows withdi�erent access patterns, we run a base �ow with a stream-ing access pattern [50, 70] together with an interfering �owwhose access pattern changes periodically between streamingand random (i.e., we can control the fraction of time the �owperforms random accesses). Figure 5 shows the slowdown ofthe two �ows and the fairness of the SSD. The x-axis showsthe randomness of the interfering �ow. We observe that asthe randomness of the interfering �ow increases, the MQ-SSDbecomes more unfair to the base �ow. Both the probabilityof resource contention and the transaction turnaround timeincrease when the two �ows are executed together. However,the base �ow, which highly exploits and bene�ts from chip-level parallelism, is more susceptible to interference from theinterfering �ow. The uneven distribution of transactions fromthe interfering �ow causes some, but not all, of the base �ow’stransactions to stall. As a result, the base �ow’s parallelismgets destroyed, and the �ow slows down until its last trans-action is serviced. We conclude that �ows with parallelism-friendly access patterns are susceptible to interference from �owswith access patterns that do not exploit parallelism.

0

0.6

1.2

1.8

2.4

3

0% 25% 50% 75% 100% 0

0.2

0.4

0.6

0.8

1

Slo

wdo

wn

Fai

rnes

s

Randomness ofthe Interfering Flow

Base flow Interfering flow Fairness

0.0

0.6

1.2

1.8

2.4

3.0

16 32 64 12825651210240.0

0.2

0.4

0.6

0.8

1.0

Slo

wdo

wn

Fai

rnes

s

Intensity ofthe Interfering Flow (MB/s)

Base flowInterfering flow

Fairness

49

Figure 4: E�ects of concur-rently executing two �owswith di�erent I/O intensitieson fairness.

0.0

0.6

1.2

1.8

2.4

3.0

0% 25% 50% 75% 100%0.0

0.2

0.4

0.6

0.8

1.0S

low

dow

n

Fai

rnes

s

Randomness ofthe Interfering Flow

Base flow Interfering flow Fairness

Figure 5: E�ects of con-currently executing two�ows with di�erent accesspatterns on fairness.

3.3. Flows with Di�erent Read/Write RatiosAs reads are 10-40x faster than writes, and are more likely

to fall on the critical path of program execution, state-of-the-art MQ-SSD I/O schedulers tend to prioritize reads over writesto improve overall performance [20, 112]. To implement readprioritization, TSUs mainly adopt two approaches: (1) trans-actions in the read queue are always prioritized over those inthe write queue, and (2) a read transaction can preempt anongoing write transaction.

Unfortunately, read prioritization increases the wait timeof write transactions, and can potentially slow down write-dominant �ows in an MQ-SSD [84]. Figure 6 compares theresponse time and fairness of an SSD using a read-prioritized

4

scheduler (RP) with that of an SSD with a �rst-come �rst-serve scheduler (FCFS), which processes all transactions inthe order they are received. In this experiment, we use twoconcurrently-running synthetic �ows, where we control therate at which the �ows can issue new requests to the SSDby limiting the size of the host-side queue. We sweep thewrite rate of the base �ow (0%, 30%, 70%, and 100%), shownas di�erent curves, and set the host-side queue length to onerequest. We set the host-side queue length for the interfering�ow to six requests (i.e., it has 6x the request rate of thebase �ow) to increase the probability of interference, andwe sweep the interfering �ow’s read ratio from 0% to 100%.In Figures 6a and 6b, the x-axis marks the read ratio of theinterfering �ow. We make three observations from the �gures.First, RP maintains or improves the average response time forall �ow con�gurations over FCFS. Second, RP leads to lowerfairness than FCFS when the interfering �ow is heavily read-dominant (i.e., its read ratio > 80%). Third, fairness in an SSDwith RP increases when the read ratio of the interfering �owdecreases. We conclude that when concurrently-running �owshave di�erent read/write ratios, existing scheduling policies arenot e�ective at providing fairness.

0%

10%

20%

30%

40%

50%

60%

0% 20% 40% 60% 80% 100%

Res

pons

e T

ime

Impr

ovem

ent

Read Ratio ofthe Interfering Flow

-75%

-50%

-25%

0%

25%

50%

75%

100%

0% 20% 40% 60% 80% 100%Fai

rnes

s Im

prov

emen

t

Read Ratio ofthe Interfering Flow

100% WR in base flow70% WR in base flow

30% WR in base flow0% WR in base flow

(a) Response time improvement ofRP compared to FCFS

(b) Fairness improvement of RPcompared to FCFS

Figure 6: E�ects of read prioritization.

3.4. Flows with Di�erent GC DemandsGC is one of the major sources of performance �uctuation in

SSDs [26,27,38,41,51,54,108]. The rate at which GC is invokeddepends primarily on the write intensity and access pattern ofthe �ow [107]. A �ow that causes GC to be invoked frequently(high-GC) can disproportionately slow down a concurrently-running �ow that does not invoke GC often (low-GC). Priorworks that optimize GC performance [26,27,38,41,51,54,108]can reduce interference between GC operations and trans-actions from I/O requests, and thus can improve fairness.However, these works cannot completely eliminate interfer-ence between a high-GC �ow and a low-GC �ow that occurswhen a high-GC �ow results in a write cli� (i.e., when thehigh intensity of the write requests causes GC to be invokedfrequently, resulting in a large drop in performance [39]).

Figure 7 shows the e�ect of GC execution on MQ-SSDfairness. We run a base �ow with moderate GC demands (i.e.,8 MB/s write rate) and an interfering �ow where we change theGC demands by varying the write rate (2 MB/s to 64 MB/s). Weevaluate three di�erent scenarios for GC: (1) no GC execution,assuming the MQ-SSD capacity is in�nite; (2) default (i.e.,non-preemptive) GC execution; and (3) semi-preemptive GC(SPGC) [51,52] execution. SPGC tries to mitigate the negativee�ect of GC on user I/O requests by allowing user transactionsto preempt GC activities when the number of free pages in theSSD is above a speci�c threshold (GChard ). Figure 7a shows

the fraction of time spent on I/O requests and on GC whenthe base �ow (the leftmost column) and the interfering �ow(all other columns) are executed alone. Figures 7b, 7c, and7d show the slowdown of the base �ow, the slowdown of theinterfering �ow, and the fairness of the MQ-SSD, respectively.The x-axis in these �gures represents the write intensity ofthe interfering �ow.

0%

20%

40%

60%

80%

100%

base flow 2 4 8 16 32 64

Fla

sh C

hip

Sta

tus

Bre

akdo

wn

Intensity ofInterfering Flow (MB/s)

User I/OGC ActivitiesIdle

(a) Breakdown of chip statusover the entire simulation time

0

1

2

3

4

2 4 8 16 32 64

Slo

wdo

wn

Intensity of the Interfering Flow (MB/s)

No GCDefault GCSPGC

0

1

2

3

4

2 4 8 16 32 64

Slo

wdo

wn


No GCDefault GCSPGC

0.0

0.2

0.4

0.6

0.8

1.0

2 4 8 16 32 64

Fai

rnes

s


No GCDefault GCSPGC

(b) Slowdown of base �ow

0

1

2

3

4

2 4 8 16 32 64

Slo

wdo

wn


No GCDefault GCSPGC

0

1

2

3

4

2 4 8 16 32 64S

low

dow

nIntensity of the Interfering Flow (MB/s)

No GCDefault GCSPGC

0.0

0.2

0.4

0.6

0.8

1.0

2 4 8 16 32 64

Fai

rnes

s


No GCDefault GCSPGC

(c) Slowdown of interfering �ow

0

1

2

3

4

2 4 8 16 32 64

Slo

wdo

wn


No GCDefault GCSPGC

0

1

2

3

4

2 4 8 16 32 64

Slo

wdo

wn


No GCDefault GCSPGC

0.0

0.2

0.4

0.6

0.8

1.0

2 4 8 16 32 64

Fai

rnes

s


No GCDefault GCSPGC

(d) Fairness

Figure 7: E�ects of GC demands on fairness.We make four key observations. First, the GC activities

increase proportionately with write intensity. Second, as thewrite intensity of the interfering �ow increases, which triggersmore GC operations, fairness decreases. Third, when thedi�erence between the GC demands of the base �ow and theinterfering �ow increases, the default GC mechanism leads to ahigher slowdown of the base �ow, degrading fairness. Fourth,SPGC improves fairness over the default GC mechanism, butonly by a small amount, as SPGC disables preemption whenthere are a large number of pending writes. We conclude thatthe GC activities of a high-GC �ow can unfairly block �ashtransactions of a low-GC �ow.4. FLIN: Fair, High-Performance Scheduling

As Section 3 shows, the four sources of interference thatslow down �ows all result from interference among the trans-actions that make up each �ow. Based on this insight, wedesign a new MQ-SSD transaction scheduling unit (TSU)called the Flash-Level INterference-aware scheduler (FLIN).FLIN schedules and reorders transactions in a way that sig-ni�cantly reduces each of the four sources of interference, inorder to provide fairness across concurrently-running �ows.

FLIN consists of three stages, as shown in Figure 8. The �rststage, fairness-aware queue insertion, inserts each new trans-action from an I/O request into the appropriate read/writequeue (Section 4.1). The position at which the transaction isinserted into the queue depends on the current intensity andaccess pattern of the �ow that corresponds to the transaction,such that (1) a low-priority �ow is not unfairly slowed down,and (2) a �ow that can take advantage of parallelism in theback end does not experience interference that underminesthis parallelism. The second stage, priority-aware queue arbi-tration, uses priority levels that are assigned to each �ow bythe host to determine the next read request and the next writerequest for each back end memory chip (Section 4.2). The ar-biter aims to ensure that all �ows assigned to the same prioritylevel are slowed down equally. The third stage, wait-balancingtransaction selection, selects the transaction to dispatch to the

5

�ash chip controllers (FCCs, which issue commands to thememory chips in the back end; see Section 2.1) in a way thatalleviates the interference introduced by read/write ratio dis-parities among �ows and by garbage collection (Section 4.3).To achieve this, the scheduler chooses to dispatch a read trans-action, a write transaction, or a garbage collection transaction,in a way that (1) balances the slowdown for read transactionswith the slowdown of write transactions, and (2) distributesthe overhead of garbage collection transactions fairly acrossall �ows. FLIN is designed to be easy to implement, with lowoverhead, as we describe in Section 5.

Stag

e 1

Fair

ne

ss-a

war

eQ

ue

ue

Inse

rtio

n

Chip 0 Queue

Chip 1 Queue

Chip 2 Queue

Chip 3 Queue

RDQ WRQ

QP

Stag

e 2

Pri

ori

ty-a

war

eQ

ueu

e A

rbit

rati

on Chip 0

QueueChip 1 Queue

Chip 2 Queue

Chip 3 Queue

GC-WRQ

Write Slot

Read Slot

GC-RDQ Stag

e 3

Wai

t-b

alan

cin

gTr

ansa

ctio

n S

ele

ctio

n

Q1

Q2

QP

Q1

Q2

Flas

h T

ran

sact

ion

s

FCC

FCC

DRAM DRAM

Figure 8: High-level overview of FLIN.

4.1. Fairness-Aware Queue InsertionWe design fairness-aware queue insertion, the �rst stage of

FLIN, to relieve interference that occurs due to the intensityand access pattern of concurrently-running �ows. Within the�rst stage, FLIN maintains separate per-chip read and writequeues (RDQ and WRQ in Figure 8, respectively) for each�ow priority level, since priorities are not processed until thesecond stage of FLIN. Fairness-aware queue insertion uses aheuristic O(t) algorithm to determine where each transactionis inserted within the queue that corresponds to the accesstype (i.e., read or write) of the transaction and the prioritylevel of the �ow that generated the transaction. Finding theoptimal ordering of transactions in each queue is a variantof the multidimensional assignment problem [85], which isNP-hard [82]. FLIN uses a heuristic that approximates theoptimal ordering, in order to bound the time required for the�rst stage and to minimize the fraction of the transactionturnaround time that is spent on scheduling.

Algorithm 1 shows the heuristic used to insert a new trans-action (TRnew) into its corresponding chip-level queue (Q).The heuristic is based on three key insights: (1) the perfor-mance of a low-intensity �ow is very sensitive to the increasedqueue length when a high-intensity �ow inserts many trans-actions (Section 3.1); (2) some �ows can change between beinghigh-intensity and low-intensity, based on the current phaseof the application [80]; and (3) some, but not all, of the transac-tions from a �ow that take advantage of back-end parallelismexperience high queue wait times when a �ow with poorparallelism inserts requests into some, but not all, of the chip-level queues (Section 3.2). The algorithm consists of two steps,which we describe in detail.

Step 1: Prioritize Low-Intensity Flows. Transactionsfrom a low-intensity �ow are always prioritized over transac-tions from a high-intensity �ow. Figure 9a shows an examplesnapshot of a chip-level queue when FLIN inserts a newly-arrived transaction (TRnew) into the queue. The heuristicchecks to see whether TRnew belongs to a low-intensity �ow(line 9 of Algorithm 1; see Section 5.1 for details). If TRnew isfrom a low-intensity �ow, the heuristic inserts TRnew into thequeue immediately after Lastlow , the location of the farthesttransaction from the head of the queue that belongs to a low-intensity �ow (line 10), and advances to Step 2a (line 12). If

Algorithm 1 Fairness-aware queue insertion in FLIN.1: Inputs:2: Q: the chip-level queue to insert the new transaction3: TRnew : the newly arriving �ash transaction4: Source: source �ow of the new transaction5: Fthr : fairness threshold for high-intensity �ows6:7: Lastlow ← farthest transaction from head of Q belonging to a low-intensity �ow8: // Step 1: Prioritize Low-Intensity Flows9: if Source is a low-intensity �ow then // more detail in Section 5.1

10: Insert(TRnew after Lastlow )11: // Step 2a: Maximize Fairness Among Low-Intensity Flows.12: Estimate waiting time of TRnew as if Source were running alone // Section 5.213: Reorder transactions of low-intensity �ows in Q for fairness // Section 5.314: else15: Insert(TRnew at the tail of Q)16: // Step 2b: Maximize Fairness Among High-Intensity Flows.17: Estimate waiting time of TRnew as if Source were running alone18: F← Estimate fairness from average slowdown of departed transactions (Q)19: if F < Fthr and Source has experienced the maximum slowdown then20: Move TRnew to Lastlow + 121: else Reorder transactions from the tail of Q to Lastlow + 1 for fairness

Tail Head

Chip-level Queue (Q)

TRnewTRnewTRnewTRnew

Chip-level Queue (Q) TRnewTRnew


If Source of TRnewis high-intensity

If Source of TRnewis low-intensity

Reorder transactions to improve fairness


TR1TR2TR3

TR1TR2TR3

TR1TR2TR3

TR4TR5TR6TR8TR9TR10TR11



Lastlow

Transactions fromhigh-intensity flows

Transactions fromlow-intensity flows

(a) Step 1 of Algorithm 1

Tail Head









TR1TR2TR3

TR1TR2TR3

TR1TR2TR3




Lastlow



(b) Step 2a of Algorithm 1

Tail Head









TR1TR2TR3

TR1TR2TR3

TR1TR2TR3




Lastlow



(c) Step 2b of Algorithm 1Figure 9: Fairness-aware queue insertion in FLIN.

TRnew belongs to a high-intensity �ow, the heuristic insertsthe transaction at the tail of the queue (line 15), and advancesto Step 2b (line 17).

Step 2a: Maximize Fairness Among Low-IntensityFlows. The heuristic begins this step by estimating how longTRnew would have to wait to be serviced if its source �owwere running alone (line 12). This estimate is also used todetermine a transaction’s slowdown and is kept alongside thetransaction in the queue. The heuristic then reorders transac-tions of low-intensity �ows in the queue for fairness (line 13).As Figure 9b shows, this involves moving TRnew from the tailof the low-intensity-�ow transactions closer to the head, inorder to maximize the ratio of the minimum slowdown tomaximum slowdown for all pending transactions from low-priority �ows. This reordering step prioritizes a transaction(i.e., moves it closer to the queue head) that has a greaterslowdown, which indicates that its source �ow either (1) has alower intensity, or (2) better utilizes the parallelism availablein the MQ-SSD back end. We describe the details of the alonewait time estimation (line 12) and the reordering step (line 13)in Sections 5.2 and 5.3, respectively.

Step 2b: Maximize Fairness Among High-IntensityFlows. Figure 9c shows a general overview of Step 2b. Theheuristic begins this step by estimating the alone wait time ofTRnew (line 17), using the same procedure in Step 2a. However,unlike the low-intensity-�ow transactions, this step performsslowdown-aware queue shu�ing in order to prevent any onehigh-intensity �ow from being treated much more unfairlythan the other high-intensity �ows. After a new high-intensitytransaction is inserted into the queue, the heuristic checks thecurrent fairness (line 18). If the fairness is lower than a prede-

6

termined threshold (Fthr ), and the new transaction belongs tothe �ow with the maximum slowdown (line 19), the heuristicmoves the transaction ahead of all other high-intensity-�owtransactions (line 20). This approach can often help �ows thatare transitioning from low-intensity to high-intensity, as such�ows are particularly susceptible to interference from otherhigh-intensity �ows. If fairness is above the threshold, whichmeans that all high-intensity �ows are being treated some-what fairly, the heuristic only reorders the high-intensity-�owtransactions for fairness (line 21).4.2. Priority-Aware Queue Arbitration

Many host–interface protocols, such as NVMe [83], allowthe host to assign di�erent priority levels to each �ow. Thesecond stage of FLIN enforces these priorities, and ensuresthat �ows at the same priority level are slowed down by anequal amount. As we discuss in Section 4.1, FLIN maintains aread and write queue for each priority level (i.e., if there are Ppriority levels as de�ned by the protocol, FLIN includes P readqueues and P write queues in the DRAM of the SSD, for each�ash chip). Priority-aware queue arbitration selects one readyread transaction from the transactions at the head of the Pread queues, and one ready write transaction from the trans-actions at the head of the P write queues. The two selectedtransactions then move to the last stage of the scheduler.

Whenever more than one queue has a transaction at thequeue head that is ready to execute (i.e., the back end re-sources it requires are not being used by another transaction),the queue arbiter uses a weighted round-robin policy [76] toselect the read and write transactions that move to the nextscheduling stage. If the protocol de�nes P priority levels,where level 0 is the lowest priority and level P – 1 is thehighest priority, FLIN assigns the weight 2i to priority leveli. Under weighted round-robin, this means that out of every∑

i 2i scheduling decisions in the second stage, transactionsin the priority level i queue receive 2i of these slots.

4.3. Wait-Balancing Transaction SelectionWe design wait-balancing transaction selection, the last

stage of FLIN, to minimize interference that occurs due to theread/write ratios (Section 3.3) and garbage collection demandsof concurrently-running �ows (Section 3.4). A transactionstalls if the back-end resources that it needs are being used byanother transaction (which can be a transaction from another�ow, or a garbage collection transaction). Wait-balancingtransaction selection attempts to distribute this stall timeevenly across all read and write transactions.

Wait-balancing transaction selection chooses one of the fol-lowing transactions to dispatch to the FCC: (1) the ready readtransaction determined by priority-aware queue arbitration(Section 4.2), which is stored in a read slot in DRAM; (2) theready write transaction determined by priority-aware queuearbitration, which is stored in a write slot in DRAM; (3) thetransaction at the head of the garbage collection read queue(GC-RDQ); and (4) the transaction at the head of the garbagecollection write queue (GC-WRQ). Algorithm 2 depicts theselection heuristic, which consists of two steps.

Step 1: Estimate the Proportional Wait. FLIN uses anovel approach to determine when to prioritize reads overwrites, which we call proportional waiting. Previous schedul-ing techniques [20, 112] always prioritize reads over writes,which as we show in Section 3.3 leads to unfairness. Propor-tional waiting avoids this unfairness.

Algorithm 2 Wait-balancing transaction selection in FLIN.1: Inputs:2: ReadSlot: the read transaction waiting to be issued to the SSD back end3: WriteSlot: the write transaction waiting to be issued to the SSD back end4: GC-RDQ, GC-WRQ: the garbage collection read and write queues5:6: PW read ← EstimateProportionalWait (ReadSlot) // Section 5.47: PWwrite← EstimateProportionalWait (WriteSlot) // Section 5.48: if PW read > PWwrite then9: Dispatch ReadSlot to FCC (�ash chip controller)

10: else11: if number of free pages < GCFLIN then12: GCMigrationCount← AssignGCMigrations(WriteSlot) // Section 5.413: while GCMigrationCount > 0 do14: Dispatch the transaction at the head of GC-RDQ to FCC15: Dispatch the transaction at the head of GC-WRQ to FCC16: GCMigrationCount← GCMigrationCount - 117: Dispatch WriteSlot to FCC

We de�ne the proportional wait time (PW ) of a transactionas the ratio of its wait time in the scheduler (Twait ), from thetime that the transaction is received by the scheduler untilthe time the transaction is dispatched to the FCC, over thesum of the time required to perform the operation in thememory (Tmemory) and transfer data back to the front end(Ttransfer ). The transaction in the read slot is prioritized overthe transaction in the write slot when the read transaction’sproportional wait time is greater than the write transaction’sproportional wait time (lines 6–8 in Algorithm 2).

Step 2: Dispatch Transactions. If the read transaction isprioritized, it is dispatched to the �ash channel controller (seeSection 2.1) right away (line 9). If the write transaction is pri-oritized, the scheduler then considers also dispatching garbagecollection operations, since a write transaction takes signi�-cantly longer to complete than a read transaction and the costof garbage collection operations can be amortized by serv-ing them together with the selected write transaction. If thenumber of free pages available in the memory is lower than apre-determined threshold (GCFLIN ), and there are garbage col-lection transactions waiting in the GC-RDQ or GC-WRQ, thescheduler (1) determines how many of these transactions toperform, which is represented by GCMigrationCount (line 12),and (2) issues GCMigrationCount transactions (lines 13–16).Since garbage collection transactions read a valid page andwrite the page somewhere else in the memory, FLIN always ex-ecutes a pair of read and write transactions. Once the garbagecollection transactions are done, the scheduler dispatches thewrite transaction (line 17).

Optimizations. FLIN employs two optimizations beyondthe basic read-write wait-balancing algorithm. First, FLINsupports write transaction suspension [112] whenever the pro-portional wait time (PW) of the transaction in the read slot islarger than the PW of the currently-executing write operation.Second, when the read and write slots are empty, FLIN dis-patches a pair of garbage collection read/write transactions.If a transaction arrives at the read slot when the garbage col-lection transactions are being executed, FLIN preempts thegarbage collection write and executes the transaction at theread slot, to avoid unnecessarily stalling the incoming read.5. Implementation of FLIN

To realize the high-level behavior of FLIN (Section 4), weneed to implement a number of functions, particularly forStages 1 and 3 of the scheduler. We discuss each function indetail in this section, and then discuss the overhead requiredto implement FLIN in an MQ-SSD.

7

5.1. Classifying Flow I/O IntensityThe �rst stage of FLIN dynamically categorizes �ows as

either low-intensity or high-intensity, with separate classi�-cations for the �ow’s read behavior and its write behavior.FLIN performs this classi�cation over �xed-length intervals.During each interval, FLIN records the number of read andwrite transactions enqueued by each �ow. At the end of theinterval, FLIN uses the transaction counts to calculate the readand write arrival rates of each �ow. If the read arrival rate ofa �ow is lower than a predetermined threshold αreadC , FLINclassi�es the �ow as low-intensity for reads; otherwise, FLINclassi�es the �ow as high-intensity for reads. FLIN performsthe same classi�cation for write intensity using the write ar-rival rate of the �ow, and a predetermined threshold αwriteC .We empirically set the epoch length to 10 ms, which we �ndis short enough to detect changes in �ow intensity, but longenough to capture di�erences in intensity among the �ows.5.2. Estimating Alone Wait Time and Slowdown

The �rst stage of FLIN estimates the alone wait time ofeach transaction when the transaction is enqueued. After-wards, whenever the transactions are reordered for fairnessin the �rst stage (see Section 4.1), FLIN uses the alone waittime estimate to predict how much each individual trans-action will be slowed down due to �ow-level interference.The slowdown of an individual transaction, STR, is calcu-lated as STR = TTR

shared/TTRalone , where TTR

shared is the transactionturnaround time when the �ow corresponding to the trans-action runs concurrently with other �ows, and TTR

alone is thetransaction turnaround time when the �ow runs alone. Forboth the shared and alone turnaround times, FLIN estimatesTTR using the following equation:

TTR = TTRmemory + TTR

transfer + TTRwait (3)

TTRmemory and TTR

transfer are the times required to execute a com-mand in the memory chip and transfer the data between theback end and the front end, respectively. Both values are con-stants that are determined by the speed of the componentswithin the MQ-SSD. TTR

wait is the amount of time that the trans-action must wait in the read or write queue before it is issuedto the back end.

To estimate TTRalone , FLIN must estimate TTR

wait_alone , which isthe alone wait time. If the �ow were running alone, the onlyother requests that the transaction would wait on are otherrequests in the same queue from the same �ow. As a result,FLIN uses the last request inserted into the same queue bythe same �ow to determine how long the current transactionwould have to wait if the other �ows were not running. Thiscan be determined by calculating how much longer the lastrequest needs to complete:

TTRwait_alone = T last

enqueue + T lastalone – Tnow (4)

Essentially, we take the timestamp at which the last re-quest was enqueued (T last

enqueue), add its estimated transactionturnaround time (T last

alone) to determine the timestamp at whichthe request is expected to �nish, and subtract the currenttimestamp (Tnow) to determine the remaining time needed tocomplete the request. If the queue contains no other transac-tions from the same �ow when the transaction arrives, thenthe transaction would not have to wait if the �ow was runalone, so FLIN sets TTR

wait_alone to 0.

To estimate TTRshared , FLIN estimates TTR

wait_shared based onthe current position of the transaction in the queue. For atransaction sitting at the p-th position in the queue (wherethe request at the head of the queue is at position 1), FLINuses the following equation:

TTRwait_shared =Tnow – Tenqueued + Tchip_busy

+p–1∑i=1

(T imemory + T i

transfer

) (5)

Tnow – Tenqueued gives the time that has elapsed since thetransaction was enqueued (i.e., how long the transaction hasalready waited for). To compute TTR

wait_shared , FLIN adds thetime elapsed so far to the amount of time left to service thecurrently-executing transaction (Tchip_busy), and the time re-quired to execute and transfer data for all transactions aheadin the queue that have not yet been issued to the back end.

While FLIN calculates T lastalone only once (when a transaction

is �rst enqueued), it calculates TTRalone and TTR

shared every timethe slowdown of an individual transaction needs to be usedduring the �rst stage.5.3. Reordering Transactions for Fairness

When the �rst stage of FLIN reorders transactions within aregion of a transaction queue for fairness (in Steps 2a and 2bin Section 4.1), it scans through all transactions in the regionto �nd a position for the newly-arrived transaction (TRnew)that maximizes the ratio of minimum to maximum slowdownof the queued transactions. This requires FLIN to performtwo passes over the transactions within the queue region.

During the �rst pass, FLIN estimates the slowdown of eachtransaction in the region (see Section 5.2). Then, it calculatesfairness across all enqueued requests by using Equation 2 overthe per-transaction slowdowns.

During the second pass, FLIN starts with the last transactionin the queue region (i.e., the one closest to the queue tail), anduses the slowdown to determine the insertion position forthe newly-arrived transaction that is expected to increasefairness by the largest amount. To do this, at every position p,FLIN estimates the change in fairness by recalculating theslowdown only for (1) the newly-arrived transaction, if itwere to be inserted into position p; and (2) the transactioncurrently at position p, if it were moved to position p–1. FLINkeeps track of the position that leads to the highest fairnessas it checks all of the positions in the queue region. After thesecond pass is complete, FLIN moves TRnew to the positionthat would provide the highest fairness.

With this two-pass approach, if TRnew is from a �ow thathas only a few transactions in the queue (e.g., the �ow islow-intensity), the reordering moves the transaction closerto the head of the queue. Hence, FLIN signi�cantly reducesthe slowdown of the transaction, making its slowdown valuemore balanced with respect to transactions from other �ows.5.4. Estimating Proportional Wait Time

The third stage of FLIN chooses whether to dispatch thetransaction in the read slot or the transaction in the write slotto the �ash chip controller (FCC), by selecting the transactionthat has the higher proportional wait time. In Section 4.3, wede�ne the proportional wait time (PW ) as:

PW = Twait ÷ (Tmemory + Ttransfer ) (6)

8

Tmemory and Ttransfer are the execution and data transfer time,respectively, of a read or write transaction that is issued tothe back end of the SSD. Twait represents how long the totalwait time of the transaction would be, starting from the timethe host made the I/O request that generated the transaction,if FLIN chooses to issue the transaction in the other slot �rst.In other words, if FLIN is evaluating PW for the transactionin the read slot, it calculates how long that transaction wouldhave to wait if the transaction in the write slot were performed�rst. We can calculate Twait as:

Twait = Tnow – Tenqueued + Tother_slot + TGC (7)Tnow – Tenqueued gives the amount of time that the transac-tion has already waited for, Tother_slot is the time required toperform the operation waiting in the other slot �rst, and TGCis the amount of time required to perform pending garbagecollection transactions. Recall from Algorithm 2 (Section 4.3)that if the transaction in the read slot is selected, no garbagecollection transactions are performed, so TGC = 0. TGC canbe non-zero if the transaction in the write slot is selected.

To determine Tother_slot , we add the time needed to exe-cute the transaction and transfer data together with the timeneeded for any garbage collection transactions that would beperformed along with the transaction based on Algorithm 2:

Tother_slot = Tmemory + Ttransfer + TGC (8)In Equations 7 and 8, TGC = 0 if either (1) the transaction

being considered is a read, or (2) there are no pending garbagecollection operations. Otherwise, TGC is estimated as:

TGC =GCM∑i=1

(2Ttransfer + T readmemory + Twrite

memory) + T erasememory (9)

In this equation, GCM stands for GCMigrationCount, whichis the number of page migrations that should be performedalong with the transaction in the write slot to fairly distributethe interference due to garbage collection (Section 3.4). Re-call from Section 2.1 that for many SSDs, such as those thatuse NAND �ash memory, garbage collection occurs at thegranularity of a block. A block that is selected for garbagecollection may still contain a small number of valid pages,which must be moved to a new block before the selected blockcan be erased. For each valid page, this requires the SSD to(1) execute a read command on the memory chip containingthe old physical page (which takes T read

memory time), (2) two datatransfers between the back end and the front end for GC readand write operations (2Ttransfer ), and (3) execute a write com-mand on the memory chip containing the new physical page(Twrite

memory). Hence, for each page migration, we add the timerequired to perform these four steps. If all GC page migrationsfrom a block are �nished, FLIN adds in the time required toperform the erase operation on the block (T erase

memory).FLIN determines the number of GC page movements (GCM)

that should be executed ahead of a write transaction gener-ated by �ow f , based on the fraction of all garbage collectionoperations that were were previously caused by the �ow. Thekey insight is to choose a GCM value that distributes a largernumber of GC operations to a �ow that is responsible forcausing more GC operations. FLIN estimates GCM by deter-mining (1) the fraction of all writes that were generated by�ow f , since write operations lead to garbage collection (seeSection 2.1); and (2) the fraction of valid pages that belong to�ow f in a block that is selected for garbage collection (which

FLIN approximates by using the fraction of all currently-validpages in the SSD that were written by �ow f ), since each validpage in the selected block requires a GC operation:

GCM =NumWritesf∑i NumWritesi

×Validf∑i Validi

× lengthGC–RDQ (10)

where NumWritesf is the number of write operations per-formed by �ow f since the last garbage collection operation,Validf is the number of valid pages in the SSD for �ow f(which is determined using the history of valid pages that ismaintained by the FTL page management mechanism [10,13]),and lengthGC–RDQ is the number of queued garbage collec-tion read transactions (which represents the total number ofpending garbage collection migrations).

While FLIN can use �ne-grained data separation techniquesproposed for multi-stream SSDs [41] for more accurate GCcost estimation, we �nd that our estimates from Equation 10are accurate enough for our purposes.5.5. Implementation Overhead and Costs

FLIN can be implemented in the �rmware of a modernSSD, and does not require specialized hardware. The re-quirements of FLIN are very modest compared to the pro-cessing power and DRAM capacity of a modern SSD con-troller [10–12, 14, 33, 37, 57–61, 104]. FLIN does not interferewith any SSD management tasks [10], such as the GC candi-date block selection policy [23,53,107], the address translationscheme, or other FTL maintenance and reliability enhance-ment tasks, and hence it can be adopted independently of SSDmanagement tasks.DRAM Overhead. FLIN requires only a small amount of

space within the in-SSD DRAM to store four pieces of informa-tion that are not kept by existing MQ-SSD schedulers: (1) theread and write arrival rates of each �ow in the last interval(Section 5.1), which requires a total of 2× #�ows × #chips 32-bit words; (2) the alone wait time of each transaction waitingin the chip-level queues (Section 5.2), which requires a totalof max_transactions 32-bit words; (3) the average slowdownof all high-intensity �ows in each queue (Section 4.1, Step 3),which requires 2× #�ows × #chips 32-bit words; (4) the GCcost estimation data (i.e., Countiwrite and Validi; Section 5.4),which requires 2 × #�ows 32 bit variables. For an SSD thathas 64 �ash chips and 2 GB of DRAM [61] (resulting in a veryconservative maximum of 262,144 8 kB transactions that theDRAM can hold), and that supports a maximum of 128 con-current �ows [30,32,33,65,66,87,103,104,110,111], the DRAMoverhead of FLIN is 1.13 MB, which is less than 0.06% of thetotal DRAM capacity.Latency Impact. ReorderForFairness is the most time-

consuming function of FLIN, as it performs two passes over thequeue each time. Using our evaluation platform (see Section 6)to model a modern SSD controller with a 500 MHz multicoreprocessor [61], we �nd that ReorderForFairness incurs aworst-case and average latency overhead of 0.5% and 0.0007%,respectively, over a state-of-the-art out-of-order transactionscheduling unit [37]. None of the FLIN functions, includingReorderForFairness, execute on the critical path of trans-action processing. Hence, the maximum data throughput ofFLIN is identical to the baseline.

6. MethodologySimulation Setup. For our model-based evaluations, we

use MQSim [101], an open-source MQ-SSD simulator that

9

accurately models all of the major front-end and back-endcomponents in a modern MQ-SSD. We have open-sourced ourimplementation of FLIN [1] to foster research into fairness inmodern SSDs. Table 1 shows the con�guration that we use tomodel a representative MQ-SSD system in our evaluations. Inour evaluation, FLIN uses αreadc =32 MB/s, αwritec =256 kB/s,Fthr = 0.6, and GCFLIN = 1 K.

Table 1: Con�guration of the simulated SSD.

SSD OrganizationHost interface: PCIe 3.0 (NVMe 1.2)User capacity: 480 GB8 channels, 4 dies-per-channel

Flash Communication ONFI 3.1 (NV-DDR2)Interface Width: 8 bit, Rate: 333 MT/s

FlashMicroarchitecture

8 KiB page, 448 B metadata-per-page, 256pages-per-block, 4096 blocks-per-plane, 2planes-per-die

Flash Latencies [63] Read latency: 75 µs, Programlatency: 1300 µs, Erase latency: 3.8 ms

Workloads. We study a diverse set of storage traces col-lected from real enterprise and datacenter workloads, whichare listed in Table 2 [7, 68, 69, 80]. We categorize these work-loads as low-interference or high-interference, based on theaverage �ash chip busy time, which captures the combinede�ect of all four types of interference. We �nd that a work-load interferes highly with other workloads if it keeps all ofthe �ash chips busy for more than 8% of its total executiontime. We form MQ-SSD workloads using randomly-selectedcombinations of four low- and high-interference traces. Weclassify each workload based on the fraction of storage tracesthat are high-interference.Metrics. We use four key metrics in our evaluations. The

�rst metric, F , is strict fairness [22, 70, 73, 74], which we cal-culate using Eq. 2. The second metric, SMax , is the maximumslowdown [17, 49, 50, 109] among all concurrently-running�ows. The third metric, STDEV , is the standard deviation ofthe slowdowns of all concurrently-running �ows. The fourthmetric, WS, measures the weighted speedup [93] of MQ-SSD’sresponse time (RT), which represents the overall e�ciency ofthe I/O scheduler for all concurrent �ows. Weighted speedupis calculated as:

WS =∑i

RTalonei

RT sharedi

(11)

where RTalonei is the average response time of transactions

from �ow i when the �ow is run alone, and RT sharedi is the

average response time when the �ow runs concurrently withother �ows.7. Evaluation

We compare FLIN to two state-of-the-art baselines. Our �rstbaseline (Sprinkler) uses three state-of-the-art schedulingtechniques: (1) Sprinkler for out-of-order transaction schedul-ing in the TSU [37], (2) semi-preemptive GC execution [52],and (3) read prioritization and write/erase suspension [112].Our second baseline (Sprinkler+Fairness) combines Sprin-kler with a state-of-the-art in-controller MQ-SSD fairnesscontrol mechanism [36].7.1. Fairness and Performance of FLIN

Figures 10, 11, and 12 compare the fairness, maximumslowdown, and STDEV, respectively, of FLIN against the two

Table 2: Characteristics of the evaluated I/O traces.

TraceRead Average Avg. Inter InterferenceRatio Size (kB) Arrival (ms) Probability

RD WR RD WR

dev [69] 0.68 21.4 31.5 4.1 6.1 Lowexch [68] 0.66 9.7 14.8 1.0 1.3 High�n1 [7] 0.23 2.3 3.7 14.3 6.4 Low�n2 [7] 0.82 2.3 2.9 11.1 10.8 Lowmsncfs [69] 0.74 8.7 12.6 6.2 1.0 Lowmsnfs [69] 0.67 10.7 11.2 0.9 0.4 Highprn-0 [80] 0.11 22.8 9.7 34.2 117.2 Lowprn-1 [80] 0.75 22.5 11.7 30 126.7 Lowprxy-0 [80] 0.03 8.3 4.6 9.1 49.4 Lowprxy-1 [80] 0.65 24.6 26.1 3.7 3.4 Lowrad-be [69] 0.18 106.2 11.7 3.6 13.5 Lowrad-ps [69] 0.10 10.3 8.2 54.2 21.5 Lowsrc1-0 [80] 0.56 36.2 52.0 11.8 21.7 Highsrc1-1 [80] 0.95 35.8 14.7 3.2 213.9 Highsrc1-2 [80] 0.25 19.1 32.5 56.5 405.6 Lowstg-0 [80] 0.15 24.9 9.2 4.6 350.3 Lowstg-1 [80] 0.64 59.5 7.9 7.4 746.2 Lowtpcc [68] 0.65 8.1 9.4 0.1 0.1 Hightpce [68] 0.92 8.0 12.6 0.1 0.1 Highwsrch [7] 0.99 15.1 8.6 3.0 1.2 Low

baselines. For workload mixes comprised of 25%, 50%, 75%,and 100% high-interference �ows, FLIN improves the aver-age fairness by 1.8x/1.3x, 2.5x/1.6x, 5.6x/2.4x, and 54x/3.2x,respectively, compared to Sprinkler/Sprinkler+Fairness(Figure 10). For the same workload mixes, FLIN reduces theaverage maximum slowdown of the concurrently-running�ows by 24x/2.3x, 1400x/5.5x, 3231x/12.x, and 1597x/18x,respectively (Figure 11), and reduces the STDEV of theslowdown of concurrently-running �ows by an averageof 137x/13x, 7504x/15x, 2784x/15x, and 2850x/21.5x (Fig-ure 12). We make four key observations from the three�gures. First, Sprinkler’s fairness decreases signi�cantlyas the fraction of high-interference �ows in a workload in-creases, and approaches zero in workloads with 100% high-interference �ows. Second, Sprinkler+Fairness improvesfairness over Sprinkler due to its inclusion of fairness con-trol, but Sprinkler+Fairness does not consider all sourcesof interference, and therefore has a much lower fairness thanFLIN. Third, across all of our workloads, no �ow has a max-imum slowdown greater than 80x under FLIN, even thoughthere are several �ows that have maximum slowdowns over500x with Sprinkler and Sprinkler+Fairness. This indi-cates that FLIN provides signi�cantly better fairness over bothbaseline schedulers. Fourth, across all of our workloads, theSTDEV of the slowdown is always lower than 10 with FLIN,while it ranges between 100 and 1000 for many �ows underthe baseline schedulers. This means that FLIN distributesthe impact of interference more fairly across each �ow inthe workload. We conclude that FLIN’s slowdown manage-ment scheme e�ectively improves MQ-SSD fairness over bothstate-of-the-art baselines.

Figure 13 compares the weighted speedup of FLINagainst the two baselines. We observe from the �g-ure that for 25%, 50%, 75%, and 100% high-interferenceworkload mixes, FLIN improves the weighted speedup by38%/21%, 74%/32%, 132%/41%, and 156%/76%, on average, overSprinkler/Sprinkler+Fairness. The improvement is a re-sult of FLIN’s fairness control mechanism, which removesperformance bottlenecks resulting from high unfairness. FLIN

10

0.0

0.2

0.4

0.6

0.8

1.0

Mix1

Mix2

Mix3

Mix4

Mix5

Mix6

Mix7

Mix8

Mix9

Mix10

AVG

Fai

rnes

s(L

arge

r is

Bet

ter)

25% High-Interference

0.0

0.2

0.4

0.6

0.8

1.0

Mix1

Mix2

Mix3

Mix4

Mix5

Mix6

Mix7

Mix8

Mix9

Mix10

AVG


0.0

0.2

0.4

0.6

0.8

1.0

Mix1

Mix2

Mix3

Mix4

Mix5

Mix6

Mix7

Mix8

Mix9

Mix10

AVG


0.0

0.2

0.4

0.6

0.8

1.0

Mix1

Mix2

Mix3

Mix4

Mix5

Mix6

Mix7

Mix8

Mix9

Mix10

AVG0.0

0.2

0.4

0.6

0.8

1.0


0.0

0.2

0.4

0.6

0.8

1.0

Sprinkler Sprinkler+Fairness FLIN

Figure 10: Fairness of FLIN vs. baseline schedulers.

1

10

100

1000

10000

100000

Mix1

Mix2

Mix3

Mix4

Mix5

Mix6

Mix7

Mix8

Mix9

Mix10

AVG

Max

imum

Slo

wdo

wn

(Sm

alle

r is

Bet

ter)


1

10

100

1000

10000

100000

Mix1

Mix2

Mix3

Mix4

Mix5

Mix6

Mix7

Mix8

Mix9

Mix10

AVG


1

10

100

1000

10000

100000

Mix1

Mix2

Mix3

Mix4

Mix5

Mix6

Mix7

Mix8

Mix9

Mix10

AVG


1

10

100

1000

10000

100000

Mix1

Mix2

Mix3

Mix4

Mix5

Mix6

Mix7

Mix8

Mix9

Mix10

AVG 1

10

100

1000

10000

100000


Figure 11: Maximum slowdown of FLIN vs. baseline schedulers.

1

10

100

1000

10000

100000

Mix1

Mix2

Mix3

Mix4

Mix5

Mix6

Mix7

Mix8

Mix9

Mix10

AVG

ST

DE

V(S

mal

ler

is B

ette

r)


1

10

100

1000

10000

100000

Mix1

Mix2

Mix3

Mix4

Mix5

Mix6

Mix7

Mix8

Mix9

Mix10

AVG


1

10

100

1000

10000

100000

Mix1

Mix2

Mix3

Mix4

Mix5

Mix6

Mix7

Mix8

Mix9

Mix10

AVG


1

10

100

1000

10000

100000

Mix1

Mix2

Mix3

Mix4

Mix5

Mix6

Mix7

Mix8

Mix9

Mix10

AVG 1

10

100

1000

10000

100000


Figure 12: STDEV slowdown of FLIN vs. baseline schedulers.

0

1

2

3

4

Mix1

Mix2

Mix3

Mix4

Mix5

Mix6

Mix7

Mix8

Mix9

Mix10

AVGWei

ghte

d S

peed

up(L

arge

r is

Bet

ter)


0

1

2

3

4

Mix1

Mix2

Mix3

Mix4

Mix5

Mix6

Mix7

Mix8

Mix9

Mix10

AVG


0

1

2

3

4

Mix1

Mix2

Mix3

Mix4

Mix5

Mix6

Mix7

Mix8

Mix9

Mix10

AVG


0

1

2

3

4

Mix1

Mix2

Mix3

Mix4

Mix5

Mix6

Mix7

Mix8

Mix9

Mix10

AVG 0

1

2

3

4


Figure 13: Weighted speedup of FLIN vs. baseline schedulers.

improves the performance of low-interference �ows whilethrottling the performance of high-interference �ows, whichimproves weighted speedup by enabling low-intensity �owsto make fast progress. In contrast, Sprinkler typically pri-oritizes transactions from high-interference �ows. This im-proves the performance of high-interference �ows, but penal-izes the low-interference �ows and, thus, reduces the overallweighted speedup by causing all �ows to progress relativelyslowly. Sprinkler+Fairness improves the performance oflow-interference �ows by controlling the throughput of high-interference �ows, but its weighted speedup remains lowbecause the throughput control mechanism leaves many re-sources idle in the SSD [89]. We conclude that FLIN’s fairnesscontrol mechanism signi�cantly improves weighted speedup,an important metric for overall MQ-SSD performance, overstate-of-the-art device-level schedulers.7.2. E�ect of Di�erent FLIN Stages

We analyze the fairness, maximum slowdown, andweighted speedup improvements when using only (1) the�rst stage of FLIN (Section 4.1), or (2) the third stage of FLIN(Section 4.3), as these two stages are designed to reduce thefour sources of interference listed in Section 3. Figure 14shows the fairness (left), maximum slowdown (center), andweighted speedup of Sprinkler, Stage 1 of FLIN only, Stage 3of FLIN only, and all three stages of FLIN. For each category,the �gure shows the average value of each metric across allworkloads in the category.

We make three observations from Figure 14. First, the indi-vidual stages of FLIN improve both fairness and performanceover Sprinkler, as each stage works to reduce some sources

0.0

0.2

0.4

0.6

0.8

1.0

25% 50% 75% 100%

Fai

rnes

s

Fraction of High-Intensity

1

10

102

103

104

25% 50% 75% 100%

Max

Slo

wdo

wn

Fraction of High-Intensity

0.0

1.0

2.0

3.0

4.0

25% 50% 75% 100%Wei

ghte

d S

peed

upFraction of High-Intensity

Sprinkler Stage 1 Stage 3 FLIN

Figure 14: E�ect of Stages 1 and 3 of FLIN.

of interference among concurrently-running �ows, but eachstage alone falls short of the improvements we observe whenFLIN as a whole is used. Second, the fairness and perfor-mance improvements of Stage 1 are much higher than thoseof Stage 3. This is because I/O intensity is the most dominantsource of interference. Since Stage 1 is designed to mitigateinterference due to di�erent I/O intensities, it delivers largerimprovements than Stage 3. Third, Stage 3 reduces the max-imum slowdown by a greater amount than Stage 1. This isbecause GC operations can signi�cantly increase the stalltime of transactions, causing a large increase in maximumslowdown, and Stage 3 manages such GC interference e�ec-tively. We conclude that Stages 1 and 3 of FLIN are e�ectiveat mitigating di�erent sources of interference, and that FLINmakes e�ective use of both stages to maximize fairness andperformance improvements over state-of-the-art schedulers.7.3. Sensitivity to FLIN and MQ-SSD Parameters

We evaluate the sensitivity of fairness to four of FLIN’sparameters, as shown in Figure 15: (1) the epoch length for de-termining I/O intensity (Section 5.1), (2) the threshold for highread intensity (αreadC ; Section 5.1), (3) the fairness threshold for

11

high-intensity transaction reordering (Fthr ; Section 4.1), and(4) the GC management threshold (GCFLIN ; Section 4.3). Wesweep the values of each parameter individually. We makefour observations from the �gure. First, for epoch length,we observe from Figure 15a that epoch lengths of 10–100 msachieve the highest fairness, indicating that at these lengths,FLIN can best di�erentiate between low-intensity and high-intensity �ows. Second, for αreadC , we observe from Figure 15bthat if αreadC is too small (e.g., below 32 MB/s), fairness de-creases, as low-intensity �ows are incorrectly classi�ed ashigh-intensity �ows. Third, we observe in Figure 15c thatfor workloads where at least half of the �ows are high inten-sity, fairness is maximized for moderate values of Fthr (e.g.,Fthr = 0.6). This is because if Fthr is too low, FLIN does notprioritize the �ow with the maximum slowdown very often(see Section 4.1, Step 2b), which keeps fairness low. If Fthr istoo high, then FLIN almost always prioritizes the �ow thatcurrently has the maximum slowdown, which causes the other�ows to stall for long periods of time, and increases their slow-down. Fourth, we �nd that fairness is not highly sensitiveto the GCFLIN parameter, though smaller values of GCFLINprovide slightly higher fairness because they cause FLIN todefer GC operations for longer.

0.00.20.40.60.81.0

0.1 1 10 100 1000

Fai

rnes

s

Epoch Length (ms)

(a)

0.00.20.40.60.81.0

8 16 32 64 256

Fai

rnes

s

αcread(MB/s)

(b)

0.00.20.40.60.81.0

0.2 0.4 0.6 0.8 1

Fai

rnes

s

Fthr

(c)

0.00.20.40.60.81.0

1K 10K 100K 1M 10M

Fai

rnes

s

GCFLIN

(d)

25% 50% 75% 100%

Figure 15: Fairness with FLIN con�gurations.

We also evaluate the sensitivity of FLIN’s fairness and per-formance bene�ts to di�erent MQ-SSD con�gurations. Wemake three observations from these studies (not shown). First,a larger page size decreases the number of transactions inthe back end and, therefore, reduces the probability of in-terference. This reduces the bene�ts that FLIN can deliver.Second, newer �ash memory technologies have higher �ashread and write latencies [24,100], which results in an increasein the busy time of the �ash chips. This leads to a higherprobability of interference, and increases both the fairnessand performance bene�ts of FLIN. Third, having a larger num-ber of memory chips reduces the interference probability, bydistributing transactions over a larger number of chip-levelqueues, and thus reduces FLIN’s ability to improve fairness.

7.4. Evaluation of Support for Flow PrioritiesThe priority-based queue arbitration stage of FLIN supports

�ow-level priorities. We evaluate FLIN’s support for �ow pri-orities in a variety of scenarios and present a representativecase study to highlight its e�ectiveness. Figure 16 shows theslowdown of four msnfs �ows concurrently running withpriority levels 0, 1, 2, and 3. With FLIN, we observe thatthe highest-priority �ow (the �ow assigned priority level 3;msnfs-3 in Figure 16) has the lowest slowdown, as the transac-tions of this �ow are assigned more than half of the schedulingslots. The slowdown of each lower-priority version of msnfsincreases proportionally with the �ow’s priority.

0 10 20 30 40 50

msnfs-3msnfs-2

msnfs-1msnfs-0

Slo

wdo

wn

1 10

100 1000

10000

msnfstpce exchsrc-1-1

Slo

wdo

wn

Figure 16: E�ects of varying�ow priorities.

20x

40x

60x

25% 50% 75% 100%

Far

ines

sIm

prov

emen

t

Workload Category

FLIN-NoCacheFLIN-WithCache

Figure 17: E�ect of writecaching.

7.5. E�ect of Write CachingModern MQ-SSDs employ a front end DRAM write bu�er

in order to bridge the performance gap between the fast hostDRAM and the slow SSD memory chips. To investigate thee�ects of write caching on fairness, we conduct a set of ex-periments assuming a 128 MB write cache per running �ow(a total of 512 MB write cache). Figure 17 shows the averagefairness improvement of FLIN with and without a write bu�er.We observe that caching has only a minimal e�ect on FLIN’sfairness bene�ts, for two reasons. First, write caching has noimmediate e�ect on read transactions, so both I/O intensityand access pattern interference are not a�ected when writecaching is enabled. Second, high-interference �ows typicallyhave high I/O intensity. When the write intensity of a �owis high, the write cache �lls up and cannot cache any morerequests. At this point, known as the write cli�, any writerequests that cannot be cached are directly issued to the backend [39], eliminating the bene�ts of the write cache. We con-clude that write caching has only a minimal e�ect on theperformance of FLIN.8. Related Work

To our knowledge, this is the �rst work to (1) thoroughlyanalyze the sources of interference among concurrently-running I/O �ows in MQ-SSDs; and (2) propose a new device-level I/O scheduler for modern SSDs that e�ectively miti-gates interference among concurrently-running I/O �owsto provide both high fairness and performance. Provid-ing fairness and performance isolation among concurrently-running I/O �ows is an active area of research for modernSSDs [2, 28, 36, 45, 46, 84, 89, 94, 120], especially for mod-ern MQ-SSD devices that support new host–interface pro-tocols [36, 101], such as the NVMe protocol.Fair Disk Schedulers. Many works [3,9,34,86,95,113,116]

propose OS-level disk schedulers to meet fairness and real-time requirements in hard drives. These approaches are notsuitable for SSDs as they (1) essentially estimate I/O access la-tencies based on the physical structure of rotating disk drives,which is di�erent in SSDs; and (2) assume read/write latenciesthat are orders of magnitude longer than those of SSDs [47].

More recent OS-level schedulers have evolved to better �tthe internal characteristics of SSDs [84, 89, 120]. FIOS [84]periodically assigns an equal number of time-slices to therunning �ows. The �ows then trade these time-slices forscheduling I/O requests based on the �ash read/write ser-vice time costs. BCQ [120] adopts a similar approach with ahigher accuracy in determining read/write costs. Both FIOSand BCQ su�er from reduced responsiveness of the storagedevice and reduced overall performance [89]. More precisely,a �ow that consumes its time budget early would have towait for other �ows to �nish before its budget is replenishedin the next round, leading to a period of unresponsivenessand resource idleness [89]. FlashFQ [89] is another OS-levelscheduler that eliminates such unresponsiveness. FlashFQestimates the progress of each running �ow based on thecost of �ash read/write operations, and allows a �ow to dis-

12

patch I/O requests as long as its progress is not much furtherahead of other �ows. If a �ow’s progress is far ahead of otherconcurrently-running �ows, FlashFQ throttles the �ow thatis far ahead to improve fairness.

FIOS, BCQ, and FlashFQ su�er from two major shortcom-ings compared to FLIN. First, they do not have informationabout the internal resources and mechanisms of an SSD, andonly speculate about resource availability and the behaviorof SSD management algorithms when making a schedulingdecision. Such speculation-based scheduling decisions can besuboptimal for mitigating GC and parallelism interference, asboth types of interference are highly dependent on internalSSD mechanisms. In contrast, FLIN has direct informationabout the status of pending requests (in the I/O queue) andthe internal SSD resources/mechanisms. Second, OS-leveltechniques have no direct control over the SSD’s internal mech-anisms. Therefore, it is likely that there is some inter-�owinterference within the SSD, even when the OS-level schedulermakes use of fairness control mechanisms. For example, theTSU may execute GC activities due to a high-GC �ow, whichcan block the transactions of a low-GC �ow. In contrast, FLINtakes GC activities into account (e.g., Algorithm 2, whichworks to minimize GC interference), and controls all majorsources of interference in an MQ-SSD.

While our goal is to design FLIN to exploit the many ad-vantages of performing scheduling and fairness control inthe SSD instead of the OS, FLIN can also work in conjunc-tion with OS-level I/O resource control mechanisms to exploitthe best of both worlds. For example, virtualized environ-ments require OS-level management to control I/O requestsacross multiple VMs/containers [43]. One could envision asystem where the OS and FLIN work together to performcoordinated scheduling and fairness control. For example, theOS could make application-/VM-aware scheduling and fair-ness decisions, while FLIN performs resource-aware schedul-ing/fairness decisions. We leave an exploration of coordinatedOS/FLIN mechanisms to future work.Fair Main Memory Scheduling. There is a large collec-

tion of prior works that propose fair memory request sched-ulers for DRAM [4,18,19,44,49,50,71–75,81,96–99,105]. Theseschedulers cannot be directly used in SSDs, as DRAM devices(1) operate at a much smaller granularity than SSDs (64 B inDRAM vs. 8 kB in SSDs); (2) have symmetric read and writelatencies; (3) do not perform garbage collection; (4) have rowbu�ers in banks that signi�cantly a�ect performance; and(5) perform periodic refresh operations.

SSD Performance Isolation. Performance isolation [15,25, 28, 29, 46, 78, 91, 94] aims to provide predictable perfor-mance by eliminating interference from di�erent tenants thatshare an SSD. Inter-tenant interference can be reduced by(1) assigning di�erent hardware resources to di�erent ten-ants [28, 29, 46, 78, 94] (e.g., di�erent tenants are mappedto di�erent channels/dies), or (2) using isolation-aware al-gorithms for time sharing the hardware resources betweentenants [15, 25, 91, 94] (e.g., credit-based algorithms).

These techniques have three main di�erences from FLIN.First, they do not provide fairness control. Instead, they re-duce the interference between �ows without considering anyfairness metric (e.g., slowdown), and do not evaluate fairnessquantitatively. Second, they are not designed to optimizethe utilization of the MQ-SSD resources, which limits theirperformance. Third, no existing approach considers all fourinterference sources considered by FLIN.

High-Performance Device Schedulers. The vast major-ity of device-level I/O schedulers [20, 37, 40, 77, 90] try toe�ectively exploit SSD internal parallelism with out-of-orderexecution of �ash transactions. These schedulers are designedfor performance, and are vulnerable to inter-�ow interference.Performance Evaluation of NVMe SSDs. A number of

prior works focus on the performance evaluation and imple-mentation issues of MQ-SSDs [5, 114, 115]. Xu et al. [114]analyze the performance implications of MQ-SSDs for mod-ern database applications. Awad et al. [5] explore di�erentimplementation aspects of NVMe via simulation, and showhow these aspects a�ect system performance. None of theseworks consider the fairness issues in MQ-SSDs.9. Conclusion

We propose FLIN, a lightweight transaction scheduler formodern multi-queue SSDs (MQ-SSDs), which provides fair-ness among concurrently-running �ows. FLIN uses a three-stage design to protect against all four major sources ofinterference that exist in real MQ-SSDs, while enforcingapplication-level priorities that are assigned by the host. Ourextensive evaluations show that FLIN e�ectively improvesboth fairness and system performance compared to state-of-the-art device-level schedulers. FLIN is simple to implementwithin the SSD controller �rmware, requires no additionalhardware, and consumes less than 0.06% of the storage spaceavailable in the in-SSD DRAM. We conclude that FLIN is apromising scheduling mechanism that enables the design offair and high-performance MQ-SSDs.Acknowledgments

We thank the anonymous referees of ISCA 2018 and AS-PLOS 2018. We thank SAFARI group members for feedbackand the stimulating research environment. We thank ourindustrial partners, especially Alibaba, Google, Huawei, In-tel, Microsoft, and VMware, for their generous support. LoisOrosa was supported by FAPESP fellowship 2016/18929-4.References

[1] “MQSim GitHub Repository,” https://github.com/CMU-SAFARI/MQSim.[2] S. Ahn et al., “Improving I/O Resource Sharing of Linux Cgroup for NVMe SSDs

on Multi-Core Systems,” in HotStorage, 2016.[3] W. G. Aref et al., “Scalable QoS-Aware Disk-Scheduling,” in IDEAS, 2002.[4] R. Ausavarungnirun et al., “Staged Memory Scheduling: Achieving High Perfor-

mance and Scalability in Heterogeneous Systems,” in ISCA, 2012.[5] A. Awad et al., “Non-Volatile Memory Host Controller Interface Performance

Analysis in High-Performance I/O Systems,” in ISPASS, 2015.[6] J. Axboe, “Linux Block I/O—Present and Future,” in Ottawa Linux Symp., 2004.[7] K. Bates and B. McNutt, “UMass Trace Repository.”[8] M. Bjørling et al., “Linux Block IO: Introducing Multi-Queue SSD Access on Multi-

Core Systems,” in SYSTOR, 2013.[9] J. Bruno et al., “Disk Scheduling with Quality of Service Guarantees,” in ICMCS,

1999.[10] Y. Cai et al., “Error Characterization, Mitigation, and Recovery in Flash-Memory-

Based Solid-State Drives,” Proc. IEEE, 2017.[11] Y. Cai et al., “Read Disturb Errors in MLC NAND Flash Memory: Characterization

and Mitigation,” in DSN, 2015.[12] Y. Cai et al., “Data Retention in MLC NAND Flash Memory: Characterization,

Optimization, and Recovery,” in HPCA, 2015.[13] Y. Cai et al., “Errors in Flash-Memory-Based Solid-State Drives: Analysis, Mitiga-

tion, and Recovery,” arXiv:1711.11427 [cs:AR], 2017.[14] Y. Cai et al., “Neighbor-Cell Assisted Error Correction for MLC NAND Flash Mem-

ories,” in SIGMETRICS, 2014.[15] D.-W. Chang et al., “VSSD: Performance Isolation in a Solid-State Drive,” TODAES,

2015.[16] L.-P. Chang et al., “Real-Time Garbage Collection for Flash-Memory Storage Sys-

tems of Real-Time Embedded Systems,” TECS, 2004.[17] R. Das et al., “Application-Aware Prioritization Mechanisms for On-Chip Net-

works,” in MICRO, 2009.[18] E. Ebrahimi et al., “Parallel Application Memory Scheduling,” in MICRO, 2011.[19] E. Ebrahimi et al., “Fairness via Source Throttling: A Con�gurable and High-

Performance Fairness Substrate for Multi-Core Memory Systems,” in ASPLOS,2010.

[20] N. Elyasi et al., “Exploiting Intra-Request Slack to Improve SSD Performance,” inASPLOS, 2017.

[21] G2M Research, “NVMe Market Forecast & Vendor Report Abstract,” 2017.

13

[22] R. Gabor et al., “Fairness and Throughput in Switch on Event Multithreading,” inMICRO, 2006.

[23] E. Gal and S. Toledo, “Algorithms and Data Structures for Flash Memories,” CSUR,2005.

[24] L. M. Grupp et al., “The Bleak Future of NAND Flash Memory,” in FAST, 2012.[25] A. Gulati et al., “PARDA: Proportional Allocation of Resources for Distributed

Storage Access,” in FAST, 2009.[26] J. Guo et al., “Parallelism and Garbage Collection Aware I/O Scheduler with Im-

proved SSD Performance,” in IPDPS, 2017.[27] S. S. Hahn et al., “To Collect or Not to Collect: Just-in-Time Garbage Collection

for High-performance SSDs with Long Lifetimes,” in DAC, 2015.[28] J. Huang et al., “FlashBlox: Achieving Both Performance Isolation and Uniform

Lifetime for Virtualized SSDs,” in FAST, 2017.[29] S.-M. Huang and L.-P. Chang, “Providing SLO Compliance on NVMe SSDs

Through Parallelism Reservation,” TODAES, 2018.[30] Huawei Technologies Co., Ltd., “Huawei ES3000 V3 NVMe SSD White Paper,”

2017.[31] Intel Corp., “Intel Solid-State Drive DC P3600 Series, Product Speci�cation,” 2014.[32] Intel Corp., “Intel Solid-State Drive DC P3500 Series, Product Speci�cation,” 2015.[33] Intel Corp., “Intel 3D NAND SSD DC P4500 Series, Product Brief,” 2018.[34] S. Iyer and P. Druschel, “Anticipatory Scheduling: A Disk Scheduling Framework

to Overcome Deceptive Idleness in Synchronous I/O,” in SOSP, 2001.[35] W. Jin et al., “Interposed Proportional Sharing for a Storage Service Utility,” in

SIGMETRICS, 2004.[36] B. Jun and D. Shin, “Workload-aware Budget Compensation Scheduling for

NVMe Solid State Drives,” in NVMSA, 2015.[37] M. Jung and M. T. Kandemir, “Sprinkler: Maximizing Resource Utilization in

Many-chip Solid State Disks,” in HPCA, 2014.[38] M. Jung et al., “HIOS: A Host Interface I/O Scheduler for Solid State Disks,” in

ISCA, 2014.[39] M. Jung and M. Kandemir, “Revisiting Widely Held SSD Expectations and Re-

thinking System-Level Implications,” in SIGMETRICS, 2013.[40] M. Jung et al., “PAQ: Physically Addressed Queuing for Resource Con�ict Avoid-

ance in Solid State Disk,” in ISCA, 2012.[41] J.-U. Kang et al., “The Multi-streamed Solid-state Drive,” in HotStorage, 2014.[42] S. Kavalanekar et al., “Characterization of Storage Workload Traces from Produc-

tion Windows Servers,” in IISWC, 2008.[43] H.-J. Kim et al., “NVMeDirect: A User-space I/O Framework for Application-

speci�c Optimization on NVMe SSD,” in HotStorage, 2016.[44] H. Kim et al., “Bounding Memory Interference Delay in COTS-Based Multi-Core

Systems,” in RTAS, 2014.[45] J. Kim et al., “I/O Scheduling Schemes for Better I/O Proportionality on Flash-

Based SSDs,” in MASCOTS, 2016.[46] J. Kim et al., “Towards SLO Complying SSDs Through OPS Isolation,” in FAST,

2015.[47] J. Kim et al., “Disk Schedulers for Solid State Drivers,” in EMSOFT, 2009.[48] T. Y. Kim et al., “Improving Performance by Bridging the Semantic Gap Between

Multi-queue SSD and I/O Virtualization Framework,” in MSST, 2015.[49] Y. Kim et al., “ATLAS: A Scalable and High-Performance Scheduling Algorithm

for Multiple Memory Controllers,” in HPCA, 2010.[50] Y. Kim et al., “Thread Cluster Memory Scheduling: Exploiting Di�erences in

Memory Access Behavior,” in MICRO, 2010.[51] J. Lee et al., “Preemptible I/O Scheduling of Garbage Collection for Solid State

Drives,” TCAD, 2013.[52] J. Lee et al., “A Semi-preemptive Garbage Collector for Solid State Drives,” in

ISPASS, 2011.[53] Y. Li et al., “Stochastic Modeling of Large-Scale Solid-State Storage Systems: Anal-

ysis, Design Tradeo�s and Optimization,” in SIGMETRICS, 2013.[54] M. Lin and S. Chen, “E�cient and Intelligent Garbage Collection Policy for NAND

Flash-based Consumer Electronics,” TCE, 2013.[55] Linux Kernel Organization, Inc., “Block IO Priorities,” https://www.kernel.org/

doc/Documentation/block/ioprio.txt.[56] Linux Kernel Organization, Inc., “CFQ (Complete Fairness Queueing),” https://

www.kernel.org/doc/Documentation/block/cfq-iosched.txt.[57] Y. Luo et al., “WARM: Improving NAND Flash Memory Lifetime With Write-

Hotness Aware Retention Management,” in MSST, 2015.[58] Y. Luo et al., “Enabling Accurate and Practical Online Flash Channel Modeling

for Modern MLC NAND Flash Memory,” JSAC, 2016.[59] Y. Luo et al., “HeatWatch: Improving 3D NAND Flash Memory Device Reliability

by Exploiting Self-Recovery and Temperature Awareness,” in HPCA, 2018.[60] Marvell Technology Group, Ltd., “Marvell 88NV9145 PCIe-NAND controller,”

2012.[61] Marvell Technology Group, Ltd., “Marvell 88SS1093 Flash Memory Controller,”

2017.[62] Micron Technology, Inc., “64Gb, 128Gb, 256Gb, 512Gb Asyn-

chronous/Synchronous NAND,” 2009.[63] Micron Technology, Inc., “NAND Flash Memory MLC MT29F256G08CKCAB

Datasheet,” 2014.[64] Micron Technology, Inc., “128Gb, 256Gb, 512Gb Async/Sync Enterprise NAND,”

2016.[65] Micron Technology, Inc., “9100 U.2 and HHHL NVMe PCIe SSDs,” 2016.[66] Micron Technology, Inc., “Micron 9200 NVMe SSDs,” 2016.[67] Micron Technology, Inc., “MLC 128Gb to 2Tb Enterprise Async/Sync NAND,”

2016.[68] Microsoft Corp., “Microsoft Enterprise Traces,” http://iotta.snia.org/traces/130.[69] Microsoft Corp., “Microsoft Production Server Traces,” http://iotta.snia.org/

traces/158.[70] T. Moscibroda and O. Mutlu, “Memory Performance Attacks: Denial of Memory

Service in Multi-Core Systems,” in USENIX Security, 2007.

[71] T. Moscibroda and O. Mutlu, “Distributed Order Scheduling and Its Applicationto Multi-Core DRAM Controllers,” in PODC, 2008.

[72] S. P. Muralidhara et al., “Reducing Memory Interference in Multicore Systems viaApplication-aware Memory Channel Partitioning,” in MICRO, 2011.

[73] O. Mutlu and T. Moscibroda, “Stall-Time Fair Memory Access Scheduling for ChipMultiprocessors,” in MICRO, 2007.

[74] O. Mutlu and T. Moscibroda, “Parallelism-Aware Batch Scheduling: EnhancingBoth Performance and Fairness of Shared DRAM Systems,” in ISCA, 2008.

[75] O. Mutlu and L. Subramanian, “Research Problems and Opportunities in MemorySystems,” SUPERFRI, 2015.

[76] J. Nagle, “On Packet Switches with In�nite Storage,” TCOM, 1985.[77] E. H. Nam et al., “Ozone (O3): An Out-of-order Flash Memory Controller Archi-

tecture,” TC, 2011.[78] M. Nanavati et al., “Decibel: Isolation and Sharing in Disaggregated Rack-Scale

Storage,” in NSDI, 2017.[79] D. Narayanan et al., “Write O�-Loading: Practical Power Management for Enter-

prise Storage,” ToS, 2008.[80] D. Narayanan et al., “Migrating Server Storage to SSDs: Analysis of Tradeo�s,” in

Eurosys, 2009.[81] K. J. Nesbit et al., “Fair Queuing Memory Systems,” in MICRO, 2006.[82] D. M. Nguyen et al., “Solving the Multidimensional Assignment Problem by a

Cross-Entropy Method,” JCO, 2014.[83] NVM Express Workgroup, “NVM Express Speci�cation, Revision 1.2,” 2014.[84] S. Park and K. Shen, “FIOS: A Fair, E�cient Flash I/O Scheduler,” in FAST, 2012.[85] W. P. Pierskalla, “The Multidimensional Assignment Problem,” OR, 1968.[86] P. E. Rocha and L. C. E. Bona, “A QoS Aware Non-Work-Conserving Disk Sched-

uler,” in MSST, 2012.[87] Samsung Electronics Co., Ltd., “Samsung SSD 960 PRO M.2, Data Sheet,” 2017.[88] SATA-IO, “Serial ATA Revision 3.3,” http://www.sata-io.org, 2016.[89] K. Shen and S. Park, “FlashFQ: A Fair Queueing I/O Scheduler for Flash-Based

SSDs,” in ATC, 2013.[90] J.-Y. Shin et al., “Exploiting Internal Parallelism of Flash-based SSDs,” CAL, 2010.[91] D. Shue et al., “Performance Isolation and Fairness for Multi-Tenant Cloud Stor-

age,” in OSDI, 2012.[92] SK Hynix Inc., “F26 32Gb MLC NAND Flash Memory TSOP Legacy,” 2011.[93] A. Snavely and D. M. Tullsen, “Symbiotic Jobscheduling for a Simultaneous Mul-

tithreaded Processor,” in ASPLOS, 2000.[94] X. Song et al., “Architecting Flash-Based Solid-State Drive for High-Performance

I/O Virtualization,” CAL, 2014.[95] M. J. Stanovich et al., “Throttling On-Disk Schedulers to Meet Soft-Real-Time

Requirements,” in RTAS, 2008.[96] L. Subramanian et al., “The Blacklisting Memory Scheduler: Achieving High Per-

formance and Fairness at Low Cost,” in ICCD, 2014.[97] L. Subramanian et al., “BLISS: Balancing Performance, Fairness and Complexity

in Memory Access Scheduling,” TPDS, 2016.[98] L. Subramanian et al., “MISE: Providing Performance Predictability and Improv-

ing Fairness in Shared Main Memory Systems,” in HPCA, 2013.[99] L. Subramanian et al., “The Application Slowdown Model: Quantifying and Con-

trolling the Impact of Inter-Application Interference at Shared Caches and MainMemory,” in MICRO, 2015.

[100] K. Suzuki and S. Swanson, “A Survey of Trends in Non-Volatile Memory Tech-nologies: 2000-2014,” in IMW, 2015.

[101] A. Tavakkol et al., “MQSim: A Framework for Enabling Realistic Studies of Mod-ern Multi-Queue SSD Devices,” in FAST, 2018.

[102] A. Tavakkol et al., “Performance Evaluation of Dynamic Page Allocation Strate-gies in SSDs,” ToMPECS, 2016.

[103] Toshiba Corp., “OCZ RD400/400A Series, Product Brief,” 2016.[104] Toshiba Corp., “PX04PMC Series, Data Sheet,” 2016.[105] H. Usui et al., “DASH: Deadline-Aware High-Performance Memory Scheduler for

Heterogeneous Systems with Hardware Accelerators,” TACO, 2016.[106] P. Valente and F. Checconi, “High Throughput Disk Scheduling with Fair Band-

width Distribution,” TC, 2010.[107] B. Van Houdt, “A Mean Field Model for a Class of Garbage Collection Algorithms

in Flash-based Solid State Drives,” in SIGMETRICS, 2013.[108] B. Van Houdt, “On the Necessity of Hot and Cold Data Identi�cation to Reduce

the Write Ampli�cation in Flash-based SSDs,” Perform. Eval., 2014.[109] H. Vandierendonck and A. Seznec, “Fairness Metrics for Multi-Threaded Proces-

sors,” CAL, 2011.[110] Western Digital Technologies, Inc., “Skyhawk & Skyhawk Ultra NVMe PCIe SSD,

Data Sheet,” 2017.[111] Western Digital Technologies, Inc., “Ultrastar SN200 Series, Data Sheet,” 2017.[112] G. Wu and X. He, “Reducing SSD Read Latency via NAND Flash Program and

Erase Suspension,” in FAST, 2012.[113] J. C. Wu et al., “Hierarchical Disk Sharing for Multimedia Systems,” in NOSSDAV,

2005.[114] Q. Xu et al., “Performance Characterization of Hyper-scale Applications on

NVMe SSDs,” in SIGMETRICS, 2015.[115] Q. Xu et al., “Performance Analysis of NVMe SSDs and their Implication on Real

World Databases,” in SYSTOR, 2015.[116] Y. Xu and S. Jiang, “A Scheduling Framework That Makes Any Disk Schedulers

Non-Work-Conserving Solely Based on Request Characteristics,” in FAST, 2011.[117] S. Yan et al., “Tiny-Tail Flash: Near-Perfect Elimination of Garbage Collection

Tail Latencies in NAND SSDs,” in FAST, 2017.[118] S. Yang et al., “Split-Level I/O Scheduling,” in SOSP, 2015.[119] J. Zhang et al., “Synthesizing Representative I/O Workloads for TPC-H,” in HPCA,

2004.[120] Q. Zhang et al., “An E�cient, QoS-Aware I/O Scheduler for Solid State Drive,” in

HPCC EUC, 2013.

14

flin: enabling fairness and enhancing performance in modern nvme solid state...

Documents