EDITOR’S NOTE ALL-FLASH STORAGE ARRAYS: PERFORMANCE VS. FUNCTION

HOW MUCH FLASH IS ENOUGH?

ALL-FLASH POWERS PERFORMANCE-HUNGRY APPS

Navigate the Complex World of All-Flash ArraysAll-flash arrays are coming in hot, but the jury is still out on the right approach for deploying solid-state storage today.

HOME

EDITOR’S NOTE

ALL-FLASH STORAGE ARRAYS:

PERFORMANCE VS. FUNCTION

HOW MUCH FLASH

IS ENOUGH?

ALL-FLASH POWERS

PERFORMANCE-HUNGRY

APPS

NAVIGATE THE COMPLEX WORLD OF ALL-FLASH ARRAYS 2

EDITOR’SNOTE

Which All-Flash Array is Right for Your Needs?

All-flash arrays are coming in hot, but the jury is still out on the right approach for deploying solid-state storage. As with all tech-nologies, of course, it’s going to come down to the needs of a given shop. However, there’s some big-picture stuff to consider if you are truly strapped for performance.

While flash is becoming more affordable these days, it’s still kind of a niche, high per-formance kind of deal. For the most part, if you are dealing with performance issues, you can throw a little flash in the mix somewhere and see positive results. In other words, if you are considering an all-flash array, well, you’ve got some serious demands from a storage stand-point. And although the enterprise flash mar-ket is still relatively young, if you need that kind of performance today, you’ve already got some options.

The market for all-flash arrays is sort of

segmented into three broad groups. First, you have the arrays that have been designed from the ground up strictly for performance. Then, you have arrays designed for flash that offer high performance alongside enterprise-class array functionality that many IT people expect and rely on. Finally, you have legacy gear (with the functionality users know well) tweaked to run with SSDs stuffed in the slots.

There is no one-size-fits-all approach, so this handbook takes a look at how to weigh the options available. Also, you’ll get tips on how to decide whether an all-flash array is right for your needs or if you would be better suited with a hybrid array. Finally, read how one shop deployed an all-flash array to meet perfor-mance and data access needs. n

Andrew BurtonSenior Site Editor, SearchSolidStateStorage

HOME

EDITOR’S NOTE

ALL-FLASH STORAGE ARRAYS:

PERFORMANCE VS. FUNCTION

HOW MUCH FLASH

IS ENOUGH?

ALL-FLASH POWERS

PERFORMANCE-HUNGRY

APPS

NAVIGATE THE COMPLEX WORLD OF ALL-FLASH ARRAYS 3

PERFORMANCE VS. FUNCTION

All-Flash Storage Arrays: Performance vs. Function

Nearly every storage vendor now offers all-flash storage arrays, and IT professionals are beginning to recognize the need for these high-performance storage systems. But how does an IT pro decide which of the many all-flash arrays are best suited for their organization and performance demands?

PERFORMANCE VS. FUNCTION

As the all-flash storage array market begins to mature, there are two categories of arrays emerging.

The first are all-flash arrays that were designed from the ground up to be all-flash arrays. They typically have optimized hardware designs that focus on extracting the maximum possible performance from the flash within the array. The vendors in this category are almost all emerging technology companies or start-ups. In most cases, their focus on hardware and

performance is at the expense of storage soft-ware services. These are the features that many storage administrators now count on to do their jobs, providing capabilities like snapshots, replication and cloning.

These arrays are known for generating mil-lions of IOPS per system. However, there really is no established method for how those high IOPS numbers are obtained. They can be gener-ated from a single workload or multiple work-loads accessing the system at the same time.

The other category is made up of all-flash arrays that are more feature-oriented. These are typically systems from established ven-dors, as well as a few startups, that choose to focus on the software functionality, often at the expense of maximum performance. Typically, these systems either use legacy hardware from the established vendor and retrofit their old arrays with solid-state drives (SSDs) or, in the case of a startup, use off-the-shelf hardware to

HOME

EDITOR’S NOTE

ALL-FLASH STORAGE ARRAYS:

PERFORMANCE VS. FUNCTION

HOW MUCH FLASH

IS ENOUGH?

ALL-FLASH POWERS

PERFORMANCE-HUNGRY

APPS

NAVIGATE THE COMPLEX WORLD OF ALL-FLASH ARRAYS 4

PERFORMANCE VS. FUNCTION

keep costs down. These systems can often generate 200-400k

IOPS per system. Some scale-out, software-rich systems claim an aggregate performance of millions of IOPS as well but the devil is in the details. They typically have a performance limit per volume or per node within the scale-out cluster. This means they can scale to millions of IOPS like the performance-focused systems described above, but it takes many nodes to get there and to see that extreme performance requires multiple workloads all running con-currently. A scale-out system cannot deliver millions of IOPS to a single workload or thread.

WHICH IS BEST?

I am often asked which method is best. The answer, as usual, depends on the needs of the data center and the specific applications that are running. Most data centers, while perfor-mance-constrained, are not constrained to the point that they typically exceed the baseline performance of a feature-rich all-flash array. Also, most organizations take great comfort in the availability of the feature sets they have

become accustomed to from legacy hard disk arrays.

Some environments need more than a half million IOPS, but it’s how those IOPS are needed that help determine the best system for a particular data center. If the need for perfor-mance is distributed across more than a few workloads, the all-flash systems that can pro-vide scale-out linear performance growth are ideal.

If the environment has a single workload that needs more than half a million IOPS, then the performance-focused systems are needed. As stated above, these systems can provide mil-lions of IOPS to a single workload.

MIDDLE GROUND?

Is there room in the middle? Does a stor-age system exist that can meet the needs of a performance-demanding workload, yet still provide the feature-rich environment that more traditional applications require? There are sev-eral vendors that provide this type of system. It must be designed first as a performance-focused system, then have software added to it. While

HOME

EDITOR’S NOTE

ALL-FLASH STORAGE ARRAYS:

PERFORMANCE VS. FUNCTION

HOW MUCH FLASH

IS ENOUGH?

ALL-FLASH POWERS

PERFORMANCE-HUNGRY

APPS

NAVIGATE THE COMPLEX WORLD OF ALL-FLASH ARRAYS 5

PERFORMANCE VS. FUNCTION

the addition of that software adds some latency, it does not affect most applications. These sys-tems typically have performance to spare.

This software can be added in several ways. Some vendors provide an appliance that the performance-focused system can be connected into, allowing it to take advantage of all the features that the appliance can provide. This storage virtualization approach also allows the all-flash array to be somewhat integrated from a software services perspective.

Other vendors can load storage software onto a co-processor within the flash array itself. This provides a tighter integration experience and saves the cost of an external appliance.

Finally, all of these hardware-focused systems could work with any software-defined stor-age technology on the market today, including converged options that run within the hypervi-sor architecture. The key is to make sure that that the software-defined approach can support external, shared storage (not all do).

While combining a hardware-focused option with either an appliance or hypervisor that

delivers the storage services, it’s important to remember that there remains one big challenge. The hardware-focused flash system must be delivered at a price point (including software) that is in the same range as the feature-rich approaches described above. In most cases, the feature-rich options are still the most cost-effective, and again, 400k IOPS is more than enough for most data centers.

All-flash arrays are becoming mainstream. Many vendors claim price parity with “perfor-mance-focused” hard drive arrays. These would be arrays from name-brand vendors that are using 15K RPM drives. This claim is generally true, so any data center looking to buy a per-formance-focused disk array should seriously consider an all-flash array.

The choice within the all-flash segment largely depends on what the data center needs. For most data centers, the feature-rich sys-tems are all they need. But it may be worth the investigative step to confirm that and to then determine if they need a scale-up or scale-out system. —George Crump

HOME

EDITOR’S NOTE

ALL-FLASH STORAGE ARRAYS:

PERFORMANCE VS. FUNCTION

HOW MUCH FLASH

IS ENOUGH?

ALL-FLASH POWERS

PERFORMANCE-HUNGRY

APPS

NAVIGATE THE COMPLEX WORLD OF ALL-FLASH ARRAYS 6

ENOUGH FLASH

How Much Flash Is Enough?

Now that solid-state storage prices have dropped, more vendors offer all-flash arrays; but are they really much better than hybrid storage arrays that mix flash with spinning disk?

Comparing technology systems has tradi-tionally relied on a “price-to-performance” analysis that tries to normalize system differ-ences into an apples-to-apples comparison. With respect to storage, the advent of flash (solid-state) drives created a situation where IT buyers had to decide between price and per-formance. Flash storage offered blazing speeds, but at a very high cost per gigabyte ($/GB). At the other end of the spectrum, multi-tera-byte hard disk drives (HDDs) are very econom-ical, but with just around 75 raw IOPS per drive there better not be much work in the workload if that’s where your data is located.

HDDs have an advantage in $/GB, while flash has an advantage in $/IOPS.

Hybrid arrays are intended to balance that equation. By adding a thin slice of flash stor-age to an array (i.e., 2% to 5% of total capac-ity), available IOPS may double and reduce read latency from 10+ milliseconds (ms) down to 3 ms to 5 ms. Even though the flash is expensive by itself, an overall 10% to 20% increase in array price to yield a 2X performance gain adds a lot of bang to the buck.

As good as 3 ms to 5 ms latency sounds, for an increasing number of applications this vari-ability is unacceptable. All-flash arrays can deliver sub-millisecond read latency with a guaranteed quality of service (QoS). There are no pre-fetch issues with flash and no variabil-ity between a cached read and a seek because on flash all I/Os are effectively cache reads. To put it in perspective, improving from 10 ms to 5 ms and then down to 0.5 ms is a 20X performance gain from HDD to all-flash on a latency basis. Although the difference between

HOME

EDITOR’S NOTE

ALL-FLASH STORAGE ARRAYS:

PERFORMANCE VS. FUNCTION

HOW MUCH FLASH

IS ENOUGH?

ALL-FLASH POWERS

PERFORMANCE-HUNGRY

APPS

NAVIGATE THE COMPLEX WORLD OF ALL-FLASH ARRAYS 7

ENOUGH FLASH

HDD prices and flash has narrowed consider-ably, most organizations still don’t have the budget to deploy hundreds of terabytes of it. So, if deploying flash technology judiciously is important to an organization, knowing where the breakpoints between hybrid and all-flash are will help managers to make the best decision.

PRICE/PERFORMANCE OF HYBRID STORAGE

ARRAYS VS. ALL-FLASH ARRAYS

With price-to-performance being the major consideration in storage purchases, let’s exam-ine those two areas first. When it comes to cost, it’s an industry truism that the purchase price is approximately 20% of the three-year cost of ownership for conventional HDD arrays. All-flash storage arrays break this model, however, partly because the purchase price goes up but operating costs go down. For example, Nimbus Data, a maker of all-flash

arrays, estimates that its arrays use just 8 watts per TB, whereas HDD arrays may use 80 watts per TB. Moreover, Nimbus advises that users can fully populate a rack with usable space and get up to 90% utilization without performance degradation. This offers the potential for less floor space, cooling and power per TB of stor-age. High-capacity SATA drives offer gigabyte density, but won’t meet even moderate IOPS or latency requirements. Given that flash is fol-lowing the downward curve of cache memory, it’s well worth the time to compare the total cost of ownership (TCO) of all-flash and hybrid storage systems.

Hewlett-Packard (HP) Co. offers a differ-ent perspective on cost metrics: $/transaction. This metric is reflective of the true business cost, whereas $/GB and $/IOPS reflect pur-chase cost only. The $/transaction metric can be applied irrespective of media type, making a “non-denominational” comparison between techs. High-transaction workloads may

Knowing where the breakpoints between hybrid and all-flash are will help managers to make the best decision.

HOME

EDITOR’S NOTE

ALL-FLASH STORAGE ARRAYS:

PERFORMANCE VS. FUNCTION

HOW MUCH FLASH

IS ENOUGH?

ALL-FLASH POWERS

PERFORMANCE-HUNGRY

APPS

NAVIGATE THE COMPLEX WORLD OF ALL-FLASH ARRAYS 8

ENOUGH FLASH

actually see a lower $/transaction on all-flash arrays, whereas lower transaction environments may have a lower $/transaction on hybrid stor-age arrays. Either way, the result is calculable and definitive.

Although it may be considered a “soft” cost, IT managers should factor in the value of the user’s experience. Justifiably or not, if users perceive an application to be slow, they’re more likely to have a negative perception of the IT organization or provider. It’s a bit like the adage that when airline passengers find coffee stains on their tray tables, they question the quality of engine maintenance. It may therefore make sense to spend a few extra dollars to improve the user’s perception of quality, especially in this era of IT outsourcing.

Deduplication and compression are staples of nearly all storage systems and are fre-quently used to generate a $/GB figure that’s much more favorable than those based on raw capacity. However, a cautionary note is in order. Increasingly, deduplication and single-instance storage (SIS) are taking place at the OS and application layers (i.e., VMware and SIS products for Exchange). Deduplication and

compression can’t occur twice on the same data, so storage managers may see less reduc-tion than they anticipate as dedupe moves up the stack.

For the purposes of commercial computing applications, storage performance has been all about IOPS until very recently. Perhaps this was because storage arrays inherently couldn’t guarantee a specific QoS. All-flash arrays change that game, and SolidFire pro-motes its flash arrays squarely as storage QoS delivery vehicles. SolidFire allows IOPS to be provisioned on a per-volume basis with minimum, maximum and burst parameters. This throttling, along with dynamic adjust-ment, gives storage managers an additional mechanism for precisely delivering perfor-mance where it’s needed. SolidFire also sug-gests that guaranteed QoS solves the “noisy neighbor” problem. For example, in a typical hybrid array, applications may compete for the flash tier of storage based on data access, resulting in oversubscribed flash and sub- optimal performance for all applications. Allocating IOPS to each application specifically solves this problem.

HOME

EDITOR’S NOTE

ALL-FLASH STORAGE ARRAYS:

PERFORMANCE VS. FUNCTION

HOW MUCH FLASH

IS ENOUGH?

ALL-FLASH POWERS

PERFORMANCE-HUNGRY

APPS

NAVIGATE THE COMPLEX WORLD OF ALL-FLASH ARRAYS 9

ENOUGH FLASH

EITHER OR BOTH?

Established vendors have extended their traditional HDD products to embrace flash technology, often with hybrid devices and all-flash units. IBM, EMC Corp., NetApp Inc. and HP each offer an “all of the above” portfolio, though the specific implementations differ in important ways.

n IBM has extended its SAN Volume Control-ler (SVC) virtualization capability to man-age “fit for purpose” all-flash devices such as the FlashSystem family or hybrid arrays in its XIV Storage System, Storwize and DS prod-uct lines. Because all devices can be managed through SVC, IBM extends the concept of hybrid to not only arrays, but the enterprise storage ecosystem. The intent is to allow ulti-mate flexibility in deploying both combinations of media, as well as combinations of arrays for scale and performance where needed.

n EMC’s VMAX can virtualize hybrid and all-flash systems into a single ecosystem. Its VMAX, VNX, VNXe and Isilon products can also be configured as either hybrid or all-flash.

Thus, users can enter at any point and evolve the solution as requirements change over time. In addition, the company’s XtremIO arrays offer an all-flash approach from the ground up. Because it doesn’t necessarily have to separate its hybrid and all-flash offerings, EMC views the market more horizontally. That is, it can offer traditional HDD arrays to the more bud-get-conscious buyer, hybrid configurations to those needing more performance and all-flash for situations that demand guaranteed QoS. Which product line will be recommended is determined by reliability, availability and ser-viceability (RAS) requirements, with customers needing up to “six nines” of availability choos-ing VMAX, regardless of hybrid or all-flash provisioning.

n HP’s 3PAR similarly offers everything from all-HDD to hybrid to flash configurations, but with some interesting wrinkles. HP offers both single-level cell and multi-level cell flash in the same arrays. The company recommends using the caching layer for writes and the flash tier for reads. This builds upon its Adaptive Optimization software that enables sub-LUN

HOME

EDITOR’S NOTE

ALL-FLASH STORAGE ARRAYS:

PERFORMANCE VS. FUNCTION

HOW MUCH FLASH

IS ENOUGH?

ALL-FLASH POWERS

PERFORMANCE-HUNGRY

APPS

NAVIGATE THE COMPLEX WORLD OF ALL-FLASH ARRAYS 10

ENOUGH FLASH

tiering. 3PAR’s OS has a built-in clustered vol-ume manager and virtual memory implementa-tion that seamlessly virtualizes all media types, including new media.

n NetApp offers flash across its entire portfolio of FAS, V-series and E-series arrays; the EF540 and EF550 are all-flash systems. NetApp usu-ally recommends Flash Pools in its arrays with 1% to 2% of total capacity in flash. The com-pany suggests that typical workloads, such as email, Web serving, app dev and collaboration are best served with hybrid configurations. In these use cases, workloads require less than 150,000 IOPS and can tolerate 3 ms to 5 ms of latency. For higher IOPS workloads or where QoS demands sub-ms latency, all-flash is prescribed.

n Oracle’s ZFS Storage ZS3 are hybrid arrays, but the company says users may get up to a 90% data hit rate using its flash/DRAM archi-tecture and sub-ms latency on those reads. Moreover, the company touts a significant cost advantage over competitive all-flash systems. Thus, users may get near all-flash performance

at a hybrid price. Oracle’s Hybrid Storage Pool dynamically and automatically moves data across DRAM, read-flash and write-flash to optimize array performance.

RAID CONSIDERATIONS

Flash devices need to be protected by RAID just like any other storage media. Although most vendors support conventional RAID techniques for flash, RAID use can extract a price in capacity terms as well as process-ing overhead just as it does for HDDs. IBM addresses this issue with “variable stripe RAID” in its FlashSystem technology where RAID 5 is built into the flash controller. The result is parity-based RAID at line speed with the workload distributed across controllers. NetApp deploys Dynamic Disk Pools in its SANtricity (E-series) software that distributes data, parity and spare capacity across drives. NetApp claims this speeds recovery of failed drives while maintaining greater performance. Nimbus uses a RAID 5 algorithm designed to avoid the wear penalty associated with writing to flash.

HOME

EDITOR’S NOTE

ALL-FLASH STORAGE ARRAYS:

PERFORMANCE VS. FUNCTION

HOW MUCH FLASH

IS ENOUGH?

ALL-FLASH POWERS

PERFORMANCE-HUNGRY

APPS

NAVIGATE THE COMPLEX WORLD OF ALL-FLASH ARRAYS 11

ENOUGH FLASH

SOME AGREEMENT ON GUIDELINES

Despite differences in architectures, the ven-dors generally agree on some hybrid vs. all-flash guidelines. First, if sub-ms or guaranteed QoS is required, then all-flash arrays are the way go, or in the case of Oracle, a hybrid that can deliver near all-flash performance. QoS application candidates include e-commerce where user experience is paramount and deci-sion support analysis where time is of the essence. Second, if variable and unpredictable

workloads are to be serviced, hybrid devices can often serve the need at a lower $/GB. Nimbus, SolidFire and other all-flash vendors may have different answers to this rule of thumb that do indeed have interesting price/performance characteristics and may have a $/IOPS advantage. Application candidates in this area include collaboration, email and any-thing where data lifecycle issues mean that not all data requires immediate access. —Phil Goodwin

HOME

EDITOR’S NOTE

ALL-FLASH STORAGE ARRAYS:

PERFORMANCE VS. FUNCTION

HOW MUCH FLASH

IS ENOUGH?

ALL-FLASH POWERS

PERFORMANCE-HUNGRY

APPS

NAVIGATE THE COMPLEX WORLD OF ALL-FLASH ARRAYS 12

ALL-FLASH POWER

All-Flash Powers Performance-Hungry Apps

As part of its move to upgrade its tech-nology to electronic records and imaging, St. Ann’s Community adopted an all-flash array to improve storage performance and data accessibility.

St. Ann’s Community has been serving seniors in Rochester, N.Y., for over 140 years. St. Ann’s CIO, David Moufarrege, said that as a housing and health system that provides mostly rehabilitation and long-term care ser-vices, St. Ann’s does not have as much need for digital storage resources as hospitals that keep electronic images, such as PACS and CT scans, online. But St. Ann’s recently added an elec-tronic health record (EHR) system, and more digital images will be coming online soon.

“We don’t have the huge amount of storage needs that hospitals have, but that’s changing, and changing rapidly,” he said. “In the last two-and-a-half years, we have made a 15-year transi-tion in what technologies have been used here.”

Before the EHR system, Moufarrege dealt mostly with file storage. But to accommodate the performance needs associated with the HealthMEDX Vision EHR system, St. Ann’s went live in January with an Astute Networks ViSX G4 all-flash array, which stores the SQL database used for EHR. The ViSX G4 is a 2U iSCSI system that holds 24 solid-state drives (SSDs) for up to 45.6 TB of raw capacity.

“We didn’t buy ViSX for traditional storage,” Moufarrege said. “There’s no reason to use this kind of performance for files. Who cares if a Word document takes 15 seconds to load? But you have a real-time need in a clinical center when you’re retrieving patient records.”

He said the ViSX can hit 140,000 IOPS, greatly increasing access time and increas-ing EHR performance from “very slow before the ViSX implementation to now being very snappy.” St. Ann’s also uses the ViSX as storage for its Citrix XenServer images.

HOME

EDITOR’S NOTE

ALL-FLASH STORAGE ARRAYS:

PERFORMANCE VS. FUNCTION

HOW MUCH FLASH

IS ENOUGH?

ALL-FLASH POWERS

PERFORMANCE-HUNGRY

APPS

NAVIGATE THE COMPLEX WORLD OF ALL-FLASH ARRAYS 13

ALL-FLASH POWER

“Productivity has gone through the roof there too,” Moufarrege said of the virtual server stor-age. “On a 15,000 rpm hard drive, you might get 250 IOPS, and ViSX increased that to 140,000 IOPS.”

Moufarrege said he met with Astute repre-sentatives at a CIO conference last year, and they offered him a testing unit. When he saw the performance, he was sold.

“In healthcare, you’re not typically an early adopter,” he said. “You want to make sure it works—you want somebody to make the mis-takes first. But this passed the initial test, then we did a real test, and it works.”

St. Ann’s still uses its Hewlett-Packard MSA systems for files. Moufarrege said he also mir-rors between the ViSX and an MSA system at the system’s second campus, using SIOS

DataKeeper block-level replication software. “That allows us at a block level to write the same item to the other site so we have a real-time clone of our databases on there,” he said.

Moufarrege said he may eventually add a ViSX at the second campus as a mirror tar-get. He also plans to add SSD capacity as more electronic records come online. He will keep data on the ViSX for a pre-determined period of time before moving it to an archiving sys-tem. “Imaging will be the big thing -- what happens when we have X-rays online and CT scans online? That will be the test for capacity,” he said.

Moufarrege offers the flash array as a sign that St. Ann’s Community is making progress with technology.

“I see flash as a disruptive technology,” he said. “This type of technology wouldn’t have been available five years ago. Even 18 months ago, it would’ve been too expensive for this application. The biggest hurdle is [whether] your traditional controller can handle that amount of throughput? Astute has its own controller design to avoid the bottleneck. It’s a Porsche.” —Dave Raffo

“ In healthcare, you’re not typically an early adopter. You want some-body to make the mistakes first.”

—DAVID MOUFARREGE, CIO, St. Ann’s Community

HOME

EDITOR’S NOTE

ALL-FLASH STORAGE ARRAYS:

PERFORMANCE VS. FUNCTION

HOW MUCH FLASH

IS ENOUGH?

ALL-FLASH POWERS

PERFORMANCE-HUNGRY

APPS

NAVIGATE THE COMPLEX WORLD OF ALL-FLASH ARRAYS 14

ABOUT THE

AUTHORS

GEORGE CRUMP is president of Storage Switzerland, an IT analyst firm focused on storage and virtualization.

PHIL GOODWIN is a storage consultant and freelance writer.

DAVE RAFFO is senior news director with TechTarget’s Storage Media Group.

Navigate the Complex World of All-Flash Arrays is a SearchSolidStateStorage.com e-publication.

Rich Castagna | Editorial Director

Andrew Burton | Senior Site Editor

Ed Hannan | Managing Editor

Dave Raffo | Senior News Editor

Linda Koury | Director of Online Design

Neva Maniscalco | Graphic Designer

Jillian Coffin | Publisher jcoffin@techtarget.com

TechTarget 275 Grove Street, Newton, MA 02466

www.techtarget.com

© 2014 TechTarget Inc. No part of this publication may be transmitted or re-produced in any form or by any means without written permission from the publisher. TechTarget reprints are available through The YGS Group.

About TechTarget: TechTarget publishes media for information technology professionals. More than 100 focused websites enable quick access to a deep store of news, advice and analysis about the technologies, products and pro-cesses crucial to your job. Our live and virtual events give you direct access to independent expert commentary and advice. At IT Knowledge Exchange, our social community, you can get advice and share solutions with peers and experts.

COVER ART: THINKSTOCK