netapp unified scale-out/clustered storage

Silverton Consulting, Inc. StorInt™ Briefing

© 2012 Silverton Consulting, Inc. of 9 All Rights Reserved twitter.com/RayLucchesi|RayOnStorage.com

+1-720-221-7270|SilvertonConsulting.com

Introduction Today’s data centers are being asked to do more with little additional financial support forthcoming. They are under a constant barrage to do things faster, more inexpensively and with no disruption to the revenue-‐generating part of the company. Frequently, they are experiencing 24 X 7 operations, numerous new application deployments and explosive data growth. Data storage often becomes a crucial limiting factor to meeting these stringent demands. Faced with these seemingly insurmountable challenges, CIO’s are discovering that the old way of responding to application and data growth is unacceptable. That is, the outdated “rip and replace” method to improve capacity and IO performance, which often meant disruptive migration, doesn’t work anymore. Instead, a more nimble alternative is needed. In response to this need for storage agility, NetApp has recently released a new version of their storage operating environment called Data ONTAP® 8.1. This new software update successfully eliminates many problems of typical monolithic or legacy storage systems.

Why unified scale-‐out architecture can help By definition, monolithic storage is only minimally flexible. Each storage system is equipped with a set of processing elements (controllers) and storage drive expansion shelves. The storage can be scaled up to match increasing capacity and performance demands, but the improvement options are limited. For example, maximum capacity for a particular system is restricted to the drives and expansion shelves supported. Similarly, performance scalability is also confined to increasing cache, drives and/or ports. Thus, any significant improvement in capacity or performance generally requires a stressful and disruptive replacement process. Probably the most difficult part of this process is the successful migration of the data itself, but perhaps the most frustrating part is knowing that the whole change out process will be repeated in a couple of years. In contrast, unified scale-‐out storage architecture provides almost limitless scale out possibilities, with support for both file and block protocols, as well as enterprise and technical workloads. By simply adding nodes to the storage group, capacity and performance can be scaled to match current needs and then later easily re-‐scaled to meet future needs. Moreover, this system enhancement requires no disruption of ongoing operations.

NetApp Unified Scale-out Storage



Technology refresh is another instance where unified scale-‐out storage architecture can be extremely beneficial. Over time a storage cluster’s technology can be upgraded by first adding new system nodes and then reconfiguring and moving the data from the old to the new storage. As a result, the entire storage cluster evolves to provide better services to the data center. Again all is done without disruption. Yet another noteworthy benefit of unified scale-‐out architecture is that both IT’s unstructured as well as structured data can be managed within the same storage cluster. Thus, as file and/or block needs change, the unified storage cluster is ready to handle both in whatever proportion the business requires.

NetApp’s unified scale-‐out storage NetApp introduced scale-‐out storage services with the release of Data ONTAP 8.0 operating in Cluster-‐Mode in 2009. With this advance, NetApp transformed their systems to include the ability to scale up and scale-‐out a multi-‐node storage cluster. As such, a single NetApp storage cluster can scale capacity from TBs to tens of PBs, but even more importantly it can scale performance from a single system to a multi-‐node storage complex, all managed as a single system.

As shown in Figure 1, a NetApp scale-‐out cluster consists of one or more HA-‐pairs of storage systems that, with the latest ONTAP 8.1.1, can grow to 3-‐HA pairs for SAN or unified storage (6-‐node SAN & NAS) or 12-‐HA pairs for NAS only storage (24-‐node NAS). Physical storage resources are pooled across the cluster and are available to service any IO request coming into the system. Nodes in the cluster communicate using a private 10GbE interconnect network. NetApp storage cluster services are defined on top of Vservers that provide a logical interface to physical resources. These Vservers can span physical storage nodes and can house multiple LUNS and volumes as needed.

With NetApp’s Data ONTAP 8.1.1, Infinite Volume support for NFS Enterprise Content Repositories is provided. With this capability, Data ONTAP 8.1.1 can link multiple volumes across cluster nodes to create a single file system that can scale up to a maximum 20PB and hold billions of files. However, the first target application for Infinite Volume requires a single Vserver spanning an entire storage cluster. Currently, an Infinite Volume storage cluster is limited to a 10-‐node (5 HA-‐pair) configuration of NetApp FAS6280s.

Figure 1 Cluster Configuration © 2012 NetApp




Under NetApp’s storage cluster design, volumes holding user data are assigned to a single Vserver but can actually be located on any physical aggregate in the cluster. As a result, these volumes can be non-‐disruptively moved between physical aggregates within the cluster. With this capability, NetApp Data ONTAP 8.1.1 creates an almost immortal storage cluster. That is, simply by adding new storage technology to the cluster and migrating data from the old to new storage, the reconfigured cluster is revitalized. In fact, over a cluster’s lifetime, storage nodes can be added and/or replaced repeatedly without disruption. Moreover, as storage nodes are changed or added, data can be easily re-‐balanced across the cluster. In this fashion, IT can choose to add performance to current workloads or reserve the new performance and capacity for other activity. All this flexibility is a result of Vservers that can grow or shrink as needs dictate, such that the cluster evolves over time without disruption. Each NetApp Vserver can represent a distinct administration domain and thus, can be managed separately from any other Vserver in the cluster. As such, separate Vservers can be used to support different customers, segregating cluster data and other resources for a multi-‐tenant storage environment. Alternatively, the whole storage cluster could be configured with just one Vserver to create a single, large administration domain. With the masterly design of Data ONTAP 8.1.1, NetApp unified scale-‐out storage can increase performance and capacity in almost any combination. But, even more impressive, the number of combinations can be multiplied by adding different models of NetApp storage to a cluster. For example, multiple HA-‐pairs of FAS3240s and FAS6240s could be combined within the same cluster. In this way, IT can determine how much performance and capacity to use for a given environment. Outside of a cluster configuration such fine granularity of capacity and performance scaling is not possible. Fortunately, NetApp’s storage cluster can also include third party storage. In fact, with NetApp V-‐Series Open Storage Controllers or storage virtualization services, even third party, legacy storage arrays can provide scale out performance. V-‐Series virtualization adds advanced functionality like storage efficiency features to existing, older legacy storage arrays. Combining storage virtualization capabilities like these with clustering makes for an even more flexible environment as migration can now be non-‐disruptive not only within a V-‐Series storage complex but also across the cluster. Of course, NetApp’s other advanced features such as integrated data protection and Flash Cache can also be automatically applied to any supported backend storage. With such capabilities, NetApp V-‐Series can indeed reinvigorate less sophisticated storage.




Other new features for Data ONTAP 8.1.1 In addition to the enhanced scale-‐out features discussed above, Data ONTAP 8.1.1 offers additional new capabilities for NetApp storage. Specifically, ONTAP 8.1.1 introduces new SSD support for aggregate level read and write caching, as well as virtual appliance capabilities to their storage portfolio.

Flash Pool NetApp was the first enterprise class storage to take advantage of NAND memory in their storage controller with Flash Cache. With Flash Pool, NetApp combines new support for SSDs and disk devices into a single Data ONTAP aggregate. A Flash Pool consists of an aggregate with one or more disk RAID groups together with a single SSD RAID group. SSDs in a Flash Pool can be used as a read or a read-‐write cache for the disk data held in the aggregate. Also, disk-‐only aggregates can be non-‐disruptively upgraded to a Flash Pool by adding SSDs. Here again caching hot data on SSDs while leaving all data on disk storage can significantly improve performance.

Data ONTAP Edge or virtual storage appliance Another addition to NetApp storage with Data ONTAP 8.1.1 is a new virtual storage appliance. Data ONTAP Edge is Data ONTAP running on a virtual machine in the VMware Vsphere environment. It supports NetApp storage services using VMware direct attached storage and a backup connection to a central data center using NetApp SnapVault. The virtual appliance is customized for those remote and/or branch office environments that need shared, sophisticated storage functionality but cannot afford the cost and power of a stand-‐alone, hardware storage system. While the virtual storage appliance doesn’t support all NetApp advanced features it does support SnapVault, FlexClone, SnapRestore, deduplication and thin provisioning, enabling a remote or branch office to quickly and efficiently backup their data to a central NetApp storage system.

Other benefits of NetApp unified scale-‐out storage NetApp’s unified scale-‐out storage can also provide all the state-‐of-‐the-‐art storage efficiency features NetApp offers such as deduplication, compression and thin provisioning. These capabilities alone can often reduce physical storage capacity requirements by up to50 percent while having little impact on IO performance. In addition, NetApp’s unified scale-‐out storage provides access to NetApp’s integrated data protection services such as Snapshot and SnapMirror that can be used to easily and efficiently protect data stored within the cluster. When combined with the storage efficiency features mentioned above, Snapshot is even more space efficient and thus, less costly. Moreover, SnapMirror provides data replication or mirroring across NetApp storage systems. SnapMirror can also make use of advanced efficiency features enabled for




the source and target storage systems to reduce data footprint and data transmission requirements, saving both capacity and bandwidth.

NetApp storage portfolio With Data ONTAP 8.1.1, NetApp’s unified storage portfolio can now span from remote/branch office virtual appliances to the highest, most demanding enterprise storage customers. In addition to the new Data ONTAP Edge virtual appliance discussed above, NetApp supports a full complement of storage models. NetApp’s wide-‐ranging storage line consists of the FAS2200, FAS/V3200 and FAS/V6200 series storage systems.

NetApp FAS2200 series storage

As can be seen in Table 1, the NetApp FAS2200 series consists of the new FAS2220, the FAS2240-‐2 and the FAS2240-‐4. All FAS2200 storage systems support SAS and/or SATA disks for capacity or performance storage environments. FAS2200 series storage can be ideal for smaller to mid-‐size data centers needing networked unified storage in an economical configuration.

NetApp FAS/V3200 series storage

Table 2 shows the FAS/V3200 product line, which consists of the FAS/V3210, FAS/V3240 and the FAS/V3270 models. The FAS/V3240 and FAS/V3270 models

Table 2 NetApp FAS3200 series storage





also offer an Expanded I/O option that triples the PCIe slots, providing increased flexibility to add more frontend and/or backend ports. The FAS/V3200 product line can be a great solution for midrange to low-‐end enterprise IT storage environments.

NetApp FAS/V6200 series enterprise storage

For more demanding enterprises, needing the utmost performance and reliability, NetApp offers the FAS/V6200 series (see Table 3) consisting of the FAS/V6210, FAS/V6240 and FAS/V6280. The FAS/V6200 series can be used to satisfy wide ranging enterprise storage needs.

NetApp unified scale-‐out storage performance results NetApp has recently benchmarked its Cluster-‐Mode performance for both SAN and NAS storage clusters. The FAS6240 storage system was used to show how enterprise storage could perform in Cluster-‐Mode for more challenging data centers. The FAS6280, which is targeted at the most demanding data centers, is expected to demonstrate even better file and block performance.

NetApp Clustering file performance NetApp ran a series of SPECsfs2008 NFS Cluster-‐Mode benchmarks that ranged from a 4-‐node FAS6240 cluster all the way up to a 24-‐node FAS62401 cluster running Data ONTAP 8.1 with NFS v3. From these benchmarks, the NetApp IO performance started at ~260,000 NFS throughput operations/second for a 4-‐node cluster and scaled to an impressive, industry-‐leading ~1,500,000 NFS throughput operations/second for a 24-‐node configuration.

1 Available from SPECsfs2008 results website http://www.spec.org/sfs2008/results/sfs2008nfs.html as of 15May2012.





Thus, as can be seen in Figure 2, one extraordinary characteristic of NetApp’s Cluster-‐Mode services is the relative linearity of incremental performance increase with the addition of more storage nodes. For example, at 4-‐nodes, the per node NFS throughput was ~65.1K operations per second and at 24-‐nodes, the per node NFS throughput was still ~63.0K operations per second.

For all the Cluster-‐Mode FAS6240 benchmarks, NFS Overall Response Time (ORT) or average responsiveness ranged from a low of 1.48 to a high of 1.55 milliseconds. Thus, as shown in Figure 3, response time did not suffer by adding further nodes. In fact, with the 4-‐node storage cluster, response time was 1.53 milliseconds and with the 24-‐node storage cluster, the response time was also 1.53 milliseconds. Even with the dramatic throughput increases afforded by multiple node additions, clustering had no apparent negative impact on response times.

NetApp Clustering block (SAN) performance In addition to the file services Cluster-‐Mode benchmarks reported on above, NetApp also submitted a 6-‐node FAS6240 unified storage cluster for Storage Performance Council SPC-‐1 block performance testing running Data ONTAP 8.1.1.[2] The results for NetApp’s Cluster-‐Mode SPC-‐1 benchmark were ~250,000 SPC-‐1 IOPS™ (I/O operations per second), $6.69 $/IOPS™ ($ per I/O operations per second) and a LRT™ (least response time) of 0.99 milliseconds. This modular scale-‐out model of NetApp provides the foundation for continued growth as both controller performance and node count increases over time. In terms of relative performance, the 6-‐Node FAS6240 is in the top 10% of submitted configurations as measured by [2] Available from SPC-‐1 results website http://www.storageperformance.org/results/benchmark_results_spc1#a00115 as of 21Jun2012.

Figure 2 NetApp cluster mode performance per node

Figure 3 NetApp cluster mode overall response time




LRT and the IOPs measures represent a 267% increase in performance and 12% reduction in cost relative to the FAS3270 SPC-‐1 publication. Further, NetApp uses list pricing in their SPC-‐1 submissions while almost all other vendors use discounted pricing, resulting in a $/IOPS that is both more conservative and verifiable than the competition

Customers benefiting from NetApp’s unified scale-‐out storage As discussed above, NetApp storage clusters have been validated with both industry standard file and block storage benchmarks and as such, are unquestionably enterprise ready. As further proof, NetApp unified scale-‐out storage supports the following partial list of applications, virtualization systems and operating systems:

• Software applications – Oracle, Microsoft SQL Server, Microsoft SharePoint, Microsoft Exchange and SAP,

• Server virtualization systems – Microsoft, VMware and Citrix, • Operating systems -‐ HP-‐UX, AIX, Red Hat, SLES, Solaris and Windows.

Given this wide array of compatibility, NetApp storage clusters could readily support virtually any file or block services environment existing today.

High file access operations Moreover, because of the exceptional ~64,000 per node NFS ops/second, the powerful NetApp FAS6240 storage cluster should perform admirably for most high-‐end file services workloads. Plus, the storage cluster could be scaled to the performance required by using other NetApp models such as FAS3200 and other FAS6200 series storage. For example, a data center could start with a 2-‐node FAS3210 CIFS/SMB or NFS storage cluster to support mid-‐size data center requirements. Rapid growth could be easily and efficiently accommodated by adding more FAS3210’s or FAS3240 HA-‐pairs. Connecting FAS6200 series storage to the cluster could readily support even further rapid growth. During this evolution, storage performance would grow from thousands to over a million NFS operations per second and capacity would grow from TB to tens of PB of storage all within the same cluster and without incurring any disruption.

Random IOPS performance Similarly, almost any block storage environment can benefit from NetApp storage clusters. By moving to NetApp scale out unified storage, random block performance can scale from thousands to over a quarter of a million IO operations per second with commensurate increases in maximum capacity.

Summary NetApp’s cluster architecture has an exceedingly versatile and agile design. Not only can different NetApp storage models (11 current models plus 9 legacy models) be configured to operate in sync within the cluster, but also with NetApp’s V-‐Series




Open Storage Controller, third party legacy storage can be included for improved scale-‐out performance. In fact, NetApp is the only vendor than supports scale-‐out clusters for both SAN and NAS storage within a single operating environment. Such system flexibility allows storage clusters to enjoy a very long life, as storage can easily be upgraded, rebalanced and reinvigorated without the stress and disruption of storage replacement outages. Furthermore, by simply adding nodes, performance was incrementally increased by ~64,000 NFS ops per second in its benchmarks released in 2011. Moreover, this dramatic increase in performance had no negative impact on NFS response times. Given all these advantages, NetApp’s unified scale-‐out storage still comes with more. This storage can also provide all of the state-‐of-‐the-‐art storage efficiency features NetApp offers like deduplication, compression and thin provisioning. These substantial capacity savings can be achieved with little or no performance impact. Additionally, this storage cluster provides access to NetApp’s outstanding array of data protection services like Snapshot and SnapMirror. Truly, the NetApp unified scale-‐out storage is remarkable. Its performance, flexibility and feature–laden capabilities have certainly been designed with the enterprise class customer, from small to high-‐end, in mind. The immortality of an enterprise’s underlying data is becoming a reality. Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community.