nexenta cloud ra 110726 final2

Intel® Cloud Builders Guide to Cloud Design and Deployment on Intel® PlatformsNexentaStor™ — An Economical, Open Approach to Scale-Out Storage for Cloud Deployments

AUDIENCE AND PURPOSE

For service providers, hosting providers, or enterprise IT organizations looking to implement a cloud infrastructure, the knowledge, experience, and best practices gained from previous implementations can help to expedite planning and deployment. This paper outlines a reference architecture for a cloud storage infrastructure that integrates pivotal “scale-out” technologies including NexentaStor™ from Nexenta Systems, Inc., Intel® Xeon® processor-based servers, and Intel® X-25M and Intel® X-25E SSD Solid State Drives (SSDs). It discusses scalable architectural components, configuration options, and essential storage services. The reference architecture presented in this paper can help to reduce the learning curve for building a cost-effective scale-out storage infrastructure.

Because the deployment of a cloud requires integration and customization to accommodate the existing IT infrastructure and business requirements, it is expected that this reference architecture will be used as the foundation of an implementation and will be altered as needed. For example, integrating this storage solution with an existing infrastructure and data management tools are topics that are beyond the scope of this paper. It is expected that cloud providers will make significant adjustments to this reference architecture to meet site-specific requirements. This paper is a starting point for the journey.

Intel® Cloud Builders GuideIntel® Xeon® Processor-based ServersNexentaStor™ from Nexenta Systems, Inc.

Intel® Server System SR2625 Intel® Xeon Processor 5600 Series Intel® X-25M and Intel® X-25E SSDs

Contents

Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Scale-Out Storage Usage Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

NexentaStor Product Brief . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Synergy with Intel® Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Leveraging the Power of ZFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Functional Block Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Test Bed Blueprint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Hardware Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Hybrid Storage Pool Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

NexentaStor Software Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Technical Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

NexentaStor Appliance Installation and Initial Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Configuring NexentaStor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Configuring Auto-Sync. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Things to Consider. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Designing Hybrid Storage Pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Virtual Machine DataCenter (VMDC) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Clustered Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2

Intel® Cloud Builder Guide: NexentaStor™

Executive SummaryContent-rich media and portal applications are driving a new generation of cloud-enabled data centers and more cost-effective approaches to cloud storage. To meet escalating capacity requirements in the cloud, providers are turning to a “scale-out” storage infrastructure that combines industry-standard servers and commodity storage components. This architectural model — basing cloud storage on a set of converged storage servers — allows providers to scale resources to boost capacity, bandwidth, and I/O performance as needed while maintaining profit margins.

Nexenta Systems, Inc. embraces this scale-out storage approach with its core product — NexentaStor™ — that counters the high cost, inefficiency, and complexity of traditional proprietary storage. Until now, finding open, reliable, and easy-to-manage storage solutions has been difficult — cloud providers have had to rely on inflexible, expensive, and proprietary storage, or they’ve had to invest heavily in developing their own software-based solutions. In this reference architecture, NexentaStor creates a converged storage server solution that blends together Nexenta’s software technologies (based on open source UNIX and ZFS file system technologies) with energy-efficient Intel® servers and Intel® SSDs. Converged storage servers built on NexentaStor offer dramatic cost savings, often 70% to 80% over traditional storage solutions.

Today cloud providers are selecting NexentaStor because of its scalability, configuration flexibility, ease of management, and cost-effectiveness. NexentaStor constructs “elastic” storage pools that are independently configured and scaled, allowing providers to

meet a spectrum of requirements for capacity, performance, and business continuity, even within a single appliance. NexentaStor centralizes management of multiple appliances, allowing providers to easily reconfigure, repurpose, and scale resources to support private, public, or hybrid service delivery. Providers can also integrate appliance management with cloud provisioning and configuration management tools via NexentaStor’s RESTful APIs.

NexentaStor’s flexibility allows it to address many different usage models for cloud storage. This paper focuses on a converged storage server solution supporting a single use case, that of a cloud’s Application Data Store. The architecture defines a converged storage solution based on NexentaStor and its built-in Auto-Sync replication service.

The reference architecture is based on a real-world use case in which a cloud provider selected NexentaStor because it delivered application storage that could scale effectively and economically over time. The design is easily extended to support alternate usage models, like that of capacity-driven Large Object Stores, in response to customer requirements.

Introduction NexentaStor consists of software services — hosted on Intel® Xeon® processor-based servers — that can manage a heterogeneous variety of back-end storage devices, from cost-effective JBODs to high-performing Intel® SSDs. For this reference architecture, Nexenta suggests the Intel® Server System SR2625 for NexentaStor appliance hosts, along with Intel® X-25M and Intel® X-25E SSDs.

NexentaStor allows providers to create storage pools that address a variety of applications and configuration requirements. Even within a single appliance, providers can tailor storage volumes, optimizing them for desktop virtualization, Web and email services, database, and other strategic data-driven applications.

This paper describes a NexentaStor implementation that meets general-purpose application storage needs for public, private, or hybrid clouds. It suggests a “pod” approach based on rack-level components (NexentaStor appliances, storage arrays, etc.) that can be scaled to meet different usage models and working set sizes, performance targets, availability goals, and price points.

Scale-Out Storage Usage Models

Application Data Store

Application-driven data, particularly file-based data, is growing rapidly, driving the need for greater storage capacities, even though IT budgets are declining or static. For cloud providers, NexentaStor offers scale-out storage in which resources can be added independently to meet application growth and increasing user populations.

Large Object Store

The explosive rise in digital content creation, especially with streaming video, voice, and image data, continues to push providers to offer new services and to expand capacity. NexentaStor is an ideal solution for this usage model since it enables cost-effective growth into the petabyte range and beyond. NexentaStor’s configuration flexibility allows providers to optimize and scale performance as well as capacity.

3


NexentaStor Product BriefLike many converged storage server solutions, NexentaStor brings dramatic cost-savings compared to traditional solutions, but it also delivers enterprise-class data integrity, optimized capacity utilization, and simplified management. Each NexentaStor server functions as a virtualization appliance that abstracts legacy and commodity storage devices, pools storage resources, and facilitates file and block access. As Figure 1 depicts, the solution supports both SAN and NAS, providing flexible data access across a number of protocols (e.g., CIFS, NFS, iSCSI, HTTP, FTP, WebDAV, and Fibre Channel).

NexentaStor offers four key advantages:

1. NexentaStor enables cost-effective scale-out storage. Because providers can deploy industry-standard Intel® servers and commodity storage arrays, they are able to design solutions that meet service delivery requirements within budget constraints. With the open NexentaStor solution, there is no vendor lock-in. The storage infrastructure scales by adding local or remote rack-based “pods” containing NexentaStor appliances, high-performance Intel® servers, and commodity storage arrays.

2. NexentaStor offers built-in virtualization and thin provisioning. NexentaStor abstracts and pools underlying storage independent of interconnects and protocols. Providers can provision storage resources as needed, growing and shrinking them to optimize utilization. Configuration options such as compression and deduplication conserve capacity and reduce the number of I/Os, which also helps to improve performance. Storage architects can implement NexentaStor using a hybrid storage pool approach, adding fast SSDs to certain pools to accelerate performance-sensitive workloads.

3. NexentaStor simplifies storage management. To the core open source code base, Nexenta adds an intuitive data management front-end for administering all appliances in the scale-out solution. Both a console and web-based GUI are included. In addition, NexentaStor can be integrated with common cloud management tools to streamline configuration and management. Using RESTful appliance APIs, providers can link NexentaStor into the BSS/OSS (Business and Operational Support System) that manages cloud service provisioning, billing, and revenue accounting.

4. NexentaStor can be configured to meet a range of availability and data redundancy requirements. NexentaStor simplifies device pooling and RAID volume configurations, supporting mirrored, RAID-Z1 (single parity), RAID-Z2 (double parity), and RAID-Z3 (triple parity) configurations. Nexenta supplies services that enable heterogeneous block and

file replication (either synchronously or asynchronously), facilitating disaster recovery from local or remote sites. In addition, NexentaStor appliances can be configured as clustered pairs, supporting automated failover and eliminating the appliance as a single point of failure.

Synergy with Intel® Technologies

A scale-out storage architecture shifts the burden of compute-intensive workloads into the storage tier where they can take advantage of underlying Intel® server technologies. For example, NexentaStor leverages multi-core, threaded compute engines that perform CPU-intensive inline deduplication and compression operations for storage workloads. With enhanced power management features, Intel® Xeon® processors deliver high performance while maintaining low power consumption. To better protect stored data, optimizations for kernel cryptography leverage Intel® Xeon® processor instructions to enable

Figure 1. NexentaStor consolidates heterogeneous storage assets and exposes them via industry-standard protocols, with a unified management front-end.

4


fast and secure data encryption and decryption. Since Intel® servers feature high-bandwidth and low latency access to memory, NexentaStor can achieve high cache hit ratios for many workloads when providers configure these servers with large memories. Effective caching helps to optimize performance of scale-out NexentaStor designs.

Leveraging the Power of ZFS

NexentaStor is built on an open source UNIX foundation and is based on ZFS, an advanced enterprise file system. ZFS underpinnings bring specific benefits to NexentaStor for cloud storage:

• Virtually unlimited file system scalability to support large-scale capacities. The maximum size of an individual file or file system in NexentaStor is 16 EiB (264 bytes). This basically removes any practical limits on the size of files, directories, and file systems.

• Inherent storage virtualization. Nexenta’s ZFS implementation enables thin provisioning, seamless growth, and improved resource utilization. NexentaStor pools blocks from almost any type of underlying device (SAS, iSCSI, FC, IDE, USB, SSD, CF, etc.). Capacity can be added dynamically, without interrupting services. Excess capacity can be released and made available to other volumes as needed.

• Increased data integrity. ZFS uses end-to-end checksumming and transactional copy-on-write to prevent silent data corruption. NexentaStor supports a variety of redundancy options when creating volumes, including mirroring and RAID-Z. RAID-Z1 is similar to RAID-5, providing single-parity protection, but RAID-Z prevents stripe corruption or “write holes” (caused by the inherent read-modify-write architecture that RAID 5 uses). NexentaStor also supports RAID-Z2, which provides double-parity protection (similar to RAID-6), and RAID-Z3, which enables triple-parity protection.

• Accelerated write performance. ZFS’s block allocation algorithms condense an I/O stream of many small random writes into a single, more efficient, sequential write operation.

• Virtually unlimited snapshot and cloning. The Nexenta ZFS implementation supports replication and system cloning through point-in-time copies or “snapshots.” Snapshots can be used to speed both VM cloning and migration, as well as appliance provisioning and data replication.

• Deduplication. Administrators can optionally configure data volumes for inline deduplication, which ZFS intrinsically supports. Deduplication conserves capacity and decreases I/Os when data is highly redundant.

• Compression. Like deduplication, compression conserves space and reduces I/O. ZFS compression occurs at the block level and not at the file level, as in other file systems. For cloud deployments, compression is often an effective option for optimizing capacity of NexentaStor storage volumes.

• Intelligent data replication. Through ZFS “send” and “receive” commands, NexentaStor replication services support highly efficient, intelligent snapshot copies. Once initial replication occurs, only the differences between source and destination snapshots are subsequently transferred, conserving network bandwidth and the number of I/O operations.

5


Functional Block Diagram

Figure 2 shows the main components of the NexentaStor software architecture — the Nexenta Management Server and its clients: the Nexenta Management Console (NMC), the Nexenta Management View (NMV), NexentaStor Runners, Nexenta Second-Tier Storage Services, and NexentaStor Plug-ins.

NexentaStor clients are based on open, published Nexenta Storage Appliance APIs (SA-APIs). The SA-APIs provide access to appliance management objects (such as LUNs, volumes, snapshots, folders, etc.), as well as storage management services.

Nexenta publishes its APIs to allow developers to extend or build new storage management services. Open Nexenta APIs provide a means of expanding the core product’s capabilities, promoting innovation from service providers, partners, and the entire open source community.

Nexenta Management Server

The Nexenta Management Server is responsible for all storage and network services, including volume configuration and fault management. The server uses NexentaStor Runners — client processes that monitor faults, collect statistics, and generate reports.

NMV and NMC Interfaces

The web-based GUI (NMV) and perl-based console (NMC) centralize management for all NexentaStor appliances, allowing a remote administrator to create, grow, delete, export, and import storage volumes. The same functionality can be made available to cloud infrastructure management solutions via integration with SA-APIs. Collected performance data can be viewed graphically or sent to SNMP-based management tools.

Plug-Ins and Second-Tier Services

Nexenta supplies additional storage services or “plug-in” extension modules written in its published APIs (see www.nexenta.com/corp/plugins-products for a complete list). Cloud providers typically deploy one of the replication services (Auto-Tier, Auto-Sync, or Auto-CDP), along with the Virtual Machine DataCenter (VMDC) and HA Cluster plug-ins, depending on specific application and availability requirements.

Replication Services

Built into the core services of the NexentaStor software, Auto-Tier and Auto-Sync services provide asynchronous replication for backup, archiving, and disaster recovery. Auto-Tier copies files and directories, while Auto-Sync replicates file system metadata and snapshots to build local or remote volume copies. Both

support intelligent replication, copying only changed blocks to conserve network bandwidth and I/O resources.

Auto-Sync is a key element in this reference architecture since it provides data redundancy in a distributed NexentaStor infrastructure. When providers deploy multiple NexentaStor appliances, either locally or remotely, Auto-Sync’s one-to-many intelligent replication capabilities replicate storage volumes at each physical site. If a problem prevents applications from accessing an appliance, they can potentially access replicated storage volumes on another appliance. Later, when the problem is resolved, updated volumes can be restored by replicating in reverse. NexentaStor replication also enables fast appliance cloning, and is commonly used to support data protection and disaster recovery plans.

Figure 2. Components in the NexentaStor software architecture.

6


www.nexenta.com/corp/plugins-products

Test Bed BlueprintThe reference architecture deploys one or more storage pods at two different sites, one local and one remote (Figure 3). Each pod contains hardware and software components listed in Table 1 and illustrated in the rack configuration in Figure 4. The workload involves using Auto-Sync to replicate NexentaStor data volumes from the local site to the remote site. Note that Auto-Sync supports replication to multiple NexentaStor appliances, even though this paper discusses how to set up replication between just two appliances.

To meet the needs of general-purpose cloud storage, NexentaStor appliances can be easily scaled using a “pod” approach. A “pod” consists of three major components: (1) the NexentaStor appliance head, (2) multiple JBOD storage arrays, and (3) the NexentaStor software. The NexentaStor solution scales easily by duplicating pods — by configuring ancillary NexentaStor appliances and additional storage arrays to meet performance, capacity, and availability needs.

Table 1. Components in a Typical NexentaStor Pod.

Pod Component Configuration

NexentaStor Appliances Intel® Server System SR2625 (see the related Product Brief):

• Intel® Server Board S5520UR

• 96 GB DDR3 DRAM

• Intel® 5500 Series Processors

• Intel® Ethernet X520 Server Adapter (dual-port 10 Gb Ethernet NIC)

• LSI 9200-16e Host Bus Adapter (HBA)

• Internal boot drives, 40GB or larger

Storage Arrays DataON DNS-1600 4u, 24 disk JBOD chassis, 3.5”

• Dual-active 6Gbps SAS I/O Modules

• Seagate Constellation ES.2 HDDs

• Intel® X-25M SSD with SATA/SAS interposers

• Intel® X-25E SSD with SATA/SAS interposers

Software • NexentaStor Enterprise Edition, Version 3.0.5

• NexentaStor Auto-Sync service, Version 3.0.4

• NexentaStor VMDC plug-in, Version 3.2.1 (optional)

• NexentaStor HA Cluster plug-in, Version 2.4.1 (optional)

Figure 3. NexentaStor Reference Architecture Site Diagram.

7


http://cache-www.intel.com/cd/00/00/41/40/414099_414099.pdf

Figure 4. Rack Containing Components for NexentaStor Pod.

Hardware Components

Figure 4 depicts the NexentaStor pod hardware components. The reference architecture’s rack configuration includes two NexentaStor appliances and four arrays, as well as switches and compute nodes to support virtualization and other applications. Each NexentaStor appliance consists of an Intel® Server System SR2625, which hosts the NexentaStor software and houses an LSI SAS 9200-16e host bus adapter. The adapter features 16 high-speed 6Gb/s ports to support large-scale storage array designs.

In the reference architecture, each NexentaStor appliance manages two of the four DataON JBOD arrays. Each array houses up to 24 3.5-inch devices. Cloud service providers must determine needed disk quantities, device capacities, and speeds, taking into consideration factors such as price/performance, the expected working set size, and specific application performance requirements.

When each array is populated with 20 6-Gb/s, 2TB Seagate Constellation ES.2 SAS drives (configured with RAIDZ2 double parity protection), the rack provides approximately 120TB of usable pooled storage. Mirrored Intel® X-25M and Intel® X-25E SSD devices populate the remaining chassis slots, forming a hybrid storage pool.

Hybrid Storage Pool Design

Since Nexenta’s ZFS implementation abstracts and pools storage devices, it is easy to deploy NexentaStor volumes using hybrid storage pools. A hybrid storage pool facilitates design trade-offs between device performance, capacity, availability, and cost.

For example, the reference architecture adds high-speed but more expensive SSDs into the storage hierarchy between costly RAM and more economical, larger capacity hard disk drives (see Figure 5). SSDs are applied to accelerate read and write I/Os, allowing the configuration to meet more demanding I/O performance targets.

Figure 5. Hybrid Storage Pool Components.

As illustrated in Figure 5, key components of the hybrid storage pool include:

• Adaptive Replacement Cache (ARC). This is the primary ZFS cache in main memory. The large memory capacities (e.g., up to 192GB) available on Intel® servers enable high cache hit ratios, especially in contrast with many proprietary storage systems that have limited cache sizes.

• Level Two Adaptive Replacement Cache (L2ARC). The L2ARC provides a larger, second-level cache to accelerate reads. In the reference architecture, 160GB Intel® X-25M SSDs are used to cache reads for cloud storage.

• ZFS Intent Log (ZIL). A separate intent log allows synchronous writes to be quickly written and acknowledged in ZFS’s transactional model. In this reference architecture, mirrored 32GB Intel® X-25E SSDs accelerate write I/Os for the cloud storage workload.

Hybrid storage pool designs allow providers to configure NexentaStor to meet the needs of different workloads and usage models. “Things to Consider” on page 16 describes several approaches that architects can take when optimizing NexentaStor configurations.

8


NexentaStor Software Components

The reference architecture uses these NexentaStor software components:

• NexentaStor Enterprise Edition, Version 3.0.5

• The no-charge NexentaStor Auto-Sync service, Version 3.0.4

Auto-Sync Replication

The reference architecture focuses on replicating NexentaStor data volumes from one appliance to another using the NexentaStor Auto-Sync service. Auto-Sync is a schedulable, fault-managed, and fully configurable replication service. Version 3.0.4 of the Auto-Sync service includes these features:

• Intelligent replication. Only changed blocks are copied after the initial snapshot replication.

• Deduplication. Auto-Sync performs inline deduplication on data prior to transmission, saving network bandwidth.

• Automated recovery. The Auto-Sync service automatically resumes after a network failure or unexpected power outage.

• Flexible scheduling. A storage administrator can configure Auto-Sync to perform incremental replications periodically, running in daemon mode or at certain times of the day.

• System folder/root replication. This capability simplifies appliance cloning.

• Reverse direction support. All possible directions are supported: local-to-local, local-to-remote, remote-to-local.

• Bandwidth throttling. Auto-Sync features the ability to limit network and I/O bandwidth consumed by the replication process.

• Configurable retention policy. Auto-Sync retains automatically generated snapshots at the source or destination for a specified number of days.

• High-performance transport. A fast mechanism based “netcat” (rather than SSH pipes) enables rapid data transport. Table 2 lists test results comparing netcat performance, SSH encryption methods, and raw disk speeds. Note how the netcat transport method achieves performance close to disk reads.

Table 2. Transport Comparison.

Method Speed, MB/sec

ssh/default (aes 128-cbc) 26.5

cat file < ./dev/null (disk read speed)

108.41

netcat 102.09

Technical ReviewThis section describes appliance and workload configuration. Specifically, it shows the steps to configure storage volumes and replication services on the NexentaStor appliances.

NexentaStor Appliance Installation and Initial Configuration

On both appliances, the NexentaStor software is installed on the server’s internal boot disks (which are themselves mirrored for reliability). Once installed, there are three simple steps to deploy the appliance at each location:

1. Register the appliance.

2. Configure the primary network interface,

3. Perform a few essential wizard-guided configuration steps.

The basic appliance configuration steps include defining root and admin passwords, network parameters, and e-mail addresses for statistics and notifications.

Configuring NexentaStor

A secondary wizard guides the process of NexentaStor volume definition and configuration. Under the Data Management -> Data Sets tab, create a new NexentaStor volume (called “pool1” in Figure 6) on the local appliance. Select RAID Z2 (double parity) for data protection and available devices to create the volume configuration.

9


Figure 6. NexentaStor Volume Configuration.

10


Next, create a NexentaStor folder, which will be shared using NFS. For general-purpose cloud storage, Nexenta recommends these folder configuration options:

• Block size. For fixed record-length applications, such as databases, NexentaStor exhibits optimal performance when the block or record size is specifically matched to the workload. For many applications, especially for VDI workloads or iSCSI or FC Zvols, a 4K block size is optimal, since these workloads tend to generate I/Os that are 4K, or multiples of 4K, in size. For folder access via NFS or CIFS, the default 128K record size is usually the most effective block size setting.

• Compression. In testing of VDI and other general-purpose workloads, the default compression algorithm in NexentaStor can save approximately 30% of raw disk storage. In some cases, compression can improve performance because fewer physical I/O operations are required. If CPU resources to compress or decompress data are readily available, a few hundred microseconds of CPU time for compression can save milliseconds of disk I/O time.

Figure 7. NexentaStor Folder Configuration Options.

Before configuring Auto-Sync services, install the appliance at the remote location and follow the same steps above to define and configure a NexentaStor volume and folder that will contain the replicated data.

11


Configuring Auto-Sync

Auto-Sync is shipped as a part of the core NextentaStor software distribution. To see if the Auto-sync service is active, go to Settings ->Appliance-> Plugin and choose “autosync” from the list of available plug-ins and services (Figure 8). The NMS server must be restarted if the service is not already active.

Figure 8. Auto-Sync Installation.

12


Setting Up Auto-Sync Replication

The NexentaStor NMV GUI provides the complete set of tools to create, delete and manage Auto-Sync services. To create a new instance of the Auto-Sync service, click on Data Management-> Auto-Services-> Auto-Sync Services and click ‘Create.” Figure 9 shows the user interface.

Figure 9. Setting Up the Auto-Sync Service.

13


Select “To Remote Host” as the replication direction and specify the source folder to replicate. Enter the IP address and folder for the destination appliance, along with a replication interval. The interval consists of two parameters: period and frequency. Frequency values depend on the time period specified — for example if period is set to “hourly,” a frequency value of 4 means that the service will run every 4 hours.

There are also options to turn on deduplication or compression of the replication stream. While deduplication consumes CPU power, it can lower the utilization of network bandwidth for replicated data. The compression option allows the administrator to specify a compression algorithm for the replication stream. The decision to use compression involves a trade-off between the availability of CPU cycles to compress data and the need to conserve space.

Note that the compression algorithm used at the replication source can be different than the algorithm used at the replica. It is also possible to turn off compression at the replication source (since it is CPU-intensive) but to use it at the destination to conserve capacity. Since decompression is less CPU-intensive, this can be a reasonable trade-off. Algorithm flexibility allows the replica to have a more CPU-intensive compression algorithm that can deliver greater space savings, which can prove useful when implementing data backup mechanisms at remote sites.

Other default settings for Auto-Sync (keep days = 7, service retry = yes, etc.) are generally acceptable for most replication workloads. After specifying the required properties, click “Create Service.” When the service instance is successfully created, it appears in the “Services” and “All Storage Services” listings, as shown in Figure 10.

Figure 10. Viewing Defined Auto-Sync Services.

14


Verifying Auto-Sync Replication

The Auto-Sync replication service can be monitored in real time in terms of bytes transferred and bandwidth. The following NMC console command shows Auto-Sync generated traffic at one- second intervals:

nmc$ show auto-sync :vol1-a-000 stats -i 1TCP CONNECTION SNEXT RNEXT TRANSFERED172.16.34.21.39305=>172.16.3.20.22 2176247586 4217818800 -172.16.34.21.39305=>172.16.3.20.22 2176255378 4217819152 8.14 KB172.16.34.21.39305=>172.16.3.20.22 2176266306 4217819632 11.41 KB172.16.34.21.39305=>172.16.3.20.22 2176273922 4217819952 7.94 KB

This NMC command shows an excerpt of the log file and the success of the most recent replication:

nmc$ show auto-sync : logtail -f[ Jan 25 05:30:17 (zfs-auto-sync, 10545) EXEC: zfs get -r -Hp creation,nms:service,referenced,used pool1/data1 ][ Jan 25 05:30:17 (zfs-auto-sync, 10545) pool1/data1 task size 0 Kb ][ Jan 25 05:30:17 (zfs-auto-sync, 10545) Summary will be sent 0 Kb ][ Jan 25 05:30:17 (zfs-auto-sync, 10545) Performing tasks send. ][ Jan 25 05:30:17 (zfs-auto-sync, 10545) Output cmd: rrmgr -sRpeFvvvu -P1024 -n4 -I pool1/data1@AutoSync:20 _ 2011-01-25-05:20:09 172.16.3.20:volume1 pool1/data1@AutoSync:21 _ 2011-01-25-05:30:14 ]...[ Jan 25 05:30:20 (zfs-auto-sync, 10545) SUCCESS: replication duration - 7 seconds. ]

There are several ways to confirm that Auto-Sync has successfully transferred data from the source to destination. The following method checks the Auto-Sync transport itself:

nmc$ show auto-sync :data-folder-000 stats -i 1TCP CONNECTIONS SNEXT RNEXT TRANSFER172.16.34.21.22-172.16.3.20.36373 3552678986 3152922376 -172.16.34.21.22-172.16.3.20.36373 3552679178 3152922376 192 B172.16.34.21.22-172.16.3.20.36373 3552679290 3152922376 112 B172.16.34.21.22-172.16.3.20.36373 3552679402 3152922376 112 B

15


Things to ConsiderNexentaStor offers tremendous flexibility in designing the cloud storage tier. The software supports a wide range of interface protocols and device types. The resulting capacity, performance, and reliability characteristics, of course, depend on the exact components and configuration. Since NexentaStor virtualizes underlying storage, a broad range of hybrid storage pool designs is possible. Storage architects can easily construct pool designs that match workload characteristics for a particular usage model. In all configurations, Intel® Xeon® processor-based servers deliver the threaded compute power and large memory capacities needed to support the NexentaStor solution while conserving power and footprint.

Designing Hybrid Storage Pools

The reference architecture describes a hybrid storage pool that functions as a general-purpose Application Data Store. For application-driven storage, response time is critical to user satisfaction, so storage architects optimize NexentaStor in this usage model for performance. (High-performance Intel® SSDs are added to the configuration to reduce I/O latency.) For other usage models, such as that of a Large Object Store, the architect designs the configuration with capacity as the primary design goal. Multiple high-performance Intel® servers in multiple pods (to achieve petabyte scale) would create a capacity-optimized design. Figure 11 illustrates different configuration possibilities for these two types of usage models.

Figure 11. Hybrid Storage Pool Configurations.

Optimizing for Performance

Minimally, a NexentaStor configuration can run on strictly low-cost HDDs in the main storage pool, ideally with mirrored drives for reliability and performance. For an Application Data Store deployment that demands reliability, the “good” configuration (top left, Figure 11) represents a starting point. Because Intel® SSDs enable low-latency reads and writes, these devices are added as logs to accelerate performance: Intel® SSDs are mirrored in the ZFS Intent Log (ZIL), which acts as a write cache, and an Intel® SSD is used as an L2ARC read cache. Hard disk drives in the main storage pool are configured using RAID-Z, which uses striping with parity (similar to RAID-5 but with atomic operations)

16


for reliability with low cost. At the next increment (labeled “better”), both mirroring and striping are used to achieve additional levels of reliability and performance. Striping can be used for the L2ARC, allowing reads to be spread concurrently across multiple SSD stripes. The “better” configuration is commonly used to support Application Data Stores.

A highly optimal configuration (labeled “best”) implements the entire storage pool using solely Intel® SSDs, which yields the highest performance levels although it does imply a greater device cost. RAID-Z can reduce the number of Intel® SSDs required, or SSDs can be mirrored and striped to optimize performance and reliability. While at first implementing SSDs exclusively may seem extraordinary from a cost perspective, it can be a viable option for storage volumes that host performance-sensitive workloads. Compression, which takes advantage of powerful Intel® Xeon® processors, can be an effective option to conserve space and further improve performance.

Optimizing for Capacity

For Large Object Store deployments (right, Figure 11), the primary design goal is to maximize capacity, making performance a secondary consideration. When a storage architect focuses on cost-per-terabyte (rather than cost-per-IOP), the “good,” “better,” and “best” configurations reverse. Although a configuration on solely SSDs offers performance benefits, this approach is not usually cost-effective when configuring the massive storage capacity needed for a Large Object Store. The “better” configuration uses less costly, high-capacity HDDs in the main storage pool, with drives configured with RAID-Z for reliability with low cost. The “best” configuration eliminates the log devices, since the logs do not offer much benefit for this type of workload.

Mixed Usage Models

Given NexentaStor’s design flexibility, service providers have many options when architecting deployments to meet specific application and business requirements. As a matter of fact, all three options shown in Figure 11 can be deployed within a single NexentaStor appliance or infrastructure.

By configuring volumes or folders on “good,” “better,” and “best” storage pools, cloud providers can offer their customers premium storage options with a range of I/O performance and capacity (and charge corresponding service fees). Auto-Sync and Auto-Tier replication services make moving data from one pool to another a straightforward task in the event a customer chooses to change the level of service.

Providers can easily integrate NexentaStor into an existing infrastructure, especially since it supports multiple device types and protocols. Nexenta SA-APIs allow providers to extend existing management services, management clients, and applications on top of NexentaStor, or to construct and merge in other functionality.

17


Virtual Machine DataCenter (VMDC)

Recommended for all VDI deployments, the VMDC plug-in provides tight integration with Citrix XenServer, VMware ESX, and Microsoft Hyper-V, supporting all three hypervisors from a consistent interface (Figure 12). With the VMDC plug-in, administrators can clone VMs quickly at the storage level, define VM-specific storage policies, and easily relocate VMs between appliances.

Figure 12. The NexentaStor VMDC plug-in centralizes virtualization management.

Clustered Configuration

For cloud deployments, two NexentaStor configuration patterns are common. In the cases where high availability with automated failover is tantamount, service providers can deploy clustered pairs of NexentaStor servers with the Nexenta HA Cluster plug-in. In the highly competitive cloud computing market, however, many service providers seek to optimize what are often already narrow profit margins. For this reason, providers generally prefer to configure NexentaStor appliances for data redundancy rather than with clustered systems for service failover. In both configuration patterns, highly reliable Intel® servers are used, and Nexenta Auto-Sync or Auto-Tier services are configured to replicate data to local or remote sites in support of backup applications and disaster recovery plans.

Some deployments have more stringent availability requirements beyond what replication services and RAID configurations can provide. For these sites, the HA Cluster module is installed on pairs of NexentaStor appliances to create active/active HA clustered pairs. The HA Cluster plug-in facilitates continuity in the event of service level exceptions, such as power outages, disk failures, lost network connectivity, insufficient appliance memory, or unplanned downtime of a single appliance node.

18


Clustered nodes run a defined set of services, including volume sharing services, and continuously monitor each other for failures. If a node detects a system failure, the ownership of shared volumes is transferred to the surviving node. Although both nodes actively control the shared storage, any given shared volume is owned by only one node at any given time.

Figure 12 shows the logical architecture of a NexentaStor clustered deployment that supports application data storage. From a provisioning perspective, there are three layers: Cloud Management, VM and Storage Services, and Back-End Storage.

At the top of Figure 12, cloud management nodes interface to NexentaStor through the storage appliance APIs. These RESTful APIs allow cloud management tools and virtualization solutions to access underlying NexentaStor functionality at the middle tier.

NexentaStor storage and virtualization services reside in the middle tier. This tier serves two key functions: to pool back-end devices into shared storage volumes, and to provision VMs using those volumes.

Appliance APIs manage and monitor all aspects of NexentaStor appliances, including underlying appliance objects (LUNs, snapshots, volumes, etc.), as well as network and data management services. Based on API client requests, the Nexenta Management Server abstracts back-end storage and thinly provisions new volumes. Since NexentaStor takes advantage of ZFS snapshots and the ease of snapshot cloning, virtualization services can leverage the APIs to rapidly clone VMs on demand. Through API integration, administrators can use integrated cloud infrastructure and virtualization management tools to configure and provision VMs using NexentaStor.

Figure 12. Logical architecture of NexentaStor clustered appliance deployment.

19


Shared Volume Configuration

The NexentaStor HA Cluster plug-in is installed on each cluster node. One or more shared volumes are created in the underlying storage repository. This can be done either through API integration, using NexentaStor console commands, or using the NMV GUI, as shown in Figure 13.

Figure 13. NexentaStor HA Cluster GUI.

Additionally, NexentaStor HA Cluster nodes typically run storage replication services such as Auto-Tier, Auto-Sync, or Auto-CDP (Continuous Data Protection). Auto-CDP supports synchronous replication, copying data between two different appliances in real time at the block level, so it is optimally used with relatively small data volumes. As part of the failover process, the HA Cluster plug-in migrates any of these three replication services associated with the shared volume(s) and restarts them on the alternate node.

Communication Requirements

The HA Cluster plug-in relies on multiple communication channels to exchange system and service state, such as:

• One or more disks in a shared volume

• Multiple network interfaces on different subnets, including aggregated links

Multiple, redundant heartbeats counter the possibility of “split brain” syndrome, in which an appliance wrongly detects a failure and attempts to start a service that is already running on the other node. Also, to avoid possible IP routing problems, appliances can also exchange heartbeats over a dedicated serial link using a custom protocol.

20


ConclusionThe need to scale storage resources — in concert with the goals of controlling cost, optimizing availability, and increasing performance — can be a challenge given the constraints of today’s IT budgets. To address this, many cloud providers are turning to “scale-out” storage architectures like NexentaStor that offer a “pay-as-you-grow” approach to expanding the storage infrastructure.

NexentaStor leverages open software technologies, industry-standard Intel® servers, and commodity storage components. Using converged storage servers based on NexentaStor, cloud providers can realize tremendous savings while deploying an enterprise-class solution that is open, reliable, and easy- to-manage.

Cloud providers can tailor NexentaStor configurations to meet different usage models, optimizing configurations for performance and capacity to meet

As shown in this paper, providers can implement NexentaStor’s replication services to achieve cost-effective data redundancy. The Auto-Sync service provides an economical means of replicating data to multiple local and remote appliances.

NexentaStor allows cloud providers to configure — on demand and at highly competitive price points — petabytes of high-performance, highly efficient, and highly available enterprise-class storage. The combination of NexentaStor and energy-efficient Intel® servers helps can help to lower CAPEX and OPEX for cloud storage. For more information, and to download a no-charge, fully functional version of NexentaStor™ Enterprise Edition for a 45-day evaluation, visit www.nexenta.com.

workload requirements. To support performance-driven Application Data Stores, cloud providers can structure hybrid storage pools that leverage Intel® server and SSD technologies. NexentaStor also makes it easy to build Large Object Stores cost-effectively, allowing providers to deploy high-capacity solutions at often 70-80% below the cost of comparable proprietary storage.

NexentaStor’s centralized configuration and management tools help to counter the complexities of many scale-out designs. Using the command line or GUI interfaces (or using NexentaStor APIs to integrate the solution with third-party provisioning tools), storage administrators can abstract and aggregate raw devices into elastic storage pools and configure large-scale storage volumes. Virtualization, thin provisioning, and compression options optimize utilization and performance for each workload.

21


Disclaimers

To learn more about deployment of cloud solutions, visit www.intel.com/software/cloudbuilder

For more information on the Intel® Xeon processors, see www.intel.com/xeon.

For more information on the Intel® Solid-State Drives, see www.intel.com/design/flash/nand.

For more information on the Intel® Server System SR2625, see the related Product Brief (cache-www.intel.com/cd/00/00/41/40/414099_414099.pdf)

For more information on NexentaStor™, see www.nexenta.com

Disclaimers∆ Intel processor numbers are not a measure of performance. Processor numbers differentiate features within each processor family, not across different processor families. See www.intel.com/products/processor_number for details.

INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL® PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL’S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. UNLESS OTHERWISE AGREED IN WRITING BY INTEL, THE INTEL PRODUCTS ARE NOT DESIGNED NOR INTENDED FOR ANY APPLICATION IN WHICH THE FAILURE OF THE INTEL PRODUCT COULD CREATE A SITUATION WHERE PERSONAL INJURY OR DEATH MAY OCCUR.

Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked “reserved” or “undefined.” Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The information here is subject to change without notice. Do not finalize a design with this information.

The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request. Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order. Copies of documents which have an order number and are referenced in this document, or other Intel literature, may be obtained by calling 1-800-548-4725, or by visiting Intel’s Web site at www.intel.com.

Copyright © 2011 Intel Corporation. All rights reserved. Intel, the Intel logo, Xeon, and Xeon inside are trademarks of Intel Corporation in the U.S. and other countries.

*Other names and brands may be claimed as the property of others.

Print Code SKU


http://www.intel.com/software/cloudbuilder

www.intel.com/xeon

http://www.intel.com/design/flash/nand

http://www.intel.com/design/flash/nand

www.nexenta.com

nexenta cloud ra 110726 final2

Documents