pcloudstorage performance results wp en ltr 02192013 web

8/11/2019 PCloudStorage Performance Results WP en Ltr 02192013 Web

1/14

White Paper

www.parallels.com

Parallels

Cloud Storage

Performance Benchmark Results


2/14

Parallels Performance Benchmark Results

2

Executive Summary ...............................................................................................................................3

Architecture Overview ............................................................................................................................ 3

Key Features .........................................................................................................................................4

No Special Hardware Requirements ................................................................................................. 4High Reliability .................................................................................................................................. 5

Extensive Scalability and Flexibility....................................................................................................5

Top Performance..............................................................................................................................6

Replication in Parallels Cloud Storage .................................................................................................... 6

Overview .........................................................................................................................................6

Results of Replication Performance Tests ........................................................................................ 8

I/O Performance Tests ........................................................................................................................... 9

Overview .........................................................................................................................................9

Test Results for Horizontal Scalability ............................................................................................ 10 Test Results for Vertical Scalability ................................................................................................. 11

Conclusion ..........................................................................................................................................12

Appendix: Test Lab Configurations and Methodologies ........................................................................ 13

Replication Testing ........................................................................................................................ 13

Performance Testing ...................................................................................................................... 14

Table of Contents


3/14


3

Executive Summary

To support growing demands for both high performance and high data availability, modern data centers

need a fast, flexible storage solution. But this requirement presents problems: besides the difficulties

involved in managing and maintaining such storage, there are flexibility and cost issues. Redundant

storage is typically either not flexible (as is the case with local RAID arrays) or too expensive for many

organizations (as is the case with SAN storage).

Parallels Cloud Storage is designed to solve both problems. It can run on commodity hardware, thus not

requiring any significant infrastructure investment; and it is a very easy solution to set up and grow on

demand.

This white paper provides a description of the architecture, major features, and performance of Parallels

Cloud Storage.

Architecture OverviewParallels Cloud Storage is a highly available distributed storage system with built-in replication and

disaster recovery. It runs on top of commodity hardware, using locally attached hard drives to create

storage clusters that span multiple machines. This approach allows rapid live migration of containers

(CTs) and virtual machines (VMs) across hardware nodes, without having to copy CT or VM data. It also

ensures high availability, as it makes storage across the cluster available remotely.

Figure 1 illustrates the logical structure and basic components of Parallels Cloud Storage. The key

components are:

Chunk Servers (CSs).All data in a Parallels Cloud Storage cluster, including disk images of both CTsand VMs, is stored in the form of fixed-size chunks on chunk servers, which provide access to the

data as needed. The cluster automatically replicates the data chunks and distributes them across the

available chunk servers to provide high availability of data. Therefore, to ensure this high availability, aParallels Cloud Storage cluster must have multiple chunk servers.

Metadata servers (MDSs).To keep track of data chunks and their replicas, the cluster storesmetadata about them (such as their file names) on metadata servers. In addition to managing

metadata, the MDSs control how files are split into chunks and where the chunks are stored. They

also track versions of chunks, ensure that the cluster has enough replicas, and keep a global log of

important events that happen in the cluster. As is the case with CSs, multiple MDSs are needed to

provide high availability.

Clients.Clients manipulate data stored in the cluster by sending different types of file requestsfor example, to modify an existing file or create a new one. Clients access a storage cluster by

communicating with the MDSs and CSs. Parallels CT and VM clients can be run nativelyi.e., directly

from the storage cluster. You can also mount storage as a conventional file system (although ParallelsCloud Storage is not POSIX-compliant). In addition, you can create and mount image files residing

on storage as loop devices by using the ploop feature. You can format these loop devices in any file

system you want (ext4, for instance) and can export them using iSCSI.

Parallels3


4/14


4

Figure 1. Key components of Parallels Cloud Storage

A recommended cluster setup would consist of three to five MDS instances (allowing you to survive

the loss of one or two of the MDSs, respectively) and three or more CSs to provide storage capacity.

(For more information, see the Recommended Configuration chapter in the Parallels Cloud Storage

Administrators Guide.)

You can set up any number of computers in the cluster to perform the role of an MDS, CS, or client.

You can also assign twoor all threeroles to the same computer. For example, you can configure a

computer to act as a client, and if you want it to allocate its local disk space to the cluster, you can set it

up to be a chunk server, as well.In this paper, unless otherwise noted, the term server refers to a physical server. In each of our tests,

three servers in each cluster are set up to host all three roles (MDS, CS, and client), and the remaining

servers in the cluster have two roles (CS and client).

Key Features

NO SPECIAL HARDWARE REQUIREMENTS

Parallels Cloud Storage has no special hardware requirements: it can be built using commodity hardware(such as SATA or SAS drives and 1Gbit+ Ethernet) and will run on your current hardware stack.

Although additional hardware resources, such as SSD drives or 10Gbit Ethernet, can significantly boost

performance, they are not required to get the major benefits of Parallels Cloud Storage: a cost-effective

distributed storage solution with built-in replication and high availability.


5/14


5

HIGH RELIABILITY

Parallels Cloud Storage is designed to ensure that data availability will not be affected by the

unavailability of up to a predefined number of logical or physical components. The following features

ensure high reliability:

Replicated metadata.Several MDSstypically three or fiverun simultaneously, enabling them tosurvive the unavailability of one or two MDSs, respectively. As long as more than half of the MDSs are

responsive, clients are guaranteed the ability to access metadata.

Replicated chunks.Every piece of data is saved in multiple copies across different CSs to ensurethat the data is available even if some CSs are in a failed state (e.g., disconnected from the network or

being rebooted). Parallels Cloud Storage also automatically maintains the required number of replicas

of each chunk. If some of the replicas become unavailable, Parallels Cloud Storage will start making

more replicas and copying them to new locations.

High availability and consistency.The existence of multiple CSs and MDSs and the replication ofdata chunks across the cluster mean that even if some CSs or MDSs are unavailable, clients can still

access and modify data. Parallels Cloud Storage also tracks any chunk modifications to guarantee

that all pieces of data are in the same stateso if a CS is down for a period of time and some of itschunks end up out of date as a result, Parallels Cloud Storage will recognize the problem and update

the chunks to latest version once the CS comes back online.

Data checksumming and scrubbing(works with SSD cache for a chunk server journal featureonly). Although Parallels Cloud Storage itself is highly available and consistent, there is always a

chance that local storage could become corrupted. To guard against such silent data corruption

which could occur, for example, due to hardware faultsParallels Cloud Storage periodically performs

data checks. When data is first written or modified, the storage solution calculates and remembers

checksums for each piece of data. Then, in background mode, Parallels Cloud Storage periodically

inspects data from local storage and compares saved checksums with newly calculated ones to

ensure that there is no corruption. If it detects any corrupted data chunks, it replaces them with

replicas of known-good copies.

EXTENSIVE SCALABILITY AND FLEXIBILITY

Another advantage of Parallels Cloud Storage is its extensive scalability and flexibility. Key capabilities in

this regard include:

Growth on demand.You can easily extend Parallels Cloud Storage by simply adding new CSsupto a total of 1PB of storage space.

Load balancing.Parallels Cloud Storage automatically balances workloads among all available CSs.In storing new chunks, Parallels Cloud Storage takes into account both the amount of free disk space

and the distribution of the I/O load, in order to ensure high performance. Additionally, when you add

new CSs to a cluster, Parallels Cloud Storage automatically balances existing data across all CSs.

Storage tiers.Since fast storage costs more than slower storage, Parallels Cloud Storage lets youdefine allocation policies for different types of local drives so as to deliver the best price/ performanceratio. For instance, you may choose to store frequently accessed data on fast, but relatively expensive,

SSD drives, while placing rarely needed backup data on inexpensive SATA drives.


6/14


6

TOP PERFORMANCE

Its hard to overestimate the importance of storage performance, which is usually the bottleneck for the

whole systems performance. Parallels Cloud Storage offers several options for enhancing performance,

including:

Thin replication. Parallels Cloud Storage supports thin replication, in which only the changed partsof chunks are updated on recovery, rather than the whole chunks. This approach significantly boosts

overall performance because it involves fewer reads, writes, and data transfers.

SSD cache for a chunk server journal.You can attach an SSD drive to a CS in the cluster andconfigure the drive to store a write CS journala step that can boost the performance of random write

operations in the cluster by a factor of two or more. The journal can also maintain chunk checksums

to improve storage reliability.

SSD read cache on the client.You can attach an SSD drive to a client and configure the driveto store a local cache of frequently accessed dataa step that can increase overall cluster read

operations performance by a factor of 10 or more.

Replication in Parallels Cloud Storage

OVERVIEW

To ensure high availability, Parallels Cloud Storage stores a number of replicas of each chunk across

multiple physical servers. If it detects the unavailability of one or more chunk servers for a predefined

amount of time, it will automatically start the replication process to ensure the availability of a sufficient

number of copies of the data. Figures 2-4 illustrate the replication process in a cluster that consists of

three CSs. With the replication level set to 2, each chunk should have two replicas. If CS #1 becomes

unavailable for some reason, chunks 1 and 2 end up with only one replica. In such a case, Parallels

Cloud Storage immediately detects that these chunks have fewer replicas than they should and creates

however many additional replicas are needed to bring the number up to the specified replication level.

Figure 2. This healthy cluster has three CSs and two replicas

of each of three data chunks.


7/14


7

Figure 3. This cluster, with one unavailable CS, has become degraded:

Chunks 1 and 2 have only one replica.

Figure 4. The cluster is healthy again after replication, with two replicas of every chunk.

Parallels Cloud Storage replication is similar to the RAID rebuild process, but it has two key differences:

Replication is much faster with Parallels Cloud Storage than with a typical online RAID 1/5/10 rebuild,

because Parallels Cloud Storage is able to replicate in parallel, across all servers in the cluster. Incontrast, RAID is only able to copy data from a single hard drive to another single hard drive.

The more CSs you have in a cluster, the less time it takes to replicate each chunk.

Replication performance is important because it minimizes the time when the system operates with a

lower than normal redundancy level. To ensure the overall reliability of the storage solution, replication

time must be much shorter than the expected time between CS faultsbecause the briefer the period

of reduced redundancy, the less likely you are to lose any data. In theory, the mean time to data loss

(MTTDL) is in reverse proportion to T2, where T is the time to recover.


8/14


8

Factors affecting replication performance include:

Number of available CSs, because replication is a process that runs in parallelso the more available

replication sources and destinations you have, the faster it goes.

Performance of the local disk system used for storing chunks.

Network performance, because as each chunk is read, its transferred by the network to write an

additional copy.

Chunk distribution, because some CSs may have much more data to be replicated than others and

therefore may become overloaded during replication.

I/O activity in the cluster under replication.

RESULTS OF REPLICATION PERFORMANCE TESTS

Figure 5 shows the results of the replication performance measurements of Parallels Cloud Storage,

running on commodity hardware. Each server has software RAID 0 arrays, using two 7200-RPM SATA

drives, with servers interconnected in a 1Gbit network. For a more complete description of the testing

methodology, please refer to the appendix.

Figure 5.The more servers a cluster has, the less time it takes to replicate data.

These performance measurements show that with Parallels Cloud Storage, replication performance

scales almost linearly as the number of servers in the cluster grows. For instance, with a cluster

consisting of 21 servers, if one server is dead (due to kernel panic, network disconnection, or any

other factor), it will take only about 20 minutes to replicate 1TB of data. Such linear scalability is not

possible with RAID. In short, these tests demonstrate that even with Parallels Cloud Storage running

on commodity hardware, its replication time is a fraction of what RAID requires, resulting in much

longer MTTDL, and therefore much higher reliability.


9/14


9

Note also that, although Parallels Cloud Storage was designed to operate efficiently when running

even on modest hardware, its able to make full use of all the resources of more powerful systems.

Figure 6 compares the performance of clusters with 7 and 14 servers running on software RAID 0

(with two 7200-RPM SATA drives) with that of servers running on hardware RAID 0 arrays (with four

7200-RPM SATA drives). In case of the hardware-based system, servers were interconnected via a

10Gbit network because the 1Gbit network used in the previous tests is limited to 110MB/sec of datatransfera limitation that can become a bottleneck when used with fast disk storage.

Figure 6. Using a more powerful hardware configuration enables a terabyte of datato be replicated much more rapidly.

I/O Performance Tests

OVERVIEW

This section presents the results of tests comparing the I/O performance of software RAID 1 (mirroring)

with that of Parallels Cloud Storage, deployed on servers configured with RAID 0 and two replicas of

each data chunk. (We chose software RAID 1 for the RAID comparison because it has the same level of

redundancy as a Parallels Cloud Storage configuration with two replicas.)According to our statistics, the vast majority of I/O operations on virtualization platforms are random.

Based on these statistics, we used mostly random I/O for our tests, although we did include one

sequential read, because its important for some scenarios (creating backups, for instance). Our tests

used the following I/O load patterns:

1. A sequential read of a 16M block(measured in MB/sec). This I/O workload is equivalent to asequential read of a large file.

2. A random read of a 4K block(measured in IOPS). This pattern represents any workload with arandom read, such as one involving small files, script codes, file system journals, or file searches.


10/14


10

3. A random write of a 4K block(measured in IOPS). This pattern represents any workload with arandom write, such one involving file system journals or small files.

4. A random write of a 4K block + fdatasync (measured in fdatasync operations per second). Thisworkload is the most important, because it simulates work that a relational database must do to

save simple database changes during a transaction.

We performed two series of tests: one series measuring horizontal scalability (that is, the effects of

adding more serversup to a total of 21) and one measuring vertical scalability (that is, the effects of

adding more workload threads to a single serverup to a total of 16 threads).

For a more complete description of the testing methodology, please refer to the appendix.

TEST RESULTS FOR HORIZONTAL SCALABILITY

Horizontal scalability is very important for distributed systems, as it allows you to add new nodes to a

cluster to increase the systems overall performance. For these tests, we compared RAID 1 performance

both with that of Parallels Cloud Storage running on commodity hardware and Parallels Cloud Storage

with SSD journaling added. In each case, we ran 16 threads of I/O workload on each server, starting

with a cluster consisting of a single server, then moving up to 10 servers, and then to 21. Figures 7 and

8 show the performance results. In all cases, Parallels Cloud Storage demonstrates I/O performance thatis at least comparable with that of local RAID storageand when SSD caching is added, Parallels Cloud

Storage dramatically exceeds RAIDs I/O performance.

Figure 7. As the number of servers increases, the read performance

of Parallels Cloud Storage increases accordingly.


11/14


11

Figure 8. As the number of servers increases, the write performance

of Parallels Cloud Storage increases accordingly.

TEST RESULTS FOR VERTICAL SCALABILITY

Vertical scalability is the ability to maintain the desired I/O performance as the number of workload

threads on a server increases. For these tests, we ran 1, 8, and 16 threads of I/O workload on a single

server in the cluster. Figures 9 and 10 show the performance results as the number of workload threads

increased. As the figures show, because Parallels Cloud Storage is able to use idle HDDs on cluster

servers, it thereby exceeds performance of local RAID storage which limited by local HDD speed.

However, the bandwidth of 1 Gbit Ethernet may limit Parallels Cloud Storage performance on sequentialreads. When SSD caching is added, the performance of Parallels Cloud Storage is several times faster.

Figure 9. Read performance of Parallels Cloud Storage scales well

as the number of workload threads increases.


12/14


12

Figure 10. Write performance of Parallels Cloud Storage scales well

as the number of workload threads increases.

Conclusion

Parallels Cloud Storage demonstrates little to no overhead compared to local RAID storage, while at

the same time providing flexibility and scalability far beyond whats available from a local RAID storage

system. Its ability to replicate data chunks in parallel, across all servers in the cluster, makes it much

faster than RAIDand because the faster replication results in a much longer MTTDL, its reliability is also

greater. In addition, when running on commodity hardware, Parallels Cloud Storage offers horizontal and

vertical scalability comparable to that of RAIDand when you add SSD for caching, its performance is

dramatically better than what RAID can deliver.

In short, Parallels Cloud Storage gives you a storage solution thats as fast or faster than RAID, while also

offering greater reliability, flexibility, and scalability.


13/14


13

Appendix: Test Lab Configurations and Methodologies

REPLICATION TESTING

Two hardware sets were used in replication performance testing: a modest hardware set (software RAID

0, 1Gbit network) and a more powerful hardware set (hardware RAID 0, 10Gbit network).

Table 1 shows the test lab configuration for replication testing.

The methodology we used for replication testing was as follows:

1. Set up Parallels Cloud Storage cluster with a client mount on each server.

2. Created 100GB of data on each client: dd if=/dev/urandom of=/pstorage/client_$i_test_lebs=1M count=102400, where/pstorageis the pstorage client mount point.

3. Chose a random server and turned off its network connection (ifdown ethX, where ethXexternalnetwork connection used by pstorage).

4. Checked pstorage event log for messages about replication start (Replication started. N chunksqueued to replication) and finish (Replication finished).

5. Replication speed is an amount of replicated data divided by time it took to recover accordingto event log.

6. Replication progress can be monitored via pstorage toptool (press v to get access to replicationperformance stat and ETA). (For more information, see the Monitoring General Cluster Parameterschapter in the Parallels Cloud Storage Administrators Guide.)

Processor 1xQuadCore Intel Core i5 2400 2xOctoCore Intel Xeon E5 4650, hyper-threading

Memory16 GB (4x4xGB), DDR3,

1333MHz32GB (8x4GB), DDR3, 1333MHz

Disk

2x2TB 7200 RPT SATA drives

connected visa SATA 3.0

interface.

Software RAID 0 with 512KB

chunk size

4x1TB 7200 RPM SATA drives connected via

SATA 3.0 interface. Hardware Intel integrated

RAID Module RMS25PB080, RAID 0 with 512KB

chunk size

Network CardIntel 82574L 1Gbit Network

ConnectionIntel 82599 10Gbit Network Connection

Network Switch Cisco Catalyst 2960, 1Gbit Cisco Nexus 5010, 10Gbit

Software Stack

Operation System Included in Parallels Cloud Storage 6.0 upd1 (build 6.0.0-789)

Parallels Cloud StorageBuild 5.0.1-340; each server has FUSE client and chunk server;

3 metadata servers in the cluster

Kernel vzkernel-2.6.32-042stab064.6.x86_64

Hardware Set 1 Hardware Set 2


14/14


PERFORMANCE TESTING

Table 2 shows the test lab configuration for performance testing.

Table 2. Performance Testing Configuration

The methodology used for performance testing was as follows:

1. We used a custom-developed I/O load tool to simulate various workloads (random and sequentialreads and writes in multiple threads). The volume of test data was 16GB for each server.

2. The set up for Parallels Cloud Storage clusters consisted of 21 servers, with a client mount on eachserver.

3. Tested performance in several configurations, with 1, 4, and 16 workload threads running on 1, 10,and 21 servers.

4. For a fair comparison between local RAID and Parallels Cloud Storage, giving each the same level ofredundancy and a similar logical structure, we used servers with RAID 1 (mirroring) for the local RAIDtests, vs. Parallels Cloud Storage configured to store 2 data replicas.

5. For the PCS+SSD cases, SSD drives were added for both CS journaling and client read caching.

2013 Parallels. All rights reserved. Parallels is a registered trademark of Parallels Software International in the USA and other

countries. All others are the trademarks or registered trademarks of their respective owners.

Processor 1xQuadCore Intel Core i5 2400

Memory 16 GB (4x4xGB), DDR3, 1333MHz

Disk2x2TB 7200 RPT SATA drives connected visa SATA 3.0 interface.

Software RAID 0 with 512KB chunk size

Network Card Intel 82574L 1Gbit Network Connection

Network Switch Cisco Catalyst 2960, 1Gbit

Software Stack

Operation System Included in Parallels Cloud Storage 6.0 upd1 (build 6.0.0-789)

Parallels Cloud StorageBuild 5.0.1-340; each server has FUSE client and chunk server;

3 metadata servers in the cluster

Kernel vzkernel-2.6.32-042stab064.6.x86_64

Hardware Set 1

pcloudstorage performance results wp en ltr 02192013 web

Documents