pernixdata fvp software and microsoft sql server · pernixdata fvp with pernixdata fvp 500k 400k...
TRANSCRIPT
1
PernixData FVP Software and Microsoft SQL Server
REFERENCE ARCHITECTURE
2
Table of Contents
Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Solution Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Software Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
PernixData FVPTM Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Microsoft SQL Server and VMware vSphere . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Hardware Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
Server and Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
Storage Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
Storage Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
HammerDB TPC-C (OLTP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
HammerDB TPC-H (DSS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
System Details and Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
FVP Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Microsoft SQL Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8
Server and Network Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Storage Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
vSphere Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Infrastructure and Management Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Appendix A: HammerDB TPC-C Test Methodology and Detailed Results . . . . . . . . . . . . . . . 13
Appendix B: HammerDB TPC-H Test Methodology and Detailed Results . . . . . . . . . . . . . . . 19
Appendix C: Bill of Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26
3
Executive Summary
Like many other applications, there are numerous benefits to virtualizing Microsoft SQL Server . Often, the
simplified operations and lowered costs enabled through physical server consolidation can be reason alone to
virtualize . Add the benefits of hardware independence and flexibility and the ability to easily scale-up (or down) by
allocating more (or less) resources and the decision to virtualize SQL Server becomes an easy one .
However, misconfigured or mismanaged virtualized environments, more often than not, lead to poor database
performance . When many virtual machines (VMs) access the same shared storage, contention for resources (disk
and/or CPU) can arise in the storage devices and fabric . This creates storage I/O bottlenecks, which result in poor
database performance (regardless of the database platform) .
This paper includes the results of a series of virtualized SQL Server database performance tests . These performance
tests were completed using the HammerDB performance benchmark . For some of the tests, FVP software was
deployed with the VMware vSphere stack running the SQL Server virtual machines, with no changes to operating
system or SQL Server configurations . In the design detailed here, FVP software uses server-deployed PCIe solid-state
disk (SSD) to increase Online Transaction Processing (OLTP) database transactions-per-minute (TPM) and Decision
Support System (DSS) database query performance . To provide optimal performance for all tests, the storage
array providing capacity was configured according to typical virtualized database server best practices .
Test results revealed that PernixData FVP™ software eliminated storage I/O bottlenecks and significantly decreased
I/O latency, allowing full CPU utilization and resulting in more than 2x OLTP TPM and 10x faster DSS query performance .
Results show that with FVP software installed, write bursts associated with OLTP database transactions are absorbed
into low latency server-side flash or RAM . This leads to substantially higher database transaction commit rates .
FVP also allows SQL Server to leverage the throughput delivered by server-side resources, leading to a greater
degree of parallelism, increased processor utilization, and faster DSS database query performance .
WITHOUTPERNIXDATA FVP
WITHPERNIXDATA FVP
500k
400k
300k
200k
100k
0
OLTP Transactions(Higher is better)
More than 2x Increase
WITHOUTPERNIXDATA FVP
WITHPERNIXDATA FVP
50k
40k
30k
20k
10k
0
DSS Query Total Completion Time (seconds)(Shorter is better)
10x Faster
Figure 1. OLTP Transactions Per Minute Figure 2. DSS Query Total Completion Time
4
Solution Overview
The Microsoft SQL Server solution with FVP software combines one of the most widely deployed database platforms
in the world with the industry’s premier platform for storage intelligence . By eliminating storage bottlenecks and
minimizing “noisy neighbor” effects, the FVP-enabled solution allows for a high performance SQL Server deployment
on a single shared VMware vSphere hosting platform . In addition, as requirements change, SQL Server performance
and capacity can be discretely scaled using a choice of many server and storage systems .
Software Components
PernixData FVP Software
PernixData FVP software puts storage intelligence into server flash and RAM to accelerate the performance of
virtualized applications . By putting storage intelligence into high speed server media, PernixData FVP software
delivers the following benefits:
• Faster SQL response times. Latency of database queries is reduced by 10x on average when data is
served from the local host, resulting in more transactions in any time period .
• Performance that scales with demand. FVP software ensures databases get the I/O resources they need,
even when demand spikes . This means your SQL Server performance always remains consistent—even
during periods of peak usage .
• Cost-effective I/O acceleration. Instead of buying expensive storage capacity to get better SQL performance,
you can add performance when and where you need it most . An FVP solution can save up to 95% when
compared to alternative storage upgrades .
Microsoft SQL Server and VMware vSphere
Microsoft SQL Server database installations can range from large data warehouses, to small, highly specialized
application databases . Whether unlocking insights with pervasive data discovery or recording online transactions,
each application has its own set of requirements for the database layer, resulting in multiple versions, patch levels,
and maintenance processes . To meet these requirements, administrators often deploy SQL Server installations
dedicated to each application . Unfortunately, application isolation can create significant waste of server resources .
This waste becomes even more costly when requirements call for high availability .
Fortunately, virtualizing SQL Server with VMware vSphere can allow the best of both worlds by simultaneously
optimizing compute resources through server consolidation and maintaining application flexibility through
role isolation . For smaller, specialized databases, vSphere offers high consolidation ratios and advanced
resource scheduling features that allow SQL Server VMs to easily scale-up (or down) based on demand . This
gives application owners the flexibility and performance they need while simplifying and lowering costs for the
enterprise . vSphere has also shown the ability to run the most challenging SQL Server workloads with near native
performance when given the proper resources .
5
Hardware Components
Server and Network
Cisco Unified Computing System is the first converged data center platform that combines industry-standard,
x86-architecture servers with networking and storage access into a single converged system . The system is
entirely programmable using unified, model-based management to simplify and speed deployment of enterprise-class
applications and services running in bare-metal, virtualized, and cloud computing environments .
The computing platform with Cisco UCS used for these tests includes:
• Cisco UCS C220 M3 Servers
• Cisco UCS 6200 Series Fabric Interconnects
• Cisco UCS 2000 Series Fabric Extenders
Storage Capacity
EMC VNXe Series is an affordable unified storage platform with solution-focused software that’s easy to manage,
provision, and protect . The VNXe series allows you to easily deploy private clouds with deep VMware integration .
It’s built for trademark ease-of-use across the entire storage lifecycle - from setup to management to support . It
not only provides a single platform for file and block data services with Ethernet and Fibre Channel connectivity,
but data efficiency services reduce your capacity requirements up to 50 percent .
The storage capacity configuration used for these tests includes:
• EMC VNXe3200 Storage Array
– 8 x 200 GB MLC SSDs
– 16 x 600 GB 10K RPM SAS Disks
– 12 x 4 TB 7 .2K RPM NL-SAS Disks
Storage Performance
The P420m SSD is Micron’s mainstream PCIe workhorse . The P420m is like having a turbo button for your workload .
It was designed to use Micron’s RAIN and DataSafe technology to deliver constant, steady performance ideal for
caching, cloud, and data center applications as well as blade enterprise servers . Micron offers HHHL card and
hot-swappable 2 .5-inch drive form factors with up to 1 .4 TB of capacity .
The storage performance configuration used for these tests includes:
• 2 x 700 GB Micron P420m (one per host)
6
Test Results
Test results are summarized in the following section . For further details, see Appendices A and B .
HammerDB TPC-C (OLTP)
HammerDB TPC-C testing showed that FVP software significantly decreased storage latency, allowing full CPU
utilization and resulting in more than 2x OLTP average transactions-per-minute (TPM) . Two identical standard
HammerDB TPC-C benchmark tests were run to simulate a typical OLTP database workload and compare performance
with and without FVP software . Test 1 ran with FVP software enabled using PCIe SSD as an acceleration resource .
Test 2 was run to measure performance without FVP software . In Figure 3 below, you can see uniform ~400k
average TPM in Test 1 on the left half and inconsistent ~200k average TPM in Test 2 on the right half .
Figure 3. HammerDB Transaction Counter - Test 1 (with FVP) on left half of chart; Test 2 (without FVP) on right half of chart
HammerDB TPC-H (DSS)
HammerDB TPC-H testing showed that the increased throughput and reduced latency provided by server-side
flash and FVP software resulted in an average of 10x faster DSS query performance . Two identical standard
HammerDB TPC-H benchmark tests were run to simulate a typical DSS database workload and compare performance
with and without FVP software . Test 1 ran with FVP software enabled using PCIe SSD as an acceleration resource .
Test 2 was run to measure performance without FVP software . In Figure 4 below, you can compare average (5 users)
query completion time of all 22 DSS queries .
5k
4k
3k
2k
1k
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Average DSS Query Completion Times(Shorter is better)
WITHOUT PERNIXDATA FVP
WITH PERNIXDATA FVP
Figure 4. Average DSS query completion times (seconds, lower is better)
7
System Details and Configurations
FVP Software
In this configuration FVP software virtualizes server-side flash across two VMware ESXi hypervisor nodes in a
compute cluster and hooks these high-speed server-side resources into existing VM I/O paths to transparently
reduce the IOPS burden on an existing storage system . The FVP software installation integrates FVP-enabled
VMware ESXi hosts and the PernixData FVP Management Server within the VMware vSphere platform . The
installation includes three components:
• The PernixData FVP Management Server software, installed on a Windows server . The Management Server
coordinates operations between FVP and the ESXi host computers .
• The PernixData FVP UI Plugin, installed on servers where the vSphere Client is installed or accessible from
within the VMware Web Client . The plug-in is the primary tool used to manage FVP .
• The PernixData FVP Host Extension software on ESXi hosts . For a virtual machine to take advantage of
FVP’s acceleration capabilities that VM must be running on a host computer where the FVP Host Extension
module has been installed .
Using the UI Plugin, a single FVP Cluster was created and vSphere infrastructure VMs were all accelerated using
a single datastore policy . SQL Server virtual machines were accelerated using VM policies . Using this configuration
method allowed all infrastructure VMs to be accelerated using a single policy, but also allowed for fine tuning SQL
Server policies between different types of tests . All virtual machines were accelerated in Write Back mode using
1 .4 TB of clustered PCIe SSD based flash .
The following FVP configuration was used in this deployment:
• One Flash FVP Cluster – All Virtual Machines
• 700 GB SSD flash per host
• Write Back (Local + 1 Peer) – vSphere infrastructure VMs
• Write Back (Local Host) – SQL Server VMs
Note: Local host acceleration was chosen for SQL Server VMs because they are protected against host and flash
device failure at the “application layer” by a SQL Server AlwaysOn Availability Group (see Microsoft SQL Server section) .
FVP Cluster Assignments
The datastores were added to FVP Clusters as summarized in Table 1:
DATASTORE/VM NAME FVP CLUSTER NAME RESOURCE TYPE RESOURCE SIZE
vnxe_7k_lun00 (Datastore) Infrastructure PCIe PCIe SSD (NAND Flash) 1 .4 TB
mssql01 (VM) Infrastructure PCIe PCIe SSD (NAND Flash) 1 .4 TB
mssql02 (VM) Infrastructure PCIe PCIe SSD (NAND Flash) 1 .4 TB
Table 1. FVP Cluster Assignments
8
Microsoft SQL Server
In this design Microsoft SQL Server 2014 Enterprise Edition was configured in a highly available manner to
demonstrate the full capabilities of FVP software with SQL Server, but the same principles and performance
aspects apply to single-VM configurations .
With the release of SQL Server 2012, Microsoft introduced “AlwaysOn” Availability Group technology to SQL Server .
This high-availability and disaster-recovery solution provides an enterprise-level alternative to database mirroring and
other previous availability methods that relied on Microsoft Windows Server Failover Clustering (WSFC) and complex
Raw Disk Mapping (RDM) configuration when used with VMware vSphere . When configured using Availability Groups
(AGs), traditional WSFC and RDMs are no longer required and FVP software acceleration can be configured using the
lowest latency local host write-back acceleration .
An Availability Group supports a failover environment for a discrete set of user databases that failover together . An
availability group supports a set of read-write primary databases and up to eight sets of corresponding replica
databases in SQL Server 2014 . For this configuration two Windows Server 2012 R2 Standard VMs (one per physical
vSphere host) were used to host a single AG . This AG hosted two databases, one for OLTP testing and another for DSS
testing, and both were configured for synchronous replication .
This SQL Server installation included the following systems:
• Two Windows Server 2012 R2 Standard Servers with SQL Server 2014 Enterprise
• One Windows Server 2012 R2 Standard Server (Quorum File Share Witness)
Note: An additional server was required to host a quorum file share witness to avoid a “split-brain” condition where
both nodes believe they are the master . For this design, the Microsoft Active Directory server was used for this purpose .
SQL Server VM Configuration
An eight virtual CPU configuration was selected to fit within the physical host server’s 8-core per socket NUMA
boundary . Since datasets will be cached in memory by SQL Server during OLTP tests, only 16 GB of memory was
allocated to the virtual machine to represent the most typical configuration where database size is larger than
available memory .
SQL Server binaries, Temp DB, log files, and data files were all placed on separate virtual disks that mapped to
individual VMFS datastores, storage array LUNs, and storage pools for maximum performance .
The datastore to storage array LUN mapping is summarized in Table 2:
SQL FILE TYPE DATASTORE NAME STORAGE POOL RESOURCE TYPE
Binaries vnxe_7k_lun00 NL-SAS-Pool (RAID 6, 8+2) 7k RPM NL-SAS Disk
Temp DB vnxe_ssd_lun01,
vnxe_ssd_lun02
SSD-Pool (RAID 5, 4+1) MLC SSD
Logs vnxe_10k_lun03,
vnxe_10k_lun04
SAS-Pool (RAID 6, 10+2) 10k RPM SAS Disk
Data vnxe_7k_lun05,
vnxe_7k_lun06
NL-SAS-Pool (RAID 6, 8+2) 7k RPM NL-SAS Disk
Table 2. SQL VM datastore to storage array LUN mapping.
9
The Windows Server 2012 R2 VMs were additionally optimized according to VMware Best Practices for running
large-scale workloads with intensive I/O patterns . These optimizations included the following:
• Multiple Paravirtual SCSI disk controllers and default queue depth increase (VMware KB2053145)
• Eager Zeroed Thick Disk Provisioning (SQL Server on VMware Best Practices Guide)
• 64K File Allocation Unit Size (Microsoft TechNet Reference)
Virtual hardware of the SQL Server virtual machines was configured and optimized as defined in Table 3:
ATTRIBUTE SPECIFICATION
Server OS Microsoft Windows Server 2012 R2 (64-bit)
VMware virtual hardware Version 10
VMware Tools version 9 .4 .11 (up to date)
Virtual CPU 8 virtual sockets – 1 virtual core per socket
Virtual memory 16 GB
vNICs 2 (VMXNET3)
Virtual network adapter 1 VMXNet3 Adapter (Primary)
Virtual network adapter 2 VMXNet3 Adapter (Cluster Link)
Virtual SCSI controller 0 LSI Logic SAS
Virtual SCSI controller 1 Paravirtual (PVSCSI)
Virtual SCSI controller 2 Paravirtual (PVSCSI)
Virtual SCSI controller 3 Paravirtual (PVSCSI)
Virtual Disk – OS - VMDK 40 GB (Thin, vSCSI 0:0)
Virtual Disk – Binaries - VMDK 40 GB (Thin, vSCSI 0:1)
Virtual Disk – SQL Temp DB - VMDK 200 GB (Eager Zeroed Thick, vSCSI 1:0)
Virtual Disk – SQL Logs - VMDK 2 TB (Eager Zeroed Thick, vSCSI 2:0)
Virtual Disk – SQL Data - VMDK 2 TB (Eager Zeroed Thick, vSCSI 3:0)
Virtual CD/DVD Drive 1 Removed
Table 3. SQL Server VM Configuration
10
Server and Network Configuration
Tests were performed on Cisco UCS C220 M3 rack-mount servers designed for performance and density over a
wide range of business workloads from web serving to distributed database . One vSphere cluster containing both
host servers was deployed and contained all SQL Server and vSphere management virtual machines . To ensure
high availability and provide the best possible test performance, vSphere DRS affinity rules were used to keep the
two SQL Server VMs on separate vSphere hosts .
Each Cisco UCS C220 M3 blade was configured as follows:
• Two Intel Xeon E5-2640 2 .0 GHz processors (16 cores total)
• 256 GB of RAM
• One VIC 1225 Dual Port 10 Gb SFP+ CNA
• One Micron P420m 700 GB PCIe SSD
The Cisco UCS 6248UP Fabric Interconnects used in this architecture provide both network connectivity and
management capabilities for the system . They offer line-rate, low-latency, lossless 10 Gigabit Ethernet and Fibre
Channel over Ethernet (FCoE) functions, and were built to consolidate LAN and SAN traffic onto a single unified
fabric . Each 6248UP was additionally connected to a single Cisco Nexus 2232TM Fabric Extender to facilitate
direct connection of the VNXe32000’s 10GBase-T interfaces to the unified fabric .
Storage Configuration
Capacity
The EMC VNXe3200 unified hybrid storage system used in this architecture brings the power of EMC’s VNX to
the IT generalist . It retains the affordability, simplicity, and efficiency of previous generation VNXe systems and
adds support for MCx multicore optimization, FAST Cache SSD caching, FAST VP auto-tiering, and Fibre Channel
host connectivity . These enterprise-class features were previously reserved for higher-end storage systems .
A single, redundant, VNXe3200 array was used for testing . The system was connected to the storage fabric by
four 10GbE (one 2-port LACP bond per SP) connections utilizing iSCSI .
The following components made up the VNXe3200 deployment:
• One RAID 6 (8+2) Storage Pool (4 TB, 7k RPM NL-SAS disks)
• One RAID 6 (10+2) Storage Pool (600 GB, 10k RPM SAS disks)
• One RAID 5 (4+1) Storage Pool (200 GB, MLC SSD disks)
Both hosts were zoned and masked to allow access to all 7 VMFS (block) iSCSI datastores in Table 4:
DATASTORE NAME DATASTORE SIZE STORAGE POOL RESOURCE TYPE
vnxe_7k_lun00 2 TB NL-SAS-Pool (RAID 6, 8+2) 7k RPM NL-SAS Disk
vnxe_ssd_lun01 200 GB SSD-Pool (RAID 5, 4+1) MLC SSD
vnxe_ssd_lun02 200 GB SSD-Pool (RAID 5, 4+1) MLC SSD
vnxe_10k_lun03 2 TB SAS-Pool (RAID 6, 10+2) 10k RPM SAS Disk
vnxe_10k_lun04 2 TB SAS-Pool (RAID 6, 10+2) 10k RPM SAS Disk
vnxe_7k_lun05 2 TB NL-SAS-Pool (RAID 6, 8+2) 7k RPM NL-SAS Disk
vnxe_7k_lun06 2 TB NL-SAS-Pool (RAID 6, 8+2) 7k RPM NL-SAS Disk
Table 4. Storage Capacity—VMFS Datastores
11
Performance
PernixData FVP Software was used in this architecture to serve reads and writes locally on host servers and
eliminate storage I/O bottlenecks . In this design, FVP software uses Micron P420m 700 GB PCIe SSDs to accelerate
all virtual machines .
The following FVP configuration was used in this deployment:
• One Flash FVP Cluster – All Virtual Machines
• 700 GB SSD flash per host
• Write Back (Local + 1 Peer) – vSphere infrastructure VMs
• Write Back (Local Host) – SQL Server VMs
The smallest P420m PCIe SSDs available were chosen for this reference architecture because on-disk database
size for all HammerDB SQL databases was ~400 GB, leaving ample flash area for the frequently accessed data of
other VMs in the environment . This may also be appropriate for many standard virtual deployments, but acceleration
resource sizing will be ultimately determined by specific workload requirements .
vSphere Configuration
One virtual data center was deployed for all virtual machines . This is the standard, recommended configuration for
most production deployments .
Infrastructure and Management Configuration
A single VMware vCenter Server instance was deployed for all virtual machines .
For simplicity, all vCenter roles (inventory, SSO, vCenter) were consolidated onto a single VM . No resource
contention was observed on any parts of the management infrastructure during tests .
All server resources were sized according to the current best practices from VMware . They are listed in the
following table:
SERVER ROLE VCPU RAM (GB) STORAGE (GB) OS
Domain Controller 2 4 40 Windows Server 2012 64-bit R2
SQL Server (vCenter) 2 8 40 Windows Server 2012 64-bit R2
vCenter Server 2 8 40 Windows Server 2012 64-bit R2
Table 5. vSphere Server Resource Sizing
12
Conclusion
HammberDB tests clearly show that PernixData FVP software is a perfect complement for SQL Servers in VMware
vSphere environments . Results demonstrate how an FVP-enabled architecture absorbs OLTP write bursts into
low latency server-side flash or RAM for 2x transactions-per-minute and how decreased latency and increased
storage throughput lead to 10x faster DSS database query performance .
In addition, contrary to traditional storage capacity/performance coupled designs, FVP software also makes it possible
to independently scale SQL Server performance while independently sizing underlying storage capacity . PernixData
FVP software enables you to get all the operational benefits of virtualizing SQL Server with the best possible database
performance . FVP’s unique scale-out approach delivers SQL performance where and when its needed most, delivering
more database transactions in less time for a fraction of the cost of other I/O acceleration solutions .
When designing this PernixData FVP Software and Microsoft SQL Server reference architecture, specific hardware
components were selected to accurately represent an existing, well-designed, virtual environment . However, it’s
important to note that although these components were used to test and validate SQL Server performance with
FVP, they are not specifically required . Decoupling storage performance from capacity using FVP software allows
for the use of many different hardware platforms designed to run VMware vSphere . The results of this document
should be used alongside Microsoft SQL Server and VMware vSphere best practice documents to help choose
hardware based on actual environmental requirements .
13
Appendix A: HammerDB TPC-C Test Methodology and Detailed Results
As defined by HammerDB’s Introduction to Transactional (TPC-C) Testing for all Databases, “a transactional or
OLTP (online transaction processing) workload is a workload typically identified by a database receiving both
requests for data and multiple changes to this data from a number of users over time where these modifications
are called transactions .”
The Transaction Processing Performance Council (TPC) is the industry body most widely recognized for defining
benchmarks in the database industry and TPC-C is the benchmark published by the TPC for Online Transaction
Processing . HammerDB includes an implementation of the TPC-C benchmark specification that can be run
against Microsoft SQL Server and other supported database environments . Implementing the TPC-C specification
ensures accurate, repeatable, and consistent results . HammerDB is designed to measure relative as opposed
to absolute database performance between systems . This means that the test results published as part of this
design shouldn’t be compared to other published tests, but do serve as a good relative measure for comparing
performance with and without FVP software .
Note: HammerDB implements a workload based on the TPC-C specification . However it does not implement a full
specification TPC-C benchmark and the transaction results from HammerDB cannot be compared with the official
published TPC-C benchmarks in any manner .
HammerDB workloads produce two statistics to compare configurations called transactions-per-minute (TPM) and
new-orders-per-minute (NOPM), respectively . TPM is the specific database transactional measurement typically
defined as the number of user commits plus the number of user rollbacks . TPM values cannot be compared
between different database types, but for the purposes the tests in this design, work well since the same SQL
Server database was used for all tests .
Database Schema
The TPC-C workload represents a system used to fulfill orders to supply products . The workload is defined by a
mix of 5 transactions selected at random according to the balance of the percentage value shown below:
• 45% - New-order: receive a new order from a customer
• 43% - Payment: update the customers balance to record a payment
• 4% - Delivery: deliver orders asynchronously
• 4% - Order-status: retrieve the status of customer’s most recent order
• 4% - Stock-level: return the status of the warehouse’s inventory
The database schema is depicted in Figure 5 .
14
Figure 5. HammerDB TPC-C Schema
Image Source: Introduction to Transactional (TPC-C) Testing for all Databases
Configuration
There are several HammerDB configuration options that can be tuned to provide the highest level of performance
for any given hardware and software configuration . Generally speaking, the highest level of TPM performance is
usually seen near maximum database server CPU utilization . Removing all other potential bottlenecks to maximize
CPU utilization was a primary testing goal . As an example, to eliminate possible contention, 500 warehouses
were generated as part of schema creation .
All schema creation options are listed in the following table:
OPTION VALUE
SQL Server mssqlclus01 .lab .pernixdata .com
SQL Server Port 1433
SQL Server ODBC Driver SQL Server Native Client 11 .0
Authentication Windows Authentication
SQL Server Database tpcc
Schema Updated
Number of Warehouses 500
Virtual Users to Build Schema 32
Table 6. HammerDB TPC-C Schema Creation Options
The Timed Test Driver Script was configured as part of the driver options to have HammerDB time the tests,
measure the results and report on an average transaction rate for a period of time . A test duration of five minutes
was selected with a rampup time of two minutes . Since these tests were not performed to compare to official
TPC-C results, keying and thinking time was disabled . When configured this way, each virtual user executes tens
of thousands of transactions a minute .
15
All driver options are listed in the following table:
OPTION VALUE
SQL Server mssqlclus01 .lab .pernixdata .com
SQL Server Port 1433
SQL Server ODBC Driver SQL Server Native Client 11 .0
Authentication Windows Authentication
SQL Server Database tpcc
TPC-C Driver Script Timed Test Driver Script
Total Transactions per User 1000000
Exit on SQL Server Error False
Keying and Thinking Time False
Checkpoint when complete False
Minutes of Rampup Time 2
Minutes for Test Duration 5
Table 7. HammerDB TPC-C Driver Options
With keying and thinking time disabled, a limited number of virtual users and warehouses are required rather
than hundreds or thousands of virtual users and warehouses . With each user executing such a large number
of transactions, additional prerequisite testing was performed to determine the optimal number of virtual users
that produced the highest TPM without exhibiting contention for data . This testing revealed the optimal number
of virtual users to be 30 . When running the Timed Test Driver Script, one additional user is required to perform
timing and collect results . All other options were configured with their default settings .
All virtual user options are listed in the following table:
OPTION VALUE
Virtual Users 31
User Delay(ms) 500
Repeat Delay(ms) 500
Iterations 1
Show Output True
Log Output to Temp True
Use Unique Log Name True
No Log Buffer False
Table 8. HammerDB TPC-C Virtual User Options
Note: As per HammerDB best practice, a separate virtual machine with HammerDB software and the Microsoft
SQL Server client installed was used to generate the workload for all tests .
16
Detailed HammerDB Results
The results of the 5 minute HammerDB TPC-C tests are summarized in the table below:
BENCHMARK TEST 1 (FVP) TEST 2 (WITHOUT FVP)
Transactions Per Minute 400,593 182,464
New Orders Per Minute 87,036 39,465
Table 9. HammerDB TPC-C 5 minute TPM results
In the following figures taken from the FVP Management Server user Interface, Read (blue) and Write (orange)
lines depict the actual storage performance of one of the SQL Server virtual machines during tests .
Virtual Machine Storage Latency
In Figure 10 below, SQL Server virtual machine storage latency in Test 1 (with FVP) is shown on the left half and
Test 2 (without FVP) on the right half . The significantly decreased latency in Test 1 allows for greater HammerDB
TPM results .
Figure 10. Virtual machine storage latency comparison between tests
In Figure 11 below, SQL Server virtual machine storage latency (orange) is barely distinguishable along the bottom
of the left half of the chart in Test 1 (with FVP) . Datastore write latency for the same period (gold) can also be seen .
This decrease in latency allows for greater HammerDB TPM results .
Figure 11. Virtual machine to datastore latency comparison
17
In Figure 12 below, SQL Server virtual machine storage latency from an additional test with FVP is shown . Note
the sub-millisecond scale on the left side of the chart .
Figure 12. Virtual machine storage latency with FVP
In Figure 13 below, SQL Server virtual machine storage latency from an additional test without FVP is shown .
Figure 13. Virtual machine storage latency without FVP
Virtual Machine Storage IOPS
In Figure 14 below, SQL Server virtual machine storage IOPS in Test 1 (with FVP) is shown on the left half and Test
2 (without FVP) on the right half . The increased lOPS in Test 1 allows for greater HammerDB TPM results .
Figure 14. Virtual machine storage IOPS comparison between tests
18
In Figure 15 below, SQL Server virtual machine storage IOPS from an additional test with FVP is shown .
Figure 15. Virtual machine storage IOPS with FVP
In Figure 16 below, SQL Server virtual machine storage IOPS from an additional test without FVP is shown .
Figure 16. Virtual machine storage IOPS without FVP
Virtual Machine Write Back Destaging
Figure 17 below illustrates the amount of data (and estimated time to commit) that FVP is able to absorb into server-side
flash during the write bursts of HammerDB TPC-C testing . This is data that the storage array is incapable of
ingesting and becomes a bottleneck . Eliminating this bottleneck allows for greater HammerDB TPM results .
Figure 17. Virtual machine write back destaging
19
Appendix B: HammerDB TPC-H Test Methodology and Detailed Results
As explained by HammerDB’s Introduction to Decision Support, Data Warehousing, Business Intelligence, and
Analytical Load Testing for all Databases, “the basis of Decision Support Systems is the ability to process complex
ad-hoc queries on large volumes of data .”
As previously mentioned, the Transaction Processing Performance Council (TPC) is the industry body most widely
recognized for defining benchmarks in the database industry . TPC-H is the benchmark published by the TPC for
decision support systems (DSS) databases . HammerDB includes an implementation of the specification of the
benchmark that can be run in a Microsoft SQL Server environment . Whereas TPC-C simulates an online ordering
system, TPC-H represents the typical workload of business users inquiring about the performance of their business .
In particular the focus is upon highly complex queries that require the processing of large volumes of data .
Note: HammerDB implements a workload based on the TPC-H specification . However it does not implement a full
specification TPC-H benchmark and the transaction results from HammerDB cannot be compared with the official
published TPC-H benchmarks in any manner .
A typical DSS performance profile is represented by the time it takes the system to process a query set from
Query 1 to Query 22 (run in a pre-determined random order) . As a simple benchmark you can compare the timings
required for a single virtual user to complete the 22 queries across different tests . Since these tests were not
performed for comparison to official TPC-H results, specific “Power” and “Throughput” tests were not performed
to compile a Composite Query-per-Hour Performance Metric (QphH) .
Database Schema
Similarly to TPC-C, the schema size for TPC-H is not fixed and is dependent upon a Scale Factor . The larger the
schema, the more powerful the SQL Server system must be to process the increased data volume for queries .
The database schema is depicted in Figure 18 .
Figure 18. HammerDB TPC-H schema
Image source: Introduction to Decision Support, Data Warehousing, Business Intelligence, and Analytical Load Testing for all Databases
20
Configuration
As with TPC-C, there are several HammerDB configuration options that can be tuned to provide the highest
level of performance for any given hardware and software configuration . For DSS, processing large amounts
of data within a single process or thread on a traditional row oriented database is time consuming . To increase
performance, SQL Server employs Parallel Query Processing to optimize query execution and index operations
for computers that have more than one CPU . With a chosen scale factor of 100 (~220 GB on disk), prerequisite
testing revealed the optimal Maximum Degree of Parallelism (MAXDOP) to be 8 .
All schema creation options are listed in the following table:
OPTION VALUE
SQL Server mssqlclus01 .lab .pernixdata .com
SQL Server Port 1433
SQL Server ODBC Driver SQL Server Native Client 11 .0
Authentication Windows Authentication
SQL Server Database tpch
MAXDOP 8
Scale Factor 100
Virtual Users to Build Schema 32
Table 10. HammerDB TPC-H Schema Creation Options
Within the driver options a MAXDOP of 8 set the value to be used for query hints to set the Maximum Degree of
Parallelism that a particular query will use . The Total number of query sets was set to one . This is the number of
times after logging on that the virtual user completes an entire sequence of queries . The refresh function can be
used to more accurately simulate official TPC-H testing, but for these tests was unused .
Note: The queries do not run in sequence order from Query 1 to Query 22 and instead run according to a
predefined random order depending on the number of virtual users . Since the same number of virtual users (5)
was used for all tests, queries ran in the same order for both Test 1 and Test 2 .
All driver options are listed in the following table:
OPTION VALUE
SQL Server mssqlclus01 .lab .pernixdata .com
SQL Server Port 1433
SQL Server ODBC Driver SQL Server Native Client 11 .0
Authentication Windows Authentication
SQL Server Database tpch
MAXDOP 8
Total Query Sets per User 1
Exit on SQL Server Error False
Verbose Output False
Refresh Function False
Table 11. HammerDB TPC-H Driver Options
21
To generate significant levels of storage i/O for performance comparison, the query sets were run with 5 simultaneous
user streams . This is similar to the number of streams required by the official Scale Factor 100 TPC-H Throughput
test . All other options were configured with their default settings .
All virtual user options are listed in the following table:
OPTION VALUE
Virtual Users 5
User Delay(ms) 500
Repeat Delay(ms) 500
Iterations 1
Show Output True
Log Output to Temp True
Use Unique Log Name True
No Log Buffer False
Table 12. HammerDB TPC-H Virtual User Options
Detailed HammerDB Results
The average total query completion times (5 users, 22 total queries each) recorded during HammerDB TPC-H
tests are summarized in the table below:
BENCHMARK TEST 1 (FVP) TEST 2 (WITHOUT FVP)
Avg Total Query Completion Time 2h:14m;14s 11h:14m:38s
Table 13. HammerDB TPC-H avg total query completion times
22
The average query completion times recorded during HammerDB TPC-H tests and the factor decrease seen in
Test 1 (with FVP) are summarized in the table below:
QUERY TEST 1 (FVP) TEST 2 (WITHOUT FVP) FACTOR DECREASE
1 405 2009 5x
2 20 .8 593 .8 28 .5x
3 112 1831 .8 16 .4x
4 330 .8 2654 .8 8x
5 335 .8 2693 .6 8x
6 143 .6 1644 .8 11 .5x
7 132 .4 1785 .6 13 .5x
8 246 .2 2557 .6 10 .4x
9 446 2390 .2 5 .4x
10 379 2631 .4 6 .9x
11 113 .8 806 .6 7 .1x
12 732 .6 1544 .8 2 .1x
13 258 .2 644 2 .5x
14 147 1776 .2 12 .1x
15 71 .4 1651 .6 23 .1x
16 125 .2 833 .8 6 .7x
17 50 .6 726 .2 14 .4x
18 665 .6 2792 .8 4 .2x
19 654 .8 2373 3 .6x
20 103 .8 1268 .6 12 .2x
21 2526 .4 4523 .2 1 .8x
22 51 .4 732 .6 14 .3x
Table 14. HammerDB TPC-H query completion times (seconds, lower is better)
In the following figures taken from the FVP Management Server user Interface, Read (blue) and Write (orange)
lines depict the actual storage performance of one of the SQL Server virtual machines during tests .
Note: The extended nature of TPC-H tests yields less granular datapoints than shorter duration TPC-C tests . The
granularity of datapoints may also differ between Test 1 and Test 2 .
23
Virtual Machine Storage Latency
In Figure 19 below, SQL Server virtual machine storage latency from Test 1 (with FVP) is shown . The decreased
latency in this test translates to faster HammerDB query completion times .
Figure 19. Virtual machine storage latency during Test 1 (with FVP)
In Figure 20 below, SQL Server virtual machine storage latency from Test 2 (without FVP) is shown .
Figure 20. Virtual machine storage latency during Test 2 (without FVP)
Virtual Machine Storage IOPS
In Figure 21 below, SQL Server virtual machine storage IOPS from Test 1 (with FVP) is shown . The increased IOPS
in this test translates to faster HammerDB query completion times .
Figure 21. Virtual machine storage IOPS during Test 1 (with FVP)
24
In Figure 22 below, SQL Server virtual machine storage IOPS from Test 2 (without FVP) is shown .
Figure 22. Virtual machine storage IOPS during Test 2 (without FVP)
Virtual Machine Storage Throughput
In Figure 23 below, SQL Server virtual machine storage throughput from Test 1 (with FVP) is shown . The increased
throughput in this test translates to faster HammerDB query completion times .
Figure 23. Virtual machine storage throughput during Test 1 (with FVP)
In Figure 24 below, SQL Server virtual machine storage IOPS from Test 2 (without FVP) is shown .
Figure 24. Virtual machine storage throughput during Test 2 (without FVP)
25
Appendix C: Bill of Materials
The test configuration bill of materials is summarized in the following table:
AREA COMPONENT QUANTITY
Host Hardware Cisco UCS C220 M3 (2 x Intel E5-2640, 256 GB RAM) 2
Storage Hardware EMC VNXe3200 Array
• 8 x 200 GB MLC SSDs
• 16 x 600 GB 10K RPM SAS Disks
• 12 x 4 TB 7 .2K RPM NL-SAS Disks
1
Network HardwareCisco UCS 6248UP Fabric Interconnect 2
Cisco Nexus 2232TM Fabric Extender 2
Software
PernixData FVP 2 .5 .0 .1 2
VMware ESXi 5 .5 Update 2 2
VMware vCenter 5 .5 Update 2c 1
Microsoft Windows 2012 R2 Standard 5
Microsoft SQL Server 2012 Standard SP1 1
Microsoft SQL Server 2014 Enterprise 2
Table 15. Bill of Materials
26
References
PernixData Product Page
http://www .pernixdata .com/products
PernixData Proven Designs Page
http://www .pernixdata .com/solutions/pernixdata-proven-designs
Microsoft SQL Server Product Page
http://www .microsoft .com/en-us/server-cloud/products/sql-server/
VMware vSphere Product Page
http://www .vmware .com/products/vsphere
SQL Server on VMware Best Practices Guide
http://www .vmware .com/files/pdf/solutions/SQL_Server_on_VMware-Best_Practices_Guide .pdf
HammerDB Product Page
http://www .hammerdb .com/about .html
HammerDB Introduction to Transactional (TPC-C) Testing for all Databases
http://hammerora .sourceforge .net/hammerdb_transactionintro .pdf
HammerDB Microsoft SQL Server OLTP (Transactional) Load Testing
http://hammerora .sourceforge .net/hammerdb_mssql_oltp .pdf
HammerDB Introduction to Decision Support, Data Warehousing, Business Intelligence, and
Analytical Load Testing for all Databases
http://hammerora .sourceforge .net/hammerdb_dssintro .pdf
HammerDB Microsoft SQL Server Decision Support (DSS) Load Testing
http://hammerora .sourceforge .net/hammerdb_mssql_dss .pdf
Copyright © 2015 PernixData, Inc . All rights reserved . This product is protected by U .S . and international copyright and intellectual property laws .
PernixData is a registered trademark and FVP, , Flash Hypervisor, FVP Cluster and Flash Cluster are trademarks of PernixData, Inc . in the United States and/or other jurisdictions . All other brands, products, marks and names mentioned herein may be trademarks or service marks of, and are used to identify, products or services of their respective owners .
www.pernixdata.com | 1-855-PERNIX-D | @PernixData