emc scaleio for oracle database 12c solutions: oracle rac ... · pdf filewhite paper emc...
TRANSCRIPT
WHITE PAPER
EMC SCALEIO FOR ORACLE DATABASE 12C SOLUTIONS: Oracle RAC and Data Guard replication
ABSTRACT
This document showcases EMC ScaleIO clusters as the ideal choice for Oracle RAC®
clusters. It also demonstrates that Oracle Data Guard® replication can be used to
replicate databases between ScaleIO clusters that have Oracle RAC clusters
configured. As a CIO or Enterprise Applications Architect, why buy expensive
dedicated storage systems when your database servers provide you all the
horsepower you need?
June 2016
2
To learn more about how EMC products, services, and solutions can help solve your business and IT challenges, contact your local
representative or authorized reseller, visit www.emc.com, or explore and compare products in the EMC Store
Copyright © 2016 EMC Corporation. All Rights Reserved.
EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without
notice.
The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with
respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a
particular purpose.
Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.
For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.
Oracle, Oracle Data Guard, Oracle RAC are registered trademarks or trademarks of Oracle Corporation in the United States and/or
other jurisdictions. All other trademarks such as Cisco used herein are the property of their respective owners.
Part Number H15217
3
TABLE OF CONTENTS
Executive Summary ........................................................................................... 4
Test Configuration .............................................................................................. 4
Storage Configuration for the Oracle Database ............................................................... 6
PERFORMANCE TESTS ........................................................................................ 8
OLTP Tests ........................................................................................................ 8
100% Read Workload ................................................................................................. 8
Switchover to Remote Site ........................................................................................ 10
Switchover to Remote Site ........................................................................................ 14
OLTP Workload after Switchover ................................................................................ 15
Data Warehouse Workload ................................................................................ 16
Different Workloads Running on Two Clusters ...................................................... 17
100% Read Workload on SLOB Database, Data Warehouse Query on TPC-H Database...... 18
OLTP Workload on SLOB Database, Data Warehouse Query on TPC-H Database ............... 18
CONCLUSIONS ........................................................................................ 19
4
Executive Summary
Oracle databases need powerful compute nodes as well as storage that offer low latencies. EMC ScaleIO® portfolio consists of software-only option, VxRack® FLEX Node or a fully engineered VxRack System 1000® with FLEX Nodes. All three deployment choices offer both these characteristics in a very small footprint. Each solution can also scale as your business needs grow. This scaling occurs both in the compute and low latency storage areas. In other words, ScaleIO clusters are truly flexible making them optimal for massive, mission critical Oracle® database environment. All of Oracle database solutions work well on ScaleIO configurations.
This document showcases ScaleIO clusters as the ideal choice for Oracle RAC® clusters. It also demonstrates that Oracle Data Guard® replication can be used to replicate databases between ScaleIO clusters that have Oracle RAC clusters configured. If the primary and remote ScaleIO clusters have identical configurations (for example; the number and type of ScaleIO nodes), then the performance measured on the primary and remote clusters will be identical.
EMC ScaleIO is a software-defined platform for block storage – software that creates a server-based SAN from local application servers and storage to deliver flexible and scalable performance and capacity on demand. It is ideal for traditional and modern applications and for those customers evaluating OpenStack deployments. Production and non-production database environments benefit since hyper-convergence allows for extremely fast writes. Its massive parallelism delivers quick recovery and stable, predictable performance.
Oracle Data Guard is a replication solution for Oracle databases. Oracle Data Guard can be used to create, maintain, manage, and monitor one or more standby databases to enable production Oracle databases to survive disasters and data corruptions. Oracle Data Guard maintains these standby databases as copies of the production database. Oracle Data Guard can also be used to switchover to a standby database for planned events like migrations and upgrades.
Test Configuration
For this lab validation, EMC ScaleIO software-only deployment option was used herein referred to as
‘ScaleIO nodes’. The configured system for this lab study is equivalent to any of the VxRack FLEX Node
configurations available. Eight ScaleIO nodes were configured as two clusters. Each cluster had four
ScaleIO nodes. Cisco C240 servers were used as ScaleIO nodes.
5
Figure 1 - Solution architecture. The primary and standby site each has four Cisco UCS 240 M4 servers.
Each Cisco C240 server was equipped as follows: Equivalent to any VxRack FLEX Node configurations
2 x E5-2650 v3 Intel Xeon processors, 10 Cores per Socket
256 GB RAM
2 x 10Gb Ethernet NICs, 1 x 1Gb Ethernet NIC
2 x 400GB 2.5” SATA eMLC SSD drives
21 x 2.5” SAS 10K RPM drives ( one drive dedicated to the Operating System and Database software) (None of the 10K RPM drives were used for the database)
Table 1 - Server Configuration
Operating System Red Hat Enterprise Linux Server release 6.5 (Santiago)
Hyper-converged platform
EMC ScaleIO Software Release 1.32.4504.5
Database Oracle 12c
Table 2 - Server Software Components
Oracle Database 12c Grid Infrastructure was used as the clusterware for Real Application Clusters. Oracle
database 12c was used as the database release software.
The Operating System software, the Grid Infrastructure software and the Database software were placed
on disk partitions of one 10K rpm SAS drive.
One ScaleIO protection domain and one storage pool were configured on each cluster. Each cluster’s
storage pool had an SSD pool that was comprised of the two SSD drives from each of the four nodes in
the cluster.
Figure 2 below showcases details about the SSD pool. This figure shows that there is one storage domain,
domain1 and there is one storage pool, pool1. Two disks from each node compose the storage pool in a
four node cluster.
6
Figure 2 - ScaleIO GUI displaying backend configuration
Storage Configuration for the Oracle Database
Oracle ASM volume manager was used as Oracle database volume manager.
To prepare for importing the ScaleIO volumes into ASM, the ScaleIO SDC agent presented two volumes from the SSD pool to each of the nodes in the cluster.
Two ASM diskgroups were configured: DATA and FRA Figure 3 below shows the following:
1. The two block devices that are mapped to the volumes presented by the ScaleIO SDC.
2. The udev rules file associated a consistently named block device with the underlying ScaleIO SDC device.
3. A list of the two block device entries in the Linux /dev directory mapped to the ASM disks.
7
Figure 3 - Udev rules were used to give persistent names to ScaleIO volumes
Two Oracle ASM disk groups, DATA and FRA, were created from the two ScaleIO volumes with ASM
External redundancy, as ScaleIO uses mesh-mirroring for data protection by default. The below shows the
two disk groups and their properties.
Figure 4 - ASM disk groups
8
After ASM was configured, two sets of workloads (OLTP and Data Warehouse) were loaded in succession and performance tests were run.
PERFORMANCE TESTS
As shown in
Figure 1, there are two Oracle RAC clusters configured on the two ScaleIO clusters with four nodes on
each.
Databases were built on the primary cluster. An Oracle physical standby database was then created on the
other cluster. Oracle Data Guard was configured in maximum performance mode to maintain the standby
database. Oracle Data Guard maintains the standby database by transmitting redo data from the primary
database and then applying the redo to the standby database.
The Data Guard broker command-line interface (the dgmgrl command) was used to switchover the
databases between the primary and replication clusters. Performance of the OLTP and Data Warehouse
workloads was measured at each site.
The following provides details of the two database models tested:
1. OLTP Model: A 600GB tablespace was created, then test data was loaded using the Silly Little
Oracle Benchmark kit (SLOB). SLOB guarantees completely random access to the data. It’s a very
simple, scalable and freely available tool kit anyone can use to test storage suitability for Oracle
Database. SLOB allows users to test varying read/write ratios as well as selectable transaction
logging weight. Details about the SLOB benchmark can be found at https://kevinclosson.net/slob/
2. Data Warehousing Model: A 600GB TPC-H Line item table was populated with data generated using
the DBGEN utility. SQL statements that resulted in Full Table Scans were run. The storage
throughput was then measured.
OLTP Tests
For these tests, two workloads were run:
1. 100% Read workload: This workload resulted in mostly 8KB Reads.
2. 75%/25% Read/Write (R/W) workload: OLTP style.
The goal of these exercises was to increase the physical IO rates until the average service times of the
reads remained at or below one millisecond.
100% Read Workload
Using SLOB, a 100% 8KB read workload with 24 load generators per node was used and the resulting IO
rate on the ScaleIO cluster was measured.
This test shows that the ScaleIO cluster sustained an IO rate of ~ 540,000 reads IOPS while maintaining a
read latency of approximately 1 ms. The screen shot below shows the read workload test.
9
Figure 5 - ScaleIO GUI showing workload during 100% Read workload test
For more information on EMC ScaleIO architecture, click here.
The I/O rate reported by ScaleIO GUI is corroborated by Oracle’s AWR reports.
The AWR report in Figure 6 shows that the IO was evenly balanced among the four RAC instances.
Figure 6 - AWR RAC statistics shows that the workload is evenly spread across Oracle Instances
The data below shows the histograms of I/O latencies. Approximately 60% of the single Oracle block
(8KB) reads completed in less than 1 ms and ~ 92% completed in less than 2 ms. Only 1% of the reads
took longer than 4 ms.
Note: The AWR report shows that there are many ‘db file parallel read’ wait events. The ‘db file parallel read’ occurs
when there is a non-contiguous multi-block read. This means that Oracle issues multiple single block read requests in
one batch. However, storage receives only single block read requests.
10
Figure 7 - Histogram of wait events during 100% Read test
Note: Higher IO rates of 575,000 I/O per second were observed when the number of I/O generators was increased. However, at this I/O rate, the read latency increased to about 1.5msec.
Switchover to Remote Site
The Oracle Data Guard command line interface (dgmgrl) was used to switchover the database from the
primary to the replication site.
The database IO was stopped before the switchover was performed. A screenshot of the switchover is
shown below.
This screenshot in Figure 8 shows that initially, database SIO is the primary and the database SIO1 is the
physical standby database. After the ‘switchover to SIO1’ is executed, database SIO1 becomes the
primary and database SIO is the physical standby.
11
Figure 8 - Screenshot showing successful switchover of database
The IO rate and latencies observed on the remote site were similar to that seen on the primary site.
OLTP Mixed IO Workload Results
When the 75/25 R/W Oracle workload was presented to the database, the 4-node ScaleIO cluster was able
to maintain an I/O rate of ~ 320,000 IOPS. This test ran for 20 minutes.
12
The two screenshots below, one represents the ScaleIO GUI and the other, Oracle Enterprise Manager
(OEM) show that the I/O rate was maintained for the duration of the test.
Figure 9 - ScaleIO GUI showing workload during Mixed IO OLTP test
Figure 10 - Oracle Enterprise Manager showing IO load during Mixed OLTP testFigure 11
shows sections of the AWR report where the average read latency was ~ 1.04 ms and that User I/O class
waits accounted for ~ 60% of all wait times.
Latencies of ~ 1 ms or less are the normal thresholds for read latencies for single Oracle block reads for
Flash drives. The AWR statistics show that this threshold is being honored by the ScaleIO cluster.
The wait event ‘gc current grant 2-way’ is to be expected in an Oracle RAC configuration. The ‘gc current
grant 2-way’ is a message-oriented wait event statistics indicating that no block was received because it
was not cached in any instance. Instead a global grant was given, allowing the requesting instance to read
the block from disk or modify it.
13
Figure 11 - Top 10 Foreground events sorted by total wait time during mixed I/O OLTP tests
The AWR histogram data in Figure 12 shows the distribution of the latencies.
The wait event ‘db file sequential read’ is a single-block random read. The histogram shows that ~ 72% of
the reads completed in less than 1 ms and ~ 87% were less than 2 ms.
Transaction log writes are also a key performance indicator. The AWR histogram data shows that ~30%
of the log file parallel writes completed in less than 1 ms, 70% completed in less than 2 ms and only ~8%
took more than 4 ms.
Figure 12 - Histogram of wait events during mixed IO OLTP test
Figure 13 displays the AWR RAC statistics. This shows that the workload was evenly distributed among the
four Oracle instances.
14
Figure 13 - AWR Oracle RAC statistics shows workload evenly distributed
Switchover to Remote Site
The Oracle Data Guard command line interface (dgmgrl) was used to switchover the database from the
primary to the replication site.
The database IO was stopped before the switchover was performed. A screenshot of the switchover is
shown below.
Figure 14 shows that initially database SIO is the primary and the database SIO1 is the physical standby
database. After the ‘switchover to SIO1’ is executed, database SIO1 becomes the primary and database
SIO is the physical standby.
15
Figure 14 - Screenshot of database switchover to SI01
OLTP Workload after Switchover
An OLTP workload (75/25 R/W) was run on the remote cluster. As expected, the I/O throughput was
similar to the one that was sustained on the original production cluster.
16
Figure 15 -ScaleIO GUI showing workload during Mixed I/O OLTP test on the remote site
Data Warehouse Workload
A 600GB scale TPC-H Lineitem table was populated using the DBGEN utility. Multiple queries that result in
tablescans, were run simultaneously. The resulting I/O throughput was measured using both the ScaleIO
GUI and the Oracle Enterprise Manager.
Both of these showed that the I/O throughput was 7.3GB/sec as shown in Figure 16 and Figure 17.
(The tables of the TPCH database were populated with data from flat files. The flat files were generated
using the dbgen utility and these files were stored (in file systems) on a pool of HDD spindles. The 6.2 TB
that is shown is the sum of the size of the SSD pool (which housed the database files) and the HDD pool
(for the flat files)).
Figure 16 - ScaleIO dashboard- Parallel Query Scan Performance
17
Figure 17 - Oracle Enterprise Manager -Parallel Scan Query performance
The database was switched over to the remote site using the Data Guard cli (dgmgrl) and performance
was measured. The throughput and latencies observed on the remote site was similar to that seen on the
primary site.
Different Workloads Running on Two Clusters
Frequently, customers will want to have different workloads running on both clusters. We demonstrate
this by showing that an OLTP database can be active on one site, and a Data Warehouse database can be
active on the second site.
To do this, we built two databases:
1. A SLOB database that was 300GB in size.
2. A TPCH database that was 300GB in size.
The SLOB database had its primary database on site A and the standby database on Site B. The TPCH
database had its primary on site B and its standby on site A.
Figure 18 – SLOB and TPC-H sharing RAC and ScaleIO clusters
The workloads were run concurrently on both the SLOB and TPCH databases.
18
100% Read Workload on SLOB Database, Data Warehouse Query on TPC-H
Database
First, a 100% read workload was run. The SLOB database was able to sustain an I/O rate of 532,000 I/O
per second. The TPC-H database was able to sustain a bandwidth of 7.3 GB/sec. These I/O rates are
similar to what was observed when only one database was active. Figure 19 shows the IO rate to the
SLOB database during this test.
Figure 19 –I/O requests by I/O function
OLTP Workload on SLOB Database, Data Warehouse Query on TPC-H Database
Next, a 75/25 R/W workload was run on the SLOB database and TPC-H queries were run on the TPC-H
database.
The SLOB database was able to sustain an I/O rate of 224,000 I/O per second and the TPC-H database
was able to sustain a bandwidth of 4.4 GB/sec.
The TPC-H database’s parallel degree had to be reduced so that the redo data of the SLOB database could
be transmitted and applied to its Standby database in a timely manner. In other words, the TPC-H
database bandwidth had to be reduced so that there were sufficient I/O resources available for the SLOB
database’s redo apply.
Figure 20 shows the IO rates on the SLOB database and the TPCH database when an OLTP workload was
run on the SLOB database and TPC-H queries were run on the TPC-H database.
19
Figure 20 - Screenshots of TPC-H and SLOB IOPs
CONCLUSIONS
EMC ScaleIO is software-defined block storage that offers multiple deployment options from a software-
only, VxRack FLEX Node as a hyper-converged appliance to a fully engineered hyper-converged VxRack
System 1000 with FLEX Nodes. Each deployment drives agility, high performance, dense compute
delivering low latencies, while remaining extremely flexible, and maintaining a small footprint.
Performance demonstrated throughout this paper proves that EMC ScaleIO portfolio offers an ideal choice
for Oracle RAC clusters. It also demonstrates that Oracle Data Guard replication can be used to replicate
databases between ScaleIO clusters that have Oracle RAC clusters configured.
For more information on EMC ScaleIO, visit http://www.emc.com/storage/scaleio/index.htm.