swingbench for oracle research - doag.org · –complicated queries –against large tables ......

MARTIN BACH#DOAGDB17

SWINGBENCHAND ORACLE RESEARCH

Accenture Enkitec Group

Copyright © 2016 Accenture All rights reserved. |

USING SWINGBENCHFOR ORACLE RESEARCH

2

Swingbench is golden!In my more than a decade-long Oracle career I have been using Dominic Giles’s Swingbench more and more in research and publications.In fact there are only few situations I encountered that couldn’t be researched with the popular benchmark.I may have tweeted that …This session isn’t about the benchmark as such, but uses the author may not have thought about.

Copyright © 2016 Accenture All rights reserved. | 3

$(WHOAMI)MARTIN BACH

• I work for Accenture Enkitec Group– Opinions are strictly my own

• Authoring– Co-author of Pro Oracle RAC 11g on Linux– Expert Consolidation in Oracle Database 12c– Lead author of Expert Oracle Exadata 2nd edition

martincarstenbach.wordpress.com

@MartinDBA


AGENDAWHAT TO EXPECT

• A very quick introduction to Swingbench• Followed by a few useful Swingbench internals• Let’s start with a little bit of RAC tuning• Data Guard 12c events and effect of network latency next• A quick review of my research about Vector Transformation• Researching the effect of Write Back Flash Cache


Ooops. This got a bit longer than I expected

Sorry

QUICK INTO TO SWINGBENCH


A QUICK INTRO TO SWINGBENCHSWINGBENCH 101

What is Swingbench anyway?• Free load generator designed to stress Oracle• Thank you Dominic Giles!• Comes with 4 benchmarks– Order Entry (OE)– Sales History (SH)– Calling Circle *– Stresstest

• Part of a suite of other useful tools– Data Generator– Trace Analyser– …


A QUICK INTRO TO SWINGBENCHBENCHMARKS – OUT OF THE BOX (1)

Order Entry• Based on the OE sample schema• Set up once, run many times– Use oewizard to create the schema

• Emphasis on – RAC Interconnect– Memory– Heavy contention on small number of tables

Sales History• Based on the SH sample schema• Create using shwizard• Emphasis on– Complicated queries– Against large tables– Read only

• Scales well


A QUICK INTRO TO SWINGBENCHBENCHMARKS – OUT OF THE BOX (2)

Stresstest• Less sophisticated than the previous ones• A matter of DML against a single table• Doesn’t require a setup wizard• Super awesome for RAC experiments– It will appear twice in this talk– More on this later

Calling Circle• Simulates an online call-centre– Who would have thought?

• Data must be generated for each test run– Use the ccwizard to do so– Slightly more complex than the other benchmarks– I haven’t used it ever

• Focus on CPU and memory– Requires 1 CPU of load generator for 2 processors

on the database server– Not so much emphasis on I/O system


A QUICK INTRO TO SWINGBENCHEXECUTING THE BENCHMARKS

How much interactivity do you want?• All of the following are part of the package– swingbench– minbench– charbench

• I prefer charbench• Options– Either an XML configuration file– Or the command line

• There’s even a coordinator for the load generator

SOME SWINGBENCHINTERNALS


USEFUL THINGS TO KNOWLIGHTS OUT SCHEMA GENERATION

We do not use X11• There is a CLI mode for each wizard– Lots of options available– Suitable when you don’t have a X Server at hand

• My preferred way to create the schemas• Don’t scale too large– Data generator is Java based– Not the fastest way to generate data


USEFUL THINGS TO KNOWTHE SBUTIL COMMAND

What if you want more data?• Choosing too large a scale in the wizard takes

forever– Java threads insert rows– Not the most effective way

• Enter sbutil– A lot more effective!– Uses the database!


USEFUL THINGS TO KNOWSCHEMA DESIGN

There is a proper data model!• But how do you start writing queries?• Reverse engineered using SQL Developer Data

Modeler!– Example: OE schema– It’s easy to get familiar with it

• Bear in mind– OE is an OLTP schema– SH is a DWH-style schema– And there isn’t really a schema for the Stresstest

USING SWINGBENCH FOR RESEARCH


HOW DO YOU RESEARCH ORACLE?NOT PARTICULARLY EASY SOMETIMES

The need for repeatable test cases• I always aim at providing repeatable test cases to allow readers to follow along• It is hard to come up with– Repeatable test cases– Semi-realistic application– Something that you can use in classes– And your readers can reproduce

• Swingbench has all that!


EXAMPLE 1

19

Real Application Clusters TuningTo me it has always been important to work out how RAC works. The stresstest is a great way to find out how RAC can be made to scale.Example 1 will start out with a vanilla run on the stresstest benchmark and will gradually try to improve things.Note that access to ASH and AWR requires at least the Diagnostics Pack, some operations require the Tuning Pack as well


EXAMPLE 1RAC TUNING

First appearance of Stresstest in thispresentation• What does it do?– When starting it creates a table and an index– Fires off inserts, deletes, and updates– Based on XML configuration

• See what happens– Snapper– OEM Express– Others


EXAMPLE 1RAC TUNING - BASELINE

Run code without modification• Make use of Real Application Clusters • Example: 2 node RAC – 12.1.0.2.160419– 100 users– Both instance

• No change to code base• Capture performance information!

[enkx3db01:oracle:MBACH1] /home/oracle/mbach/swingbench/bin> ./charbench -cs //enkx3-scan/bothinstances -uc 100 \> -r st_100_both.xml -c ../configs/stresstest.xmlAuthor : Dominic GilesVersion : 2.5.0.971

Results will be written to st_100_both.xml.Hit Return to Terminate Run...

Time Users TPM TPS

2:55:01 PM 100 1310672 21664Completed Run.



Tanel Poder’s script


EXAMPLE 1RAC TUNING – WHAT TO IMPROVE?

Where do you start?• Simple solution: do not use the cluster– Common wisdom– Start the benchmark with a connection to a single instance– Does the node cope with that load?

• More complex solution– Review the code– Review the DDL– Use sqld360?


EXAMPLE 1RAC TUNING – SINGLE INSTANCE

Solution to common issue• Create a new service with a preferred and available

node• Restart benchmark– Gather performance information– New figures looking good!

[enkx3db01:oracle:MBACH1] /home/oracle/mbach/swingbench/bin> ./charbench -cs //enkx3-scan/singleinstance -uc 100 \>-r st_100_single.xml -c ../configs/stresstest.xmlAuthor : Dominic GilesVersion : 2.5.0.971

Results will be written to st_100_both.xml.Hit Return to Terminate Run...

Time Users TPM TPS

3:15:52 PM 100 1786072 29853


EXAMPLE 1RAC TUNING – APPLICATION

How can we change the application logic?• We can check the source!• StressTest.java– Comment out DDL in init()– Compile using antbuild script

• Make sure you create the table and indexesyourself from now on


EXAMPLE 1RAC TUNING – LOCAL INDEX

Change Table and Index DDL• Add hash partitioning• Indexes are local

[enkx3db01:oracle:MBACH11] /home/oracle/mbach/swingbench/bin> ./charbench -cs //enkx3-scan/bothinstances -uc 100 \> -r st_100_both_part.xml -c ../configs/stresstest.xmlAuthor : Dominic GilesVersion : 2.5.0.971

Results will be written to st_100_both_part.xml.Hit Return to Terminate Run...

Time Users TPM TPS

3:59:31 PM 100 1631527 27177


EXAMPLE 1RAC TUNING – LOCAL INDEX


EXAMPLE 1RAC TUNING – REVERSE KEY PK

Change Table and Index DDL• Add hash partitioning• Indexes are local; PK reverse

[enkx3db01:oracle:MBACH11] /home/oracle/mbach/swingbench/bin> ./charbench -cs //enkx3-scan/bothinstances -uc 100 \> -r st_100_both_unq_rev.xml -c ../configs/stresstest.xmlAuthor : Dominic GilesVersion : 2.5.0.971

Results will be written to st_100_both_unq_rev.xml.Hit Return to Terminate Run...

Time Users TPM TPS

5:33:12 AM 100 1673931 28249


EXAMPLE 1RAC TUNING – SEQUENCES

Change code to use sequences• Leave data model as it is• Create a new sequence• Reference it in the code

SQL> sho userUSER is "SOE"SQL> create sequence stresstest_seq cache 100000;

Sequence created.

SQL> alter table stresstesttable modify id2 default stresstest_seq.nextval;

Table altered.



[enkx3db01:oracle:MBACH1] /home/oracle/mbach/swingbench/bin> ./charbench -cs //enkx3-scan/bothinstances -uc 100 \> -r st_100_both_unq_rev_seq.xml -c ../configs/stresstest.xmlAuthor : Dominic GilesVersion : 2.5.0.971

Results will be written to st_100_both_unq_rev_seq.xml.Hit Return to Terminate Run...

Time Users TPM TPS

6:02:59 AM 100 1897763 31818


EXAMPLE 2

43

Data Guard: Impact of Network LatencySwingbench helps not only stress your benchmark-database, but it can also help you understand the effect of using different protection modes and/or understand the effect of the network.Oracle’s white paper about Synchronous Data Guard states that 5ms latency work, anything beyond maybe not so much.Really?


EXAMPLE 2DATA GUARD LATENCY

How could you come up with a test case about network latency?• Start with creating a DG environment– Two node RAC primary– Two node RAC standby– All virtualised on KVM– Using Oracle 12.1.0.2.170117 throughout

• Data Guard environment– NCDBA – primary database on rac12pri– NCDBB – corresponding standby database on rac12sec

• Benchmarks– Baseline with known good settings– Introduce Evil bad network – 100ms latency



Testcase I: Baseline• Using Maximum Performance– Log shipping is set to asynchronous

• “Ideal” network latency of < 1ms• Use swingbench to generate some load– Monitor transactions per second– Monitor primary database’s wait events

DGMGRL> show configuration

Configuration - ractest

Protection Mode: MaxPerformanceMembers:NCDBA - Primary databaseNCDBB - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:SUCCESS (status updated 16 seconds ago)

DGMGRL> show database 'NCDBA'

Database - NCDBA

Role: PRIMARYIntended State: TRANSPORT-ONInstance(s):NCDBA1NCDBA2

Database Status:SUCCESS

DGMGRL> show database 'NCDBB'

Database - NCDBB

Role: PHYSICAL STANDBYIntended State: APPLY-ONTransport Lag: 0 seconds (computed 1 second ago)Apply Lag: 0 seconds (computed 0 seconds ago)Average Apply Rate: 44.00 KByte/sReal Time Query: OFFInstance(s):NCDBB1NCDBB2 (apply instance)



Testcase I: Baseline• Based on Order Entry• Data Guard mode: Maximum Performance– Log shipping is set to asynchronous– “Ideal” network latency of < 1ms

• Use swingbench to generate some load– Monitor transactions per second– Monitor primary database’s wait events

$ ./charbench -cs //rac12pri-scan/swingbench_both \> -uc 20 -r results_20_maxperf.xml -rt 00:10

Author : Dominic GilesVersion : 2.5.0.971

Results will be written to results_20_maxperf.xml.Hit Return to Terminate Run...

Time Users TPM TPS1:59:55 PM 20 3852 255



Testcase II: Synchronous Log Shipping & Bad Network• Using Maximum Availability– Log shipping is set to synchronous

• Evil network – 100ms latency– Yes this is not realistic– But NetEM is fun to play with

• Use swingbench to generate some load– Monitor transactions per second– Monitor primary database’s wait events

DGMGRL> edit database 'NCDBA' set property LogXptMode = 'SYNC';Property "logxptmode" updated

DGMGRL> edit database 'NCDBB' set property LogXptMode = 'SYNC';Property "logxptmode" updated

DGMGRL> EDIT CONFIGURATION SET PROTECTION MODE AS MaxAvailability;Succeeded.

DGMGRL> show configuration

Configuration - ractest

Protection Mode: MaxAvailabilityMembers:NCDBA - Primary databaseNCDBB - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:SUCCESS (status updated 19 seconds ago)

DGMGRL>



What did I learn?• First of all: 100ms latency is unfair to apps• NetEM really works• Performance on my system is greatly impacted• Test your Data Guard • Be sensitive to network latency


EXAMPLE 3

57

In-Memory AggregationPart of the In-Memory OptionThe new hot technology for star queries• New SQL transformation operation• Designed to optimise joins between fact and

dimension• Oracle claims it gets better with more complexityDoes not require all tables to be in the IMCWill be chosen automatically based on CBO cost


EXAMPLE 3VECTOR TRANSFORMATION

Where does it help? And why would you care?• More performance• Less CPU required• No need for (bitmap) indexes, MViews, GTTs etc.• At least so the marketing material claims• Can we check?

Example query



Deliberately turned off

SQL> select /*+ gather_plan_statistics no_vector_transform */2 t.calendar_year, t.calendar_quarter_desc, c.cust_state_province, sum(s.quantity_sold)3 from sb_sh.customers c, sb_sh.sales s, sb_sh.times t4 where c.cust_id = s.cust_id and s.time_id = t.time_id and t.calendar_year = 2011 5 and c.cust_state_province = 'County Down'6 group by t.calendar_year, t.calendar_quarter_desc, c.cust_state_province;

CALE CALEND CUST_STATE_ SUM(S.QUANTITY_SOLD)---- ------ ----------- --------------------2011 2011-4 County Down 48460082011 2011-3 County Down 39280322011 2011-1 County Down 27185602011 2011-2 County Down 3442208

4 rows selected.

Elapsed: 00:00:18.08



And now turned on

SQL> select /*+ gather_plan_statistics vector_transform */2 t.calendar_year, t.calendar_quarter_desc, c.cust_state_province, sum(s.quantity_sold)3 from sb_sh.customers c, sb_sh.sales s, sb_sh.times t4 where c.cust_id = s.cust_id and s.time_id = t.time_id and t.calendar_year = 2011 5 and c.cust_state_province = 'County Down'6 group by t.calendar_year, t.calendar_quarter_desc, c.cust_state_province;

CALE CALEND CUST_STATE_ SUM(S.QUANTITY_SOLD)---- ------ ----------- --------------------2011 2011-4 County Down 48460082011 2011-3 County Down 39280322011 2011-1 County Down 27185602011 2011-2 County Down 3442208

4 rows selected.

Elapsed: 00:00:03.23


EXAMPLE 3: VECTOR TRANSFORMATIONEXADATA BENEFITS

Watch out for key vectors in the predicates section• Exadata example: key vectors are offloaded• Visible in the predicates for example

Predicate Information (identified by operation id):---------------------------------------------------

7 - storage("CUST_STATE_PROVINCE"='County Down')filter("CUST_STATE_PROVINCE"='County Down')

11 - storage(TO_NUMBER("T"."CALENDAR_YEAR")=2011)filter(TO_NUMBER("T"."CALENDAR_YEAR")=2011)

13 - access("ITEM_10"=INTERNAL_FUNCTION("C0") AND "ITEM_11"="C2" AND "ITEM_8"=INTERNAL_FUNCTION("C0") AND "ITEM_9"="C3")24 - storage((SYS_OP_KEY_VECTOR_FILTER("S"."CUST_ID",:KV0000) AND SYS_OP_KEY_VECTOR_FILTER("S"."TIME_ID",:KV0001)))

filter((SYS_OP_KEY_VECTOR_FILTER("S"."CUST_ID",:KV0000) AND SYS_OP_KEY_VECTOR_FILTER("S"."TIME_ID",:KV0001)))

Note-----

- vector transformation used for this statement


EXAMPLE 4

63

Write Back Flash CacheA new feature that came to Exadata in 11.2.3.2.1 allows systems that suffer from ”free buffer waits” to get some more breathing space.But how can this be seen on the database and/or storage cell level?


EXAMPLE 4WRITE BACK FLASH CACHE

Are you suffering from free buffer waits?• Very write-heavy workloads can benefit from switching to Write Back Flash Cache– Write to flash cards instead of disk– Flash Logging remains unaffected!

• The feature came out with the X3 – Exadata 11.2.3.2.1 is the minimum required version– A bit problematic at first – Reduces the flash capacity – Promise of higher write IOPS



Flash Cache in Exadata X5-2 and newer cellsCellCLI> list physicaldisk attributes name,deviceName,diskType,makeModel

8:0 /dev/sda HardDisk "HGST H7240AS60SUN4.0T"8:1 /dev/sdb HardDisk "HGST H7240AS60SUN4.0T"8:2 /dev/sdc HardDisk "HGST H7240AS60SUN4.0T"8:3 /dev/sdd HardDisk "HGST H7240AS60SUN4.0T"8:4 /dev/sde HardDisk "HGST H7240AS60SUN4.0T"8:5 /dev/sdf HardDisk "HGST H7240AS60SUN4.0T"8:6 /dev/sdg HardDisk "HGST H7240AS60SUN4.0T"8:7 /dev/sdh HardDisk "HGST H7240AS60SUN4.0T"8:8 /dev/sdi HardDisk "HGST H7240AS60SUN4.0T"8:9 /dev/sdj HardDisk "HGST H7240AS60SUN4.0T"8:10 /dev/sdk HardDisk "HGST H7240AS60SUN4.0T"8:11 /dev/sdl HardDisk "HGST H7240AS60SUN4.0T"FLASH_1_1 /dev/nvme3n1 FlashDisk "Oracle Flash Accelerator F160 PCIe Card"FLASH_2_1 /dev/nvme2n1 FlashDisk "Oracle Flash Accelerator F160 PCIe Card"FLASH_4_1 /dev/nvme0n1 FlashDisk "Oracle Flash Accelerator F160 PCIe Card"FLASH_5_1 /dev/nvme1n1 FlashDisk "Oracle Flash Accelerator F160 PCIe Card"



How do we measure?• The impact can be measured in the usual places– Storage servers– Compute nodes

• View them using– The cellcli tool– metric_iorm.pl– SQL*Plus or other Oracle client– Enterprise Manager

• The following examples are based on the Stresstestbenchmark

• An X3-2 quarter rack was tortured to obtain these numbers

[oracle@exa01dbadm01] /home/oracle/mbach/swingbench/bin > ./charbench -bs 00:01 -c ../configs/stresstest.xml \> -r stresstest1-out.xml -rt 00:45 -uc 40Author : Dominic GilesVersion : 2.5.0.971

Results will be written to stresstest1-out.xml.Hit Return to Terminate Run...

Time Users TPM TPS

9:44:16 AM 40 1224024 20758



Metrics on the storage serverCellCLI> list flashcachecontent where dbuniquename like 'MBACH.*' -> attributes objectnumber, cachedkeepsize, cachedsize, cachedwritesize, hitcount, misscount -> order by hitcount desc limit 10

169917 0 4782358528 4684587008 1654014 20312169914 0 2755551232 2681511936 597712 93314294967294 0 1228800 1130496 35954 087687 0 6013886464 4410646528 15001 087685 0 2094071808 66232320 7120 0169905 0 545161216 32792576 5582 562169919 0 94695424 91021312 4048 1992169916 0 6496256 6496256 2072 637169918 0 284581888 284205056 1929 872169913 0 479412224 4145152 1748 462

SQL> select owner, object_type, data_object_id, object_name from dba_objects2 where data_object_id = 169917;

OWNER OBJECT_TYP DATA_OBJECT_ID OBJECT_NAME--------------- ---------- -------------- ---------------SOE TABLE 169917 STRESSTESTTABLE

Previous execution



Measured by the RDBMS (1)

SQL> SELECT sid, name, value2 FROM v$sesstat NATURAL JOIN v$statname3 WHERE sid IN (SELECT sid FROM v$session WHERE program LIKE '%DBW%')4 AND (name LIKE '%flash cache%' or name like 'physical write%')5 AND value <> 0 ORDER BY sid;

SID NAME VALUE---------- ---------------------------------------------------------------- ----------

361 physical write total bytes 6.5745E+10361 cell writes to flash cache 8716578361 cell flash cache read hits 712361 physical write total IO requests 4486042361 physical writes non checkpoint 4959917361 physical write IO requests 4486042361 physical write requests optimized 4354086361 cell partial writes in flash cache 448361 physical writes 8025549361 physical writes from cache 8025549361 physical write total bytes optimized 5.3004E+10361 cell overwrites in flash cache 7665945361 physical write bytes 6.5745E+10361 physical write total multi block requests 96168

...



Measured by the RDBMS (2)

SQL> select metric_name, metric_value, metric_type from v$cell_global2 where lower(metric_name) like '%flash cache%writ%'3 and cell_name = '192.168.12.3’ and metric_value <> 0 order by metric_value desc;

METRIC_NAME --------------- -----------------Flash cache write bytes - first writes, overwrites, partial writes METRIC_VALUE METRIC_TYPE---------------------------------------------------------------------- 190438121472 bytesFlash cache write bytes - overwrites 154322657280 bytesFlash cache write bytes - first writes 36058734592 bytesFlash cache read bytes to hard disk writes 12534489088 bytesFlash cache write bytes - population writes due to read misses 7466672128 bytesFlash cache write bytes - populate columnar 1153433600 bytesFlash cache write requests - first writes, overwrites, partial writes 15539829Flash cache write requests - overwrites 13619454Flash cache write requests - first writes 1919265Flash cache write requests - population writes due to read misses 130588Flash cache read requests to hard disk writes 16034Flash cache write requests - populate columnar 1100

12 rows selected.


SUMMARY

71

Swingbench is pretty coolIn my career I have relied on it heavily. The first time I heard about it was probably in 2006, quite some time ago.Since then it hasn’t lost any appeal to me, and when researching I regularly use it to find out how a certain feature interacts with the database.I hope this could inspire you, too!

swingbench for oracle research - doag.org · –complicated queries –against large tables ......

Documents