![Page 1: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/1.jpg)
Load Testing Toolsfor Troubleshooting MySQL Concurrency Issues
May, 23, 2018
Sveta Smirnova
![Page 2: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/2.jpg)
• This is very personal webinar• No intended use• No best practices• No QA-specific tools
Real life don’t generate queriesEach customer has its own corner caseTypical issues usually already found by QA team before first GA release date
• Creative by nature
• I usually don’t do benchmarks• I use load tools for reproducible scenarios• It helps to fix them!
Introduction
2
![Page 3: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/3.jpg)
• This is very personal webinar• I usually don’t do benchmarks
• Unless have a reason toPerformance Schema Benchmarks: OLTP RW
• I use load tools for reproducible scenarios• It helps to fix them!
Introduction
2
![Page 4: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/4.jpg)
• This is very personal webinar• I usually don’t do benchmarks• I use load tools for reproducible scenarios
• Locking issues• Crashes• Custom slowdowns• Replication issues• Almost everything!
• It helps to fix them!
Introduction
2
![Page 5: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/5.jpg)
• This is very personal webinar• I usually don’t do benchmarks• I use load tools for reproducible scenarios• It helps to fix them!
Introduction
2
![Page 6: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/6.jpg)
Create binary log events3. Connect to master, create database foo.
4. In parallel client start some load:
mysqlslap --user=root --host=127.0.0.1 --port=13000 --create-schema=foo \
--query="create table if not exists t1(f1 int); insert into t1 values(1); \
drop table if exists t1;" -c 10 -i 10000
5. kill -9 slave process
6. Restart MTR in dirty mode:
...
Few Examples
3
![Page 7: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/7.jpg)
Generate parallel load6. Run concurrent environment:
sysbench --test=/home/sveta/src/sysbench/sysbench/tests/db/oltp.lua --mysql-engine-trx=yes \
--mysql-table-engine=innodb --oltp_table_size=1000 --oltp_tables_count=1 --mysql-user=root \
--mysql-host=127.0.0.1 --mysql-port=13000 --num-threads=8 --max-requests=1000 run
7. Turn OFF and ON slow log:
mysql> set global slow_query_log=0;
Query OK, 0 rows affected (0,09 sec)
mysql> set global slow_query_log=1;
Query OK, 0 rows affected (0,00 sec)
8. Now you have 4 descriptors:
...
Few Examples
3
![Page 8: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/8.jpg)
Prepare data...
--disable_query_log
--let $i=10
while($i)
{
insert into cartest values (null, 1);
--dec $i
}
--enable_query_log
...
Few Examples
3
![Page 9: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/9.jpg)
•What Do I Do?•Case Study: Slave Crash•Why Do I Do Load Tests?•Machines Which I Use•Binaries Which I Use•How Do I Scale•Summary
Table of Contents
4
![Page 10: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/10.jpg)
What Do I Do?
![Page 11: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/11.jpg)
• Benchmarking tool by naturemysql> SELECT id, k FROM sbtest2 WHERE id=5014;
+------+------+
| id | k |
+------+------+
| 5014 | 3972 |
+------+------+
1 row in set (0.16 sec)
• Access to• Status variables• Engine status• Performance Schema• All internal troubleshooting tools
Single Thread Issues: MySQL CLI
6
![Page 12: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/12.jpg)
• Benchmarking tool by naturemysql> SELECT id, k FROM sbtest2 WHERE id=5014;
+------+------+
| id | k |
+------+------+
| 5014 | 3972 |
+------+------+
1 row in set (0.16 sec)
• Access to• Status variables• Engine status• Performance Schema• All internal troubleshooting tools
Single Thread Issues: MySQL CLI
6
![Page 13: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/13.jpg)
• Shell is our friend$ while (true); do mysql -e "flush slow logs"; done
...
$ for i in ‘seq 1 1 100000‘; \
do mysql -e "insert into t1(id, myfield) values \
($i, ’‘cat /dev/urandom | tr -dc A-Za-z0-9 | head -c $(( RANDOM % (250 - 5 + 1 ) + 5 ))‘’)"; \
done
• Helps to• Generate data• Craft queries with slightly various parameters
• Execute in multiple terminals• To imitate concurrent load
More than One Query: SHELL
7
![Page 14: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/14.jpg)
• Shell is our friend$ while (true); do mysql -e "flush slow logs"; done
...
$ for i in ‘seq 1 1 100000‘; \
do mysql -e "insert into t1(id, myfield) values \
($i, ’‘cat /dev/urandom | tr -dc A-Za-z0-9 | head -c $(( RANDOM % (250 - 5 + 1 ) + 5 ))‘’)"; \
done
• Helps to• Generate data• Craft queries with slightly various parameters
• Execute in multiple terminals• To imitate concurrent load
More than One Query: SHELL
7
![Page 15: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/15.jpg)
• Shell is our friend$ while (true); do mysql -e "flush slow logs"; done
...
$ for i in ‘seq 1 1 100000‘; \
do mysql -e "insert into t1(id, myfield) values \
($i, ’‘cat /dev/urandom | tr -dc A-Za-z0-9 | head -c $(( RANDOM % (250 - 5 + 1 ) + 5 ))‘’)"; \
done
• Helps to• Generate data• Craft queries with slightly various parameters
• Execute in multiple terminals• To imitate concurrent load
More than One Query: SHELL
7
![Page 16: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/16.jpg)
• Queries without variations$ mysqlslap --concurrency=8 --number-of-queries=1000 \
--query="REPLACE INTO mytest (ts, id, f1, f2, f3, f4) \
VALUES (now(), 1000*rand(), ’field text’, 1000000*rand(), ’more text’, 1)" \
--create-schema=test \
--pre-query="CREATE TABLE test.mytest(ts TIMESTAMP, id INT NOT NULL PRIMARY KEY, \
f1 VARCHAR(255), f2 INT, f3 CHAR(20), f4 INT) ENGINE=INNODB" \
--post-query="DROP TABLE test.mytest"
Benchmark
Average number of seconds to run all queries: 113.591 seconds
Minimum number of seconds to run all queries: 113.591 seconds
Maximum number of seconds to run all queries: 113.591 seconds
Number of clients running queries: 8
Average number of queries per client: 125
• Supports auto-generated SQL• Helpful for• Should be used with care
• In auto-generate mode drops schemaNo matter if it has other tables of not!
• Always use option --no-drop• Do not use on production databases
Simple Tests: mysqlslap
8
![Page 17: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/17.jpg)
• Queries without variations• Supports auto-generated SQL
$ time mysqlslap --concurrency=8 --auto-generate-sql --number-of-queries=100
Benchmark
Average number of seconds to run all queries: 8.126 seconds
Minimum number of seconds to run all queries: 8.126 seconds
Maximum number of seconds to run all queries: 8.126 seconds
Number of clients running queries: 8
Average number of queries per client: 12
real 1m37.932s
user 0m0.032s
sys 0m0.008s
• Helpful for• Should be used with care
• In auto-generate mode drops schemaNo matter if it has other tables of not!
• Always use option --no-drop• Do not use on production databases
Simple Tests: mysqlslap
8
![Page 18: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/18.jpg)
• Queries without variations• Supports auto-generated SQL• Helpful for
• Simple query patterns• When you need to imitate some activity• Generate binary log events
• Should be used with care• In auto-generate mode drops schema
No matter if it has other tables of not!
• Always use option --no-drop• Do not use on production databases
Simple Tests: mysqlslap
8
![Page 19: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/19.jpg)
• Queries without variations• Supports auto-generated SQL• Helpful for• Should be used with care
• In auto-generate mode drops schemaNo matter if it has other tables of not!
• Always use option --no-drop• Do not use on production databases
Simple Tests: mysqlslap
8
![Page 20: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/20.jpg)
• Comes with standard OLTP tests$ sysbench --test=/home/sveta/src/sysbench/sysbench/tests/db/oltp.lua \
--mysql-engine-trx=yes --mysql-table-engine=innodb --oltp_table_size=10000 \
--oltp_tables_count=2 --num-threads=8 \
--mysql-user=root --mysql-host=127.0.0.1 --mysql-port=13002 prepare
sysbench 0.5: multi-threaded system evaluation benchmark
Creating table ’sbtest1’...
Inserting 10000 records into ’sbtest1’
Creating secondary indexes on ’sbtest1’...
Creating table ’sbtest2’...
Inserting 10000 records into ’sbtest2’
Creating secondary indexes on ’sbtest2’...
• Standard tests help to• Standard OLTP tests include• SysBench has standard hardware tests• I use mostly OLTP for troubleshooting
Standard and Custom Tests: SysBench
9
![Page 21: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/21.jpg)
• Comes with standard OLTP tests$ sysbench --test=/home/sveta/src/sysbench/sysbench/tests/db/oltp.lua \
--mysql-engine-trx=yes --mysql-table-engine=innodb --oltp_table_size=10000 \
--oltp_tables_count=2 --num-threads=8 \
--mysql-user=root --mysql-host=127.0.0.1 --mysql-port=13002 run
sysbench 0.5: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 8
Random number generator seed is 0 and will be ignored
Threads started!
...
Threads fairness:
events (avg/stddev): 1281.0000/12.29
execution time (avg/stddev): 1564.7510/1.16
• Standard tests help to• Standard OLTP tests include• SysBench has standard hardware tests• I use mostly OLTP for troubleshooting
Standard and Custom Tests: SysBench
9
![Page 22: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/22.jpg)
• Comes with standard OLTP tests• Standard tests help to
• Imitate some activity• Generate binary log events• Test effect of options/upgrades
• Standard OLTP tests include• SysBench has standard hardware tests• I use mostly OLTP for troubleshooting
Standard and Custom Tests: SysBench
9
![Page 23: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/23.jpg)
• Comes with standard OLTP tests• Standard tests help to• Standard OLTP tests include
• Read only• INSERTs only• DELETEs only• Mixed• More
• SysBench has standard hardware tests• I use mostly OLTP for troubleshooting
Standard and Custom Tests: SysBench
9
![Page 24: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/24.jpg)
• Comes with standard OLTP tests• Standard tests help to• Standard OLTP tests include• SysBench has standard hardware tests
• I use mostly OLTP for troubleshooting
Standard and Custom Tests: SysBench
9
![Page 25: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/25.jpg)
• Comes with standard OLTP tests• Standard tests help to• Standard OLTP tests include• SysBench has standard hardware tests• I use mostly OLTP for troubleshooting
Standard and Custom Tests: SysBench
9
![Page 26: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/26.jpg)
• True power for troubleshooting• You can imitate any load• Supports Lua scripting
Custom SysBench Tests
10
![Page 27: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/27.jpg)
• thread init• Initialize thread
• set vars• prepare• event• cleanup• Version 1.0+
• sysbench.cmdline.options• sysbench.cmdline.commands
SysBench: Test Structure
11
![Page 28: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/28.jpg)
• thread init• set vars
• Set user variables
• prepare• event• cleanup• Version 1.0+
• sysbench.cmdline.options• sysbench.cmdline.commands
SysBench: Test Structure
11
![Page 29: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/29.jpg)
• thread init• set vars• prepare
• Prepare test
• event• cleanup• Version 1.0+
• sysbench.cmdline.options• sysbench.cmdline.commands
SysBench: Test Structure
11
![Page 30: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/30.jpg)
• thread init• set vars• prepare• event
• Run event
• cleanup• Version 1.0+
• sysbench.cmdline.options• sysbench.cmdline.commands
SysBench: Test Structure
11
![Page 31: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/31.jpg)
• thread init• set vars• prepare• event• cleanup
• Cleanup test
• Version 1.0+• sysbench.cmdline.options• sysbench.cmdline.commands
SysBench: Test Structure
11
![Page 32: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/32.jpg)
• thread init• set vars• prepare• event• cleanup• Version 1.0+
• sysbench.cmdline.options• sysbench.cmdline.commands
SysBench: Test Structure
11
![Page 33: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/33.jpg)
function set_vars()
branches_size = branches_size or 1
tellers_size = tellers_size or 10
accounts_size = accounts_size or 100000
scale = scale or 1
mysql_table_engine = mysql_table_engine or "innodb"
end
function thread_init(thread_id)
set_vars()
end
SysBench 0.5+: Variables
12
![Page 34: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/34.jpg)
sysbench.cmdline.options = {
branches_size = {"Size of branches table", 1},
tellers_size = {"Size of tellers table", 10},
accounts_size = {"Size of accounts table", 100000},
scale = {"Test scale", 1},
mysql_table_engine = {"MySQL storage engine", "innodb"}
}
SysBench 1.0+: Variables
13
![Page 35: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/35.jpg)
function prepare()
set_vars()
db_connect()
db_query([[
create table accounts(
aid int not null primary key,
bid int,
abalance int,
filler char(84)
)
/*! ENGINE = ]] .. mysql_table_engine ..
[[ default character set=latin1 */
]])
db_bulk_insert_init("insert into branches (bid, bbalance, filler) values")
for i=1,(branches_size * scale) do
db_bulk_insert_next("(" .. i .. ", 0, ’’)")
end
db_bulk_insert_done()
end
SysBench 0.5+: Test Preparation
14
![Page 36: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/36.jpg)
function event(thread_id)
local aid = sb_rand(1, accounts_size * scale)
db_query("SELECT abalance FROM accounts WHERE aid = " .. aid)
end
SysBench 0.5+: Test Run
15
![Page 37: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/37.jpg)
num-threads Number of threads to use
max-requests Maximum number of events
max-time Total execution time
mysql-db Schema to run tests in
mysql-host|user|port|... Connection options
SysBench 0.5+: Hot Options
16
![Page 38: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/38.jpg)
threads Number of threads to use
events Maximum number of events
time Total execution time
db-ps-mode Prepared statements mode
mysql-db Schema to run tests in
mysql-host|user|port|... Connection options
SysBench 1.0+: Hot Options
17
![Page 39: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/39.jpg)
• Scripts can declare options• Default values• SysBench can validate options• Options are visible in help output
• Parallel execution• Processing result sets• Custom commands• Better reporting
sysbench 1.0: teaching an old dog new tricks
SysBench 1.0+: Major Scripting Improvements
18
![Page 40: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/40.jpg)
• Scripts can declare options• Parallel execution
• Processing result sets• Custom commands• Better reporting
sysbench 1.0: teaching an old dog new tricks
SysBench 1.0+: Major Scripting Improvements
18
![Page 41: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/41.jpg)
• Scripts can declare options• Parallel execution• Processing result sets
• Custom commands• Better reporting
sysbench 1.0: teaching an old dog new tricks
SysBench 1.0+: Major Scripting Improvements
18
![Page 42: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/42.jpg)
• Scripts can declare options• Parallel execution• Processing result sets• Custom commands
• Better reportingsysbench 1.0: teaching an old dog new tricks
SysBench 1.0+: Major Scripting Improvements
18
![Page 43: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/43.jpg)
• Scripts can declare options• Parallel execution• Processing result sets• Custom commands• Better reporting
sysbench 1.0: teaching an old dog new tricks
SysBench 1.0+: Major Scripting Improvements
18
![Page 44: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/44.jpg)
• Scripts can declare options• Parallel execution• Processing result sets• Custom commands• Better reporting
sysbench 1.0: teaching an old dog new tricks
SysBench 1.0+: Major Scripting Improvements
18
![Page 45: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/45.jpg)
Case Study: Slave Crash
![Page 46: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/46.jpg)
• Multi-channel slave• One master runs load on a table• Another master runs FLUSH SLOW LOGS• Is this FLUSH reason for the crash?
The Issue
20
![Page 47: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/47.jpg)
• Slave and two masters
• Copy options from production• To imitate load on updating master• Load on flushing master• Standard Sysbench OLTP load on slave• Slave did not crash
• I will create better test
Setup
21
![Page 48: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/48.jpg)
• Slave and two masters• Copy options from production
• To imitate load on updating master• Load on flushing master• Standard Sysbench OLTP load on slave• Slave did not crash
• I will create better test
Setup
21
![Page 49: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/49.jpg)
• Slave and two masters• Copy options from production• To imitate load on updating master
$ mysqlslap --defaults-file=./.my_updating_master.cnf --concurrency=50 \
--number-of-queries=500000000 \
--pre-query="CREATE TABLE IF NOT EXISTS test.t(ts varchar(26), \
id int unsigned not null primary key, f1 varchar(255), \
f2 bigint unsigned, f3 varchar(255), f4 bigint unsigned);" \
--query="REPLACE INTO t (ts, id, f1, f2, f3, f4) VALUES \
(now(), 1000*rand(), md5(rand()), 1000000*rand(), ’fixed’, 10000*rand())" \
--create-schema=test
• Load on flushing master• Standard Sysbench OLTP load on slave• Slave did not crash
• I will create better test
Setup
21
![Page 50: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/50.jpg)
• Slave and two masters• Copy options from production• To imitate load on updating master• Load on flushing master
$ while (true); do mysql -e "flush slow logs"; done
• Standard Sysbench OLTP load on slave• Slave did not crash
• I will create better test
Setup
21
![Page 51: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/51.jpg)
• Slave and two masters• Copy options from production• To imitate load on updating master• Load on flushing master• Standard Sysbench OLTP load on slave
$ sysbench --threads=16 --events=0 --time=0 \
--mysql-host=127.0.0.1 --mysql-port=13002 --mysql-user=root \
./oltp_read_only.lua --tables=4 --table_size=10000 prepare
• Slave did not crash• I will create better test
Setup
21
![Page 52: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/52.jpg)
• Slave and two masters• Copy options from production• To imitate load on updating master• Load on flushing master• Standard Sysbench OLTP load on slave
$ sysbench --threads=16 --events=0 --time=0 \
--mysql-host=127.0.0.1 --mysql-port=13002 --mysql-user=root \
./oltp_read_only.lua --tables=4 --table_size=10000 run
• Slave did not crash• I will create better test
Setup
21
![Page 53: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/53.jpg)
• Slave and two masters• Copy options from production• To imitate load on updating master• Load on flushing master• Standard Sysbench OLTP load on slave• Slave did not crash
• I will create better test
Setup
21
![Page 54: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/54.jpg)
• Optionssysbench.cmdline.options = {
table_size = {"Number of rows per table", 1000},
tables = {"Number of tables", 1}
}
• Now wait ,
Custom SysBench Script on Updating Master
22
![Page 55: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/55.jpg)
• Commandssysbench.cmdline.commands = {
prepare = {cmd_prepare, sysbench.cmdline.PARALLEL_COMMAND},
run = {cmd_run, sysbench.cmdline.PARALLEL_COMMAND}
}
• Must be after function definitions
• Now wait ,
Custom SysBench Script on Updating Master
22
![Page 56: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/56.jpg)
• Thread initialization for run commandfunction thread_init()
drv = sysbench.sql.driver()
con = drv:connect()
end
• Now wait ,
Custom SysBench Script on Updating Master
22
![Page 57: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/57.jpg)
• Values templateslocal ts_value_template = "####-##-#####:##:##.######"
local f1_value_template = "######-###.######"
local f3_value_template = "#####-###.######"
• Now wait ,
Custom SysBench Script on Updating Master
22
![Page 58: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/58.jpg)
• Prepare commandfunction cmd_prepare()
local drv = sysbench.sql.driver()
local con = drv:connect()
for i = sysbench.tid % sysbench.opt.threads + 1, sysbench.opt.tables, sysbench.opt.threads
do
create_table(con, i)
end
end
• Now wait ,
Custom SysBench Script on Updating Master
22
![Page 59: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/59.jpg)
• create table definitionfunction create_table(con, table_num)
local query
print(string.format("Creating table ’sbtest%d’...", table_num))
query = string.format([[
CREATE TABLE sbtest%d(
ts varchar(26),
id int unsigned not null primary key,
f1 varchar(255),
f2 bigint unsigned,
f3 varchar(255),
f4 bigint unsigned
) engine=innodb]],
table_num)
con:query(query)
• Now wait ,
Custom SysBench Script on Updating Master
22
![Page 60: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/60.jpg)
• create table definitionif (sysbench.opt.table_size > 0) then
print(string.format("Inserting %d records into ’sbtest%d’",
sysbench.opt.table_size, table_num))
end
query = "INSERT INTO sbtest" .. table_num .. "(ts, id, f1, f2, f3, f4) VALUES"
con:bulk_insert_init(query)
for i = 1, sysbench.opt.table_size do
query = string.format("(’%s’, %d, ’%s’, %d, ’%s’, %d)",
sysbench.rand.string(ts_value_template), i,
sysbench.rand.string(f1_value_template),
sb_rand(1, sysbench.opt.table_size),
sysbench.rand.string(f3_value_template),
sb_rand(1, sysbench.opt.table_size))
con:bulk_insert_next(query)
end
con:bulk_insert_done()
end
• Now wait ,
Custom SysBench Script on Updating Master
22
![Page 61: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/61.jpg)
• Load test itselffunction event()
local table_name = "sbtest" .. sysbench.rand.uniform(1, sysbench.opt.tables)
local query = string.format("REPLACE INTO %s (ts, id, f1, f2, f3, f4) VALUES " ..
"(’%s’, %d, ’%s’, %d, ’%s’, %d)",
table_name, sysbench.rand.string(ts_value_template),
sb_rand(1, sysbench.opt.table_size),
sysbench.rand.string(f1_value_template),
sb_rand(1, sysbench.opt.table_size),
sysbench.rand.string(f3_value_template),
sb_rand(1, sysbench.opt.table_size))
con:query(query)
end
• Now wait ,
Custom SysBench Script on Updating Master
22
![Page 62: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/62.jpg)
• Preparing...$ sysbench --threads=16 --events=0 --time=0 \
--mysql-host=127.0.0.1 --mysql-port=13001 --mysql-user=root \
slave_crash.lua --tables=4 --table_size=10000 prepare
WARNING: Both event and time limits are disabled, running an endless test
sysbench 1.0.14-aa52c53 (using bundled LuaJIT 2.1.0-beta2)
Initializing worker threads...
Creating table ’sbtest2’...
Creating table ’sbtest1’...
Creating table ’sbtest4’...
Creating table ’sbtest3’...
Inserting 10000 records into ’sbtest1’
Inserting 10000 records into ’sbtest2’
Inserting 10000 records into ’sbtest4’
Inserting 10000 records into ’sbtest3’
• Now wait ,
Custom SysBench Script on Updating Master
22
![Page 63: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/63.jpg)
• And executing$ sysbench --threads=16 --events=0 --time=0 \
--mysql-host=127.0.0.1 --mysql-port=13001 --mysql-user=root \
slave_crash.lua --tables=4 --table_size=10000 run
WARNING: Both event and time limits are disabled, running an endless test
sysbench 1.0.14-aa52c53 (using bundled LuaJIT 2.1.0-beta2)
Running the test with following options:
Number of threads: 16
Initializing random number generator from current time
Initializing worker threads...
Threads started!
• Now wait ,
Custom SysBench Script on Updating Master
22
![Page 64: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/64.jpg)
• Now wait ,
Custom SysBench Script on Updating Master
22
![Page 65: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/65.jpg)
Why Do I Do Load Tests?
![Page 66: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/66.jpg)
• An option makes our server X % slower• Feature is slow• Query is slow• Benchmarks techniques
Customer’s Performance Issues
24
![Page 67: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/67.jpg)
• Before recommending for customers• To identify improvements in new versions• To find out if common guess is working
Effect of Options
25
![Page 68: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/68.jpg)
• Locking issues• Crashes• Other concurrency issues
To Repeat
26
![Page 69: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/69.jpg)
• Built-in data generation functionality• Scriptable data distribution
function set_vars()
branches_size = branches_size or 1
tellers_size = tellers_size or 10
accounts_size = accounts_size or 100000
scale = scale or 1
mysql_table_engine = mysql_table_engine or "innodb"
end
Generate Data
27
![Page 70: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/70.jpg)
• Built-in data generation functionality• Scriptable data distribution
function prepare()
set_vars()
db_connect()
...
db_bulk_insert_init("insert into branches (bid, bbalance, filler) values")
for i=1,(branches_size * scale) do
db_bulk_insert_next("(" .. i .. ", 0, ’’)")
end
db_bulk_insert_done()
...
Generate Data
27
![Page 71: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/71.jpg)
• Built-in data generation functionality• Scriptable data distribution
function prepare()
...
db_bulk_insert_init("insert into tellers (tid, bid, tbalance, filler) values")
for i=1,(tellers_size * scale) do
db_bulk_insert_next("(" .. i .. ", " .. ((i % (branches_size * scale)) + 1) .. ", 0, ’’)")
end
db_bulk_insert_done()
db_bulk_insert_init("insert into accounts (aid, bid, abalance, filler) values")
for i=1,(accounts_size * scale) do
db_bulk_insert_next("(" .. i .. ", " .. ((i % (branches_size * scale)) + 1) .. ", 0, ’’)")
end
db_bulk_insert_done()
...
Generate Data
27
![Page 72: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/72.jpg)
Machines Which I Use
![Page 73: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/73.jpg)
• Ideal solution• Requirement for true benchmarks• Performance tests• Test which require use of all machine resources
Exclusive Lab Machine
29
![Page 74: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/74.jpg)
• Tests which needs to be shared• High resource usage tests• Long running tests• Can affect and be affected by tests of others
Percona Shared Lab Machine
30
![Page 75: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/75.jpg)
• Short tests• Not requiring powerful hardware• Test drafts before porting them to lab machines
My Own Laptop
31
![Page 76: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/76.jpg)
Binaries Which I Use
![Page 77: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/77.jpg)
• Same as customers use• With thoroughly tested options• Optimized for performance
Standard Binaries
33
![Page 78: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/78.jpg)
• Quick tests• Special options• Debug builds for crashes
Self-Compiled
34
![Page 79: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/79.jpg)
How Do I Scale
![Page 80: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/80.jpg)
• Copy only relevant data• Only tables which participate in the problem• Part of data if table is huge
LIMIT
WHERE
mysqldump --where
• Only columns which participate in the query• Not always leads to issue repetition!
Lack of disk space
36
![Page 81: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/81.jpg)
• Some options must be scaled• innodb thread concurrency• innodb concurrency tickets
If you changed innodb thread concurrency, check the user manual
• innodb commit concurrency• innodb purge threads• innodb [read|write] io threads• thread concurrency• thread cache size
Only if you scale number or connections in your test
• Use scale coefficient• SCALE = NUMBER OF CPU CORES ON PRODUCTION/ NUMBER OF CPU CORES ON TEST
• Example• innodb thread concurrency test =innodb thread concurrency / SCALE
• Only if greater than number of CPU cores on thetest server or unlimited
Lack of CPU cores
37
![Page 82: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/82.jpg)
• Some options must be scaled• Use scale coefficient
• SCALE = NUMBER OF CPU CORES ON PRODUCTION/ NUMBER OF CPU CORES ON TEST
• Example• innodb thread concurrency test =innodb thread concurrency / SCALE
• Only if greater than number of CPU cores on thetest server or unlimited
Lack of CPU cores
37
![Page 83: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/83.jpg)
• Some options must be scaled• Use scale coefficient
• SCALE = NUMBER OF CPU CORES ON PRODUCTION/ NUMBER OF CPU CORES ON TEST
• Example• innodb thread concurrency test =innodb thread concurrency / SCALE
• Only if greater than number of CPU cores on thetest server or unlimited
Lack of CPU cores
37
![Page 84: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/84.jpg)
Table : innodb thread concurrency scale
Production: 16 cores Test: 4 cores Adjust number of client threads?
0 0 It depends
4 4 Not
16 4 Yes
32 8 Yes
128 32 Yes
Scale coefficient example
38
![Page 85: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/85.jpg)
• innodb io capacity• Depends on disk speed!• Can give very different results if disk speed is not
taken in account• innodb io capacity max
Disk-related options
39
![Page 86: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/86.jpg)
• Some options must be scaled
• Use scale coefficient• SCALE = RAM ON PRODUCTION / RAM ON TEST
• Scale return time for performance tests• Use less data to repeat production slowdowns
• Care about data distribution!• Check EXPLAIN output• More waiting time can be better for testing:
easier to notice difference
Lack of RAM
40
![Page 87: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/87.jpg)
• Some options must be scaled• Use scale coefficient
• SCALE = RAM ON PRODUCTION / RAM ON TEST
• Scale return time for performance tests• Use less data to repeat production slowdowns
• Care about data distribution!• Check EXPLAIN output• More waiting time can be better for testing:
easier to notice difference
Lack of RAM
40
![Page 88: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/88.jpg)
• Some options must be scaled• Use scale coefficient
• SCALE = RAM ON PRODUCTION / RAM ON TEST• Scale return time for performance tests
• Use less data to repeat production slowdowns• Care about data distribution!• Check EXPLAIN output• More waiting time can be better for testing:
easier to notice difference
Lack of RAM
40
![Page 89: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/89.jpg)
• Some options must be scaled• Use scale coefficient
• SCALE = RAM ON PRODUCTION / RAM ON TEST• Scale return time for performance tests• Use less data to repeat production slowdowns
• Care about data distribution!• Check EXPLAIN output• More waiting time can be better for testing:
easier to notice difference
Lack of RAM
40
![Page 90: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/90.jpg)
• innodb buffer pool size• innodb log file size
• only if changed buffer pool size• innodb buffer pool chunk size
• only if very large on production• It is unlikely what you will have to tune InnoDB
Buffer Pool resizing
• innodb buffer pool instances• query cache size, key buffer size• Do not adjust session options• table open cache,table definition cache
RAM-related options: scale them
41
![Page 91: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/91.jpg)
• innodb buffer pool size• innodb log file size• innodb buffer pool chunk size• innodb buffer pool instances
• Only if current number of instances does notmake sense on test server.
• Example: 96G buffer pool on production with 12instances and you test on laptop with 8G memory
• query cache size, key buffer size• Do not adjust session options• table open cache,table definition cache
RAM-related options: scale them
41
![Page 92: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/92.jpg)
• innodb buffer pool size• innodb log file size• innodb buffer pool chunk size• innodb buffer pool instances• query cache size, key buffer size
• only if larger than available memory
• Do not adjust session options• table open cache,table definition cache
RAM-related options: scale them
41
![Page 93: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/93.jpg)
• innodb buffer pool size• innodb log file size• innodb buffer pool chunk size• innodb buffer pool instances• query cache size, key buffer size• Do not adjust session options
• table open cache,table definition cache
RAM-related options: scale them
41
![Page 94: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/94.jpg)
• innodb buffer pool size• innodb log file size• innodb buffer pool chunk size• innodb buffer pool instances• query cache size, key buffer size• Do not adjust session options• table open cache,table definition cache
• Do not adjust
RAM-related options: scale them
41
![Page 95: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/95.jpg)
• join buffer size• [max ]binlog [stmt ]cache size• max heap table size• tmp table size• net buffer length• parser max mem size• read [rnd ]buffer size• sort buffer size
Session options: do not scale
42
![Page 96: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/96.jpg)
Summary
![Page 97: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/97.jpg)
MySQL Command Line client• Always
BASH
mysqlslap
SysBench
Tools Described
44
![Page 98: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/98.jpg)
MySQL Command Line client
BASH• Many queries in single thread
mysqlslap
SysBench
Tools Described
44
![Page 99: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/99.jpg)
MySQL Command Line client
BASH
mysqlslap• Simple queries concurrently
SysBench
Tools Described
44
![Page 100: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/100.jpg)
MySQL Command Line client
BASH
mysqlslap
SysBench• Custom tests
Tools Described
44
![Page 101: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/101.jpg)
• Benchmarking tools• More than just measuring performance• Help when more than single query involved• Imitate real life workload
• Do much more than described in theirs usermanuals if used creatively!
Summary
45
![Page 102: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/102.jpg)
• Benchmarking tools• More than just measuring performance• Help when more than single query involved• Imitate real life workload
• Do much more than described in theirs usermanuals if used creatively!
Summary
45
![Page 103: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/103.jpg)
Future webinar with MySQL benchmarking experts
Send your questions
https://twitter.com/svetsmirnova/status/999066023755223040
Call for Questions!
46
![Page 104: Load Testing Tools - Percona · Load Testing Tools for Troubleshooting MySQL Concurrency Issues May, 23, 2018 Sveta Smirnova This is very personal webinar No intended use No best](https://reader034.vdocument.in/reader034/viewer/2022052000/60128152dcc4321d2e0db5ff/html5/thumbnails/104.jpg)
http://www.slideshare.net/SvetaSmirnova
https://twitter.com/svetsmirnova
Thank you!
47