mysql query profiling and optimization€¦ · query profiling and optimization bogdan kecman mysql...

MySQLquery profiling and optimization

Bogdan KecmanMySQL Principal Technical Engineerbogdan.kecman@oracle.combogdan.kecman@bad-team.net

Developer’s mDayNovi Sad, November 2018

Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

2

Industry Leaders Rely on MySQL

3

CloudCloud

Web & Enterprise Web & Enterprise OEM & ISVsOEM & ISVs

MySQL Cluster is running most real time systems!

4

MySQL is no1 database in the cloud

5

HostingHosting IaaS, PaaSIaaS, PaaS

SaaSSaaS

6

Speed!!!

7

What if we don’t get it?

How is query executed?

• parser

• optimizer

• executor

• storage engine

8

How to understand what’s going on?

• information_schema

• performance_schema

• sys

• EXPLAIN SELECT...

Special databases / schemas

9

SYS database/schema

• Used to be external, embedded in MySQL since 5.7.7

• Safe to drop if you don’t want it

• performance_schema must be enabled for it to work– performance_schema=ON in [mysqld] section of your config file– SHOW VARIABLES LIKE 'perf%'; – will show you running configuration

• User access SYS need to have SELECT and EXECUTE on SYS db, INSERT and UPDATE on sys_config table, and additional privileges for some of the stored procedures (check documentation for each of them), SELECT on performance_schema, PROCESS for i_schema

Collection of objects that will help you interpret performance data

10

Performance instrumentation

• For most uses default performance_schema configuration is enough

• Turning on instruments have performance impact (not significant ..)

• Resetting instruments to default value:CALL sys.ps_setup_reset_to_default(TRUE);

• Turning instruments on/offCALL sys.ps_setup_enable_instrument('wait');

CALL sys.ps_setup_disable_instrument('wait');

CALL sys.ps_setup_enable_consumer('current');CALL sys.ps_setup_disable_consumer('current');

enable/disable instruments

11

Performance instrumentationTurn everything on :)

12

mysql> CALL ps_setup_enable_consumer('');+----------------------+| summary |+----------------------+| Enabled 15 consumers |+----------------------+1 row in set (0.00 sec)

mysql> CALL ps_setup_enable_instrument('');+--------------------------+| summary |+--------------------------+| Enabled 1208 instruments |+--------------------------+1 row in set (0.01 sec)

Performance instrumentation

• ps_setup_save();

• ps_setup_reload_saved();

• https://dev.mysql.com/doc/refman/8.0/en/sys-ps-setup-save.html

SAVE before you play!

13

QUERY TUNING

• Find the query to tune– mysqldumpslow– MySQL Enterprise Monitor

• Execution plan

• Query Statistics

Basics

14

QUERY TUNINGExternal tools

15

• MySQL WorkBench

• Others

QUERY TUNINGManual way

16

mysql> SET SESSION profiling = 1;Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> select * from products join productlines using (productLine) join orderdetails using (productCode) join orders using (orderNumber) right join customers using (customerNumber) where customers.salesRepEmployeeNumber in (select employeeNumber from employees join offices using (officeCode) where offices.city='Boston');

...RESULT...

mysql [localhost] {root} (classicmodels) > show profiles;

| 1 | 0.01033250 | select * from products …

QUERY TUNINGManual way

17

mysql> SELECT SEQ,STATE,DURATION,CPU_USER,CPU_SYSTEM,SOURCE_FUNCTION,SOURCE_FILE,SOURCE_LINE FROM INFORMATION_SCHEMA.PROFILING WHERE QUERY_ID=1;+-----+----------------------+----------+----------+------------+-----------------------+----------------------+-------------+| SEQ | STATE | DURATION | CPU_USER | CPU_SYSTEM | SOURCE_FUNCTION | SOURCE_FILE | SOURCE_LINE |+-----+----------------------+----------+----------+------------+-----------------------+----------------------+-------------+| 2 | starting | 0.000180 | 0.000177 | 0.000000 | NULL | NULL | NULL || 3 | checking permissions | 0.000010 | 0.000009 | 0.000000 | check_access | sql_authorization.cc | 1869 || 4 | checking permissions | 0.000003 | 0.000003 | 0.000000 | check_access | sql_authorization.cc | 1869 || 5 | checking permissions | 0.000003 | 0.000002 | 0.000000 | check_access | sql_authorization.cc | 1869 || 6 | checking permissions | 0.000002 | 0.000003 | 0.000000 | check_access | sql_authorization.cc | 1869 || 7 | checking permissions | 0.000005 | 0.000004 | 0.000000 | check_access | sql_authorization.cc | 1869 || 8 | checking permissions | 0.000003 | 0.000003 | 0.000000 | check_access | sql_authorization.cc | 1869 || 9 | checking permissions | 0.000007 | 0.000007 | 0.000000 | check_access | sql_authorization.cc | 1869 || 10 | Opening tables | 0.000166 | 0.000167 | 0.000000 | open_tables | sql_base.cc | 5514 || 11 | init | 0.000014 | 0.000014 | 0.000000 | execute | sql_select.cc | 514 || 12 | System lock | 0.000017 | 0.000016 | 0.000000 | mysql_lock_tables | lock.cc | 332 || 13 | optimizing | 0.000017 | 0.000017 | 0.000000 | optimize | sql_optimizer.cc | 213 || 14 | statistics | 0.000087 | 0.000087 | 0.000000 | optimize | sql_optimizer.cc | 422 || 15 | preparing | 0.000072 | 0.000072 | 0.000000 | optimize | sql_optimizer.cc | 511 || 16 | executing | 0.000005 | 0.000004 | 0.000000 | exec | sql_executor.cc | 211 || 17 | Sending data | 0.003201 | 0.002744 | 0.000100 | exec | sql_executor.cc | 286 || 18 | end | 0.000007 | 0.000005 | 0.000000 | execute | sql_select.cc | 564 || 19 | query end | 0.000017 | 0.000016 | 0.000002 | mysql_execute_command | sql_parse.cc | 4320 || 20 | closing tables | 0.000012 | 0.000010 | 0.000002 | mysql_execute_command | sql_parse.cc | 4366 || 21 | freeing items | 0.006470 | 0.000021 | 0.000003 | mysql_parse | sql_parse.cc | 4978 || 22 | cleaning up | 0.000036 | 0.000032 | 0.000004 | dispatch_command | sql_parse.cc | 1960 |+-----+----------------------+----------+----------+------------+-----------------------+----------------------+-------------+

QUERY TUNINGManual way

18

mysql> SHOW PROFILE for query 1;+----------------------+----------+| Status | Duration |+----------------------+----------+| starting | 0.000180 || checking permissions | 0.000010 || checking permissions | 0.000003 || checking permissions | 0.000003 || checking permissions | 0.000002 || checking permissions | 0.000005 || checking permissions | 0.000003 || checking permissions | 0.000007 || Opening tables | 0.000166 || init | 0.000014 || System lock | 0.000017 || optimizing | 0.000017 || statistics | 0.000087 || preparing | 0.000072 || executing | 0.000005 || Sending data | 0.003201 || end | 0.000007 || query end | 0.000017 || closing tables | 0.000012 || freeing items | 0.006470 || cleaning up | 0.000036 |+----------------------+----------+

QUERY TUNINGManual way

19

mysql> SHOW PROFILE CPU FOR QUERY 1;+----------------------+----------+----------+------------+| Status | Duration | CPU_user | CPU_system |+----------------------+----------+----------+------------+| starting | 0.000180 | 0.000177 | 0.000000 || checking permissions | 0.000010 | 0.000009 | 0.000000 || checking permissions | 0.000003 | 0.000003 | 0.000000 || checking permissions | 0.000003 | 0.000002 | 0.000000 || checking permissions | 0.000002 | 0.000003 | 0.000000 || checking permissions | 0.000005 | 0.000004 | 0.000000 || checking permissions | 0.000003 | 0.000003 | 0.000000 || checking permissions | 0.000007 | 0.000007 | 0.000000 || Opening tables | 0.000166 | 0.000167 | 0.000000 || init | 0.000014 | 0.000014 | 0.000000 || System lock | 0.000017 | 0.000016 | 0.000000 || optimizing | 0.000017 | 0.000017 | 0.000000 || statistics | 0.000087 | 0.000087 | 0.000000 || preparing | 0.000072 | 0.000072 | 0.000000 || executing | 0.000005 | 0.000004 | 0.000000 || Sending data | 0.003201 | 0.002744 | 0.000100 || end | 0.000007 | 0.000005 | 0.000000 || query end | 0.000017 | 0.000016 | 0.000002 || closing tables | 0.000012 | 0.000010 | 0.000002 || freeing items | 0.006470 | 0.000021 | 0.000003 || cleaning up | 0.000036 | 0.000032 | 0.000004 |+----------------------+----------+----------+------------+

QUERY TUNINGManual way – NEW using performance_schema instead of i_s

20

mysql> select * from performance_schema.events_statements_history;mysql> select * from performance_schema.events_statements_history_long;mysql> select * from performance_schema.events_stages_history;mysql> select * from performance_schema.events_stages_history_long;mysql> show tables like 'events_%';+------------------------------------------------------+| Tables_in_performance_schema (events_%) |+------------------------------------------------------+| events_errors_summary_by_account_by_error || events_errors_summary_by_host_by_error || events_errors_summary_by_thread_by_error || events_errors_summary_by_user_by_error || events_errors_summary_global_by_error || events_stages_current || events_stages_history || events_stages_history_long || events_stages_summary_by_account_by_event_name |……| events_waits_history_long || events_waits_summary_by_account_by_event_name || events_waits_summary_by_host_by_event_name || events_waits_summary_by_instance || events_waits_summary_by_thread_by_event_name || events_waits_summary_by_user_by_event_name || events_waits_summary_global_by_event_name |+------------------------------------------------------+42 rows in set (0.00 sec)

How to prepare?

• An Introduction to Database Systems (8th Edition)– By C.J. Date

• Any book that teaches relational algebra, or wikipedia, or ..– https://en.wikipedia.org/wiki/Cartesian_product

– https://sh.wikipedia.org/wiki/Основе_теорије_скупова

– https://en.wikipedia.org/wiki/Relational_algebra

– http://poincare.matf.bg.ac.rs/~nenad/rbp/3.Relaciona_algebra_i_racun.pdf

• Be sure to understand all normalization methods from 1st to 5th normal form including Boyce-Codd normal form

Where to learn this art from?

21

What to look for?

• it’s art more than exact science :(

• Find and solve common mistakes– Large IN() set– Subquery with IN and EXIST (convert into JOIN)– Wrong joins

• Check what indexes are used and why

• Check the DB model

• Profile the query, see where the time is being spent

How to actually do it

22

ExamplesUsual crappy sub-select query rewritten to JOIN

23

Original:

DELETE FROM t1 WHERE id NOT IN (SELECT id FROM t1_data);

Query time 3.5s - 5s

Optimized to:

DELETE t1 FROM t1 LEFT OUTER JOIN t1_dataON msg.id=msg_data.id WHERE t1_data.id IS NULL;

Optimized query time below 1.5s! More then 3 times faster!

Examples

• MySQL support non-recursive common table expression

• MySQL support recursive common table extpressions

• MySQL support window functions

• You can usually rewrite a number of slow queries in to one single CTE query that will be up to 1000 times faster then a set of queries to fetch same data. If the original set of queries was generated by ORM this number can go up to 100000 times faster!!

MySQL finally support CTE

24

ExamplesMySQL finally support CTE - sample

25

Simple fast non-cte query:SELECT DATE_FORMAT(r.rental_date, '%Y-%m-01') AS FirstOfMonth, SUM(p.amount) as SalesFROM sakila.payment p INNER JOIN sakila.rental r USING (rental_id) GROUP BY FirstOfMonth;

Now if we want to include the change in sales from previous month we have to join it to itself and… and... we come to this:

SELECT YEAR(cur.FirstOfMonth) AS 'Year', MONTHNAME(cur.FirstOfMonth) AS 'Month', cur.Sales, (cur.Sales - IFNULL(prev.Sales, 0)) AS Delta FROM (SELECT DATE_FORMAT(r.rental_date, '%Y-%m-01') AS FirstOfMonth, SUM(p.amount) as Sales FROM sakila.payment p INNER JOIN sakila.rental r USING (rental_id) GROUP BY FirstOfMonth) cur LEFT OUTER JOIN ( SELECT DATE_FORMAT(r.rental_date, '%Y-%m-01') AS FirstOfMonth, SUM(p.amount) as Sales FROM sakila.payment p INNER JOIN sakila.rental r USING (rental_id) GROUP BY FirstOfMonth) prev ON prev.FirstOfMonth = cur.FirstOfMonth - INTERVAL 1 MONTH ORDER BY cur.FirstOfMonth;

Ugly hah? This is not auto-generated, it’s rather “good” query and pretty “fast” if we are using MySQL version that does not support CTE.

ExamplesMySQL finally support CTE – sample continuing

26

So what are the issues with previous query?

● Hard to read● Duplicates logic● Subquery is executed twice resulting in overhead● If you tomorrow change your business logic there’s a 100% chance you will

make error rewriting this query

ExamplesMySQL finally support CTE – sample continuing

27

CTE version of the same query (nice hah?):

WITH monthly_sales(FirstOfMonth, Sales) AS (SELECT DATE_FORMAT(r.rental_date, '%Y-%m-01') AS FirstOfMonth, SUM(p.amount) as Sales FROM sakila.payment p INNER JOIN sakila.rental r USING (rental_id) GROUP BY FirstOfMonth) SELECT YEAR(cur.FirstOfMonth) AS 'Year', MONTHNAME(cur.FirstOfMonth) AS 'Month', cur.Sales, (cur.Sales - IFNULL(prev.Sales, 0)) AS Delta FROM monthly_sales cur LEFT OUTER JOIN monthly_sales prev ON prev.FirstOfMonth = cur.FirstOfMonth – INTER VAL 1 MONTH ORDER BY cur.FirstOfMonth;

ExamplesMySQL finally support CTE – sample continuing

28

Optimize the query even better using CTE with WINDOW function:

WITH monthly_sales(FirstOfMonth, Sales) AS (SELECT DATE_FORMAT(r.rental_date, '%Y-%m-01') AS FirstOfMonth, SUM(p.amount) as Sales FROM sakila.payment p INNER JOIN sakila.rental r USING (rental_id) GROUP BY FirstOfMonth) SELECT YEAR(FirstOfMonth) AS 'Year', MONTHNAME(FirstOfMonth) AS 'Month', Sales, (Sales - LAG(Sales, 1, 0) OVER (ORDER BY FirstOfMonth)) AS DeltaFROM monthly_sales ORDER BY FirstOfMonth;

Now, this one is not “ideal” as it does not cover “every” circumstance so we have to optimize further

ExamplesMySQL finally support CTE – sample continuing (final query)

29

WITH RECURSIVE months(FirstOfMonth) AS (SELECT '2005-05-01' UNION SELECT FirstOfMonth + INTERVAL 1 MONTH FROM months WHERE FirstOfMonth < '2006-02-01'), monthly_sales(FirstOfMonth, Sales) AS (SELECT DATE_FORMAT(r.rental_date, '%Y-%m-01') AS FirstOfMonth, SUM(p.amount) as Sales FROM sakila.payment p INNER JOIN sakila.rental r USING (rental_id) GROUP BY FirstOfMonth )SELECT YEAR(FirstOfMonth) AS 'Year', MONTHNAME(FirstOfMonth) AS 'Month', IFNULL(Sales, 0) AS Sales,(IFNULL(Sales, 0) - LAG(IFNULL(Sales, 0), 1, 0) OVER (ORDER BY FirstOfMonth)) AS DeltaFROM months LEFT OUTER JOIN monthly_sales USING (FirstOfMonth)ORDER BY FirstOfMonth;

Examples

• To play with the query download sample data from mysql– http://downloads.mysql.com/docs/sakila-db.zip

• CTE query sample courtesy of Jesper Krogh (MySQL Suport team)

• CTE query in the sample is running over a very small dataset and the original query was properly written so the difference is not huge (5-15% speedup only) but similar optimization can make huge difference (usually more then 50% faster query but we seen even 90+% speedup with our customers)

• The speed of the query is not the only thing to worry about!

MySQL finally support CTE (conclusion)

30

Examples10 minutes to 10 seconds (1/4)

31

Original query (ORM generated):

SELECT t1.wid, t2.wname, Date(t1.t_stamp) AS 'dateOnly', t1.rpd + ( t1.runminpd / 60 ) AS 'PDRuntime', Min(t1.t_stamp), l.name, fg.fieldgroupnameFROM arch.lfedata plc LEFT JOIN arch.lfe_wellsites w ON t1.wid = t2.id LEFT JOIN arch.lfe_locations l ON t2.locationid = l.id LEFT JOIN arch.lfe_fieldgroups fg ON l.fieldgroupid = fg.fieldgroupid

Examples10 minutes to 10 seconds (2/4)

32

WHERE t1.t_stamp BETWEEN '2016-06-01 00:00:00' AND '2016-08-31 23:00:00' AND quality_code = 192 AND t2.compreport = 1GROUP BY t1.wid, Date(t1.t_stamp)UNION(SELECT 2, 'NONE', '2000-00-00 00:00:00', 0, '2000-00-00 00:00:00', 'ZZ_NoName', 'NoFieldGroup' ORDER BY name, wname, dateonly DESC);

Examples10 minutes to 10 seconds (3/4)

33

The problem? Query takes 10+ minutes to execute.

● Do they really need all these joins?● Can they make this a 2step query (creating temp table first and then query using it)?● Since customer cannot make this 2step query (they can’t change their app, some ORM

limitation, this has to be single query solution).. we came up with next solution

Examples10 minutes to 10 seconds (4/4)

34

Final optimized query:

SELECT t.wid, t.wname, t. 'dateOnly', plc.rpd +(plc.RunMinPD/60) as 'PDRuntime', l.name, fg.FieldGroupNameFROM (SELECT plc.wid, DATE(plc.t_stamp) as 'dateOnly', MIN(lfedata_ndx) as 'ndx', t2.wname, t2.locationId FROM arch.lfedata plc INNER JOIN arch.lfe_wellsites w ON plc.wid = t2.id WHERE plc.t_stamp BETWEEN '2016-06-01 00:00:00' AND '2016-08-31 23:55:00' AND plc.quality_code = 192 AND t2.compReport = 1 GROUP BY plc.wid, DATE(plc.t_stamp) ) tINNER JOIN arch.lfedata plc ON t.ndx = plc.lfedata_ndxLEFT JOIN arch.lfe_locations l ON t.locationId = l.idLEFT JOIN arch.lfe_fieldgroups fg ON l.fieldGroupId = fg.FieldGroupId;

Oracle University MySQL Training ServicesPrepare Your Organization to Enable Reliable and High-Performance Web-Based Database Applications

“Training and team skill have the most significant impact on overall performance of technology and success of technology projects.” - IDC, 2013

Premier Support customers eligible to save 20% on learning credits. Premier Support customers eligible to save 20% on learning credits.

Benefits Benefits Expert-led training to support your MySQL learning needs

Flexibility to train in the classroom or online

Hands-on experience to gain real world experience

Key skills needed for database administrators and developers

• MySQL for Beginners MySQL for Database Administrators MySQL Performance Tuning MySQL Cluster MySQL and PHP - Developing Dynamic Web Applications MySQL for Developers MySQL Developer Techniques

MySQL Database Administrator MySQL Developer

To find out more about available MySQL Training & Certification offerings, go to: education.oracle.com/mysql

Top Courses for Administrators and Developers Top Courses for Administrators and Developers

Top Certifications Top Certifications

35

TRAINING

Go to https://education.oracle.com/mysqland you will find there professional learning paths to educate yourself about MySQL.

Click on “MySQL Performance Tuning” course to learn how to fine tune your server or “MySQL for Developers” to learn how to optimize your data model and your queries.

HVALA!

Questions?Bogdan Kecman

MySQL Principal Technical Engineerbogdan.kecman@oracle.com

bogdan.kecman@bad-team.net

Developer’s mDayNovi Sad, November 2018