mysql explain explained-norvald h. ryeng

MySQL EXPLAIN Explained

Norvald H. Ryeng

Software Engineer

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 2

議程

傳統的, 結構化的, 和視覺化的EXPLAIN

用EXPLAIN來優化查詢

EXPLAIN和子查詢

對INSERT/UPDATE/DELETE的EXPLAIN

Optimizer trace


查詢優化器優化器做啥?

Optimizer

SELECT a, b

FROM t1, t2, t3

WHERE t1.a = t2.b

AND t2.b = t3.c

AND t2.d > 20

AND t2.d < 30;

Metadata:

- Index information

- Uniqueness

- Nullability

Statistics:

- Table sizes

- Cardinality

JOIN

JOIN

Table

scan

Range

scan

Ref

access

t2 t3

t1

Query

Query Execution

Plan (QEP)


MySQL EXPLAIN 不同的佈局

+----+-------------+------------+-------+---------------+-------------+---------+-------------------------+---------+…

| id | select_type | table | type | possible_keys | key | key_len | ref | rows |…

+----+-------------+------------+-------+---------------+-------------+---------+-------------------------+---------+…

| 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 1050000 |…

| 2 | DERIVED | customer | index | PRIMARY | PRIMARY | 4 | NULL | 150000 |…

| 2 | DERIVED | orders | ref | i_o_custkey | i_o_custkey | 5 | dbt3.customer.c_custkey | 7 |…

+----+-------------+------------+-------+---------------+-------------+---------+-------------------------+---------+…

{

"query_block": {

"select_id": 1,

"cost_info": {

"query_cost": "262510.00"

},

"ordering_operation": {

"using_filesort": true,

"grouping_operation": {

"using_temporary_table": true,

"using_filesort": false,

"table": {

"table_name": "c_orders",

"access_type": "ALL",

"rows_examined_per_scan": 1050000,

"rows_produced_per_join": 1050000,

"filtered": 100,

"cost_info": {

"read_cost": "52510.00",

"eval_cost": "210000.00",

"prefix_cost": "262510.00",

"data_read_per_join": "24M”

...


傳統的EXPLAIN

film

film_

category

JOIN

category

JOIN

EXPLAIN為在SELECT指令中的每個表傳回一列訊息這個表可能是真的表,也可以是洐生的,臨時表,子查詢,或一個union的結果


了解EXPLAIN

EXPLAIN SELECT l_returnflag, l_linestatus, SUM(l_quantity) AS sum_qty

FROM lineitem WHERE l_shipdate <= DATE_SUB('1998-12-01', INTERVAL '118' DAY)

GROUP BY l_returnflag, l_linestatus ORDER BY l_returnflag, l_linestatus\G

id: 1

select_type: SIMPLE

table: lineitem

type: ALL

possible_keys: i_l_shipdate

key: NULL

key_len: NULL

ref: NULL

rows: 6001215

Extra: Using where; Using temporary; Using filesort

Full table scan

Considered indexes

Number of rows to be read

Chosen index (none)


傳統的EXPLAIN的結果可能會很複雜

id select type table type possible

keys

key key len ref rows extra

1 PRIMARY cust index cust_id,

cust_name

cust_name 40 const 10 Using where; Start

materialize; Scan

1 PRIMARY orders ref order_id order_id 8 ordid 32 Using where; End

materialize; Using join

buffer (Block Nested

Loop)

1 PRIMARY <derived2> ref <auto_key0> <auto_key0> 23 vc1 100 Using index

2 DERIVED flights ALL NULL NULL NULL NULL 10000 Using where; Start

materialize; Scan

2 DERIVED storage eq_ref PRIMARY PRIMARY 8 fl_store 1 Using where; End

materialize; Using join

buffer (Block Nested

Loop)

4 SUBQUERY buzz range buzzez buzzez 11 NULL 42 Using index; Using

where

6 SUBQUERY shortage index shrt_idx shrt_idx 100 NULL 243 Using index; Using

where


傳統的EXPLAIN: 缺點

複雜的查詢很難讀得懂

複雜的查詢所產生的結果不清楚/不明顯

– 可能要用很長的時間了解發生了什麼事

能為查詢計畫提供的資訊有限

– 條件是如何劃分? 子查詢是何時評估?

非常難擴充

– 顯示用到了一個功能僅限於 “Using <something>”


EXPLAIN

{ "query_block": {

"select_id": 1,

"ordering_operation": {

"using_filesort": false,

"grouping_operation": {

"using_temporary_table": true,

"using_filesort": true,

"table": {

"table_name": "lineitem“,

"access_type": "ALL“,

"possible_keys": [

"i_l_shipdate”

],

"rows": 2829575,

"filtered": 50,

"attached_condition":

"(`dbt3`.`lineitem`.`l_shipDATE` <=

<cache>(('1998-12-01' - interval '118' day)))”

} /* table */

} /* grouping_operation */

} /*ordering_operation */

} /*query_block */ }

結構化 EXPLAIN

EXPLAIN FORMAT=JSON

SELECT l_returnflag, l_linestatus,

SUM(l_quantity) AS sum_qty

FROM lineitem

WHERE l_shipdate <=

DATE_SUB('1998-12-01',

INTERVAL '118' DAY)

GROUP BY l_returnflag, l_linestatus

ORDER BY l_returnflag, l_linestatus;

MySQL 5.6的新功能


圖示EXPLAIN MySQL Workbench

Cost estimate ( MySQL 5.7)


圖示EXPLAIN 滑鼠掃過就能看到詳情


單一個表的查詢

SELECT * FROM customer WHERE c_custkey = 570887;

主鍵查找

id select

type table type

possible

keys key key len ref rows extra

1 SIMPLE customer const PRIMARY PRIMARY 4 const 1 NULL


單一個表的查詢次要鍵查找

SELECT * FROM orders WHERE o_orderdate = '1992-09-12';

id

select

type table type

possible

keys key

key

len ref rows extra

1 SIMPLE orders ref i_o_orderdate i_o_orderdate 4 const 6271 NULL


單一個表的查詢索引區域掃瞄

SELECT * FROM orders

WHERE o_orderdate BETWEEN '1997-05-01' AND '1997-05-31‘;

id select

type table type

possible

keys key

key

len ref rows extra

1 SIMPLE orders range i_o_orderdate i_o_orderdate 4 NULL 376352 Using index

condition


範圍優化器: 案例


WHERE YEAR(o_orderdate) = 1997 AND MONTH(o_orderdate) = 5 AND

o_clerk LIKE '%01866';

為何要做表掃?

id select

type table type

possible

keys key key len ref rows extra

1 SIMPLE orders ALL NULL NULL NULL NULL 15000000 Using where

mysql> SELECT * FROM orders WHERE year(o_orderdate) = 1997 AND MONTH(…

...

15 rows in set (8.91 sec)

未考量索引




WHERE o_orderdate BETWEEN '1997-05-01' AND '1997-05-31'

AND o_clerk LIKE '%01866';

重寫查詢以避免把加了索引的欄位放入function的參數中

id select

type table type

possible

keys key

key

len ref rows extra

1 SIMPLE orders range i_o_orderdate i_o_orderdate 4 NULL 376352 Using index

condition;

Using where

mysql> SELECT * FROM orders WHERE o_orderdate BETWEEN '1997-05-01' AND …

...




CREATE INDEX i_o_clerk ON orders(o_clerk);



AND o_clerk LIKE '%01866';

加上另一個索引

id select

type table type

possible

keys key

key

len ref rows extra

1 SIMPLE orders range i_o_orderdate i_o_orderdate 4 NULL 376352

Using index

condition;

Using where

New index not considered





AND o_clerk LIKE 'Clerk#000001866';

再次重寫查詢

id select

type table type possible keys key

key

len ref rows extra

1 SIMPLE orders range i_o_orderdate,

i_o_clerk

i_o_clerk 16 NULL 1504 Using index

condition;

Using where


...




CREATE INDEX i_o_clerk_date

ON orders(o_clerk, o_orderdate);




建立多欄位索引

id select


key

len ref rows extra


i_o_clerk,

i_o_clerk_date

i_o_clerk_date 20 NULL 14 Using index

condition


...



多欄位索引

CREATE INDEX i_o_clerk_status_date

ON orders(o_clerk, o_orderstatus, o_orderdate);




key_len對應使用索引欄位

id select


key

len ref rows extra


i_o_clerk_status

_date

i_o_clerk_status

_date

16 NULL 1504 Using index

condition


Multi-Column index

CREATE INDEX i_o_clerk_date_status

ON orders(o_clerk, o_orderdate, o_orderstatus);




key_len對應使用索引欄位 - 續

id select


key

len ref rows extra


i_o_clerk_status

_date,i_o_clerk_

date_status

i_o_clerk_date_

status

20 NULL 15 Using index

condition


使用加了索引欄位結構化的 EXPLAIN

EXPLAIN

{ "query_block": {

"select_id": 1,

"table": {

"table_name": "orders",

"access_type": "range",

"possible_keys": [

"i_o_orderdate",

"i_o_clerk_status_date"

],

"key": "i_o_clerk_status_date",

"used_key_parts": [

"o_clerk"

], "key_length": "16",

"rows": 1504,

"filtered": 100,

"index_condition": "((`dbt3`.òrders`.ò_orderDATE` between

'1997-05-01' and '1997-05-31') and (`dbt3`.òrders`.ò_clerk` like

'Clerk#000001866'))"

} } }

EXPLAIN

{ "query_block": {

"select_id": 1,

"table": {

"table_name": "orders",


"possible_keys": [

"i_o_orderdate",

"i_o_clerk_status_date",

"i_o_clerk_date_status"

],

"key": "i_o_clerk_date_status",

"used_key_parts": [

"o_clerk",

"o_orderDATE"

], "key_length": "20",

"rows": 15,

"filtered": 100,

"index_condition": "((`dbt3`.òrders`.ò_orderDATE` between

'1997-05-01' and '1997-05-31') and (`dbt3`.òrders`.ò_clerk` like

'Clerk#000001866'))"

} } }


索引合併

SELECT COUNT(*) FROM lineitem

WHERE l_shipdate = '1997-05-01'

AND l_commitdate = '1997-05-01';

id select


key

len ref rows extra

1 SIMPLE lineitem index_

merge

i_l_shipdate,

i_l_commitdate

i_l_shipdate,

i_l_commitdate

4,4 NULL 43 Using intersect

(i_l_shipdate,

i_l_commitdate);

Using where;

Using index


Join 查詢

SELECT *

FROM orders JOIN customer

ON c_custkey = o_custkey

WHERE o_orderdate = '1992-09-12';

id

select


key

len ref rows extra

1 SIMPLE orders ref i_o_orderdate,

i_o_custkey

i_o_orderdate 4 const 6271 Using

where

1 SIMPLE customer eq_ref PRIMARY PRIMARY 4 dbt3.orders.

o_custkey

1 NULL


Join查詢 (續)

SELECT *

FROM orders JOIN customer

ON c_custkey = o_custkey

WHERE c_acctbal < -1000;

id

select

type table type

possible

keys key

key

len ref rows extra

1 SIMPLE customer ALL PRIMARY NULL NULL NULL 1500000 Using

where

1 SIMPLE orders ref i_o_custkey i_o_custkey 5 dbt3.customer

.c_custkey

7 NULL


EXPLAIN

{

"query_block": {

"select_id": 1,

"nested_loop": [

{

"table": {

"table_name": "customer“,

"access_type": "ALL“,

"possible_keys": [

"PRIMARY”

],

"rows": 147973,

"filtered": 100,

"attached_condition":

"(`dbt3`.`customer`.`c_acctbal` < <cache>(-(1000)))”

} /* table */

},

{

"table": {

"table_name": "orders“,

"access_type": "ref“,

"possible_keys": [

"i_o_custkey”

],

"key": "i_o_custkey“,

"used_key_parts": [

"o_custkey”

],

"key_length": "5“,

"ref": [

"dbt3.customer.c_custkey

],

"rows": 7,

"filtered": 100

} /* table */

}

] /* nested_loop */

} /* query_block */

}

結構化EXPLAIN: Join 查詢


EXPLAIN: 子查詢

SELECT SUM(l_extendedprice) / 7.0 AS avg_yearly FROM lineitem JOIN part ON l_partkey = p_partkey WHERE p_brand = 'Brand#33‘ AND p_container = 'LG CAN' AND l_quantity <

(SELECT 0.2 * AVG(l_quantity) FROM lineitem WHERE l_partkey = p_partkey);

依附的子查詢 (DBT-3 Query 17)

id select


key

len ref rows extra

1 PRIMARY

part ALL PRIMARY 2000000 Using where

1 PRIMARY lineitem ref i_l_suppkey_partkey i_l_suppkey_partkey 5 dbt3.part.

p_partkey

14 Using where

2 DEPENDENT

SUBQUERY

lineitem ref i_l_suppkey_partkey i_l_suppkey_partkey 5 dbt3.part.

p_partkey

14


圖示 EXPLAIN DBT-3 Query 17

SELECT SUM(l_extendedprice) / 7.0 AS avg_yearly

FROM lineitem JOIN part ON l_partkey = p_partkey

WHERE p_brand = 'Brand#33‘ AND p_container = 'LG CAN'

AND l_quantity <

(SELECT 0.2 * AVG(l_quantity) FROM lineitem

WHERE l_partkey = p_partkey);


依附的子查詢 DBT-3 Query 11

SELECT ps_partkey, SUM(ps_supplycost * ps_availqty) AS value FROM partsupp JOIN supplier ON ps_suppkey = s_suppkey JOIN nation ON s_nationkey = n_nationkey WHERE n_name = 'KENYA' GROUP BY ps_partkey HAVING SUM(ps_supplycost * ps_availqty) >

(SELECT SUM(ps_supplycost * ps_availqty) * 0.0001000000 FROM partsupp JOIN supplier ON ps_suppkey = s_suppkey JOIN nation ON s_nationkey = n_nationkey WHERE n_name = 'KENYA‘)

ORDER BY value DESC;


EXPLAIN: 獨立的子查詢 DBT-3 Query 11

id select

type table type

possible

keys key

key

len ref rows extra

1 PRIMARY nation ALL PRIMARY 25 Using where; Using

temporary; Using filesort

1 PRIMARY supplier ref PRIMARY,

i_s_nationkey

i_s_nationkey 5 dbt3.nation.

n_nationkey

2000 Using index

1 PRIMARY partsupp ref PRIMARY,

i_ps_partkey,

i_ps_suppkey

i_ps_suppkey 4 dbt3.supplier.

s_suppkey

40

2 SUBQUERY nation ALL PRIMARY 25 Using where

2 SUBQUERY supplier ref PRIMARY,

i_s_nationkey


n_nationkey

2000 Using index

2 SUBQUERY partsupp ref i_ps_suppkey i_ps_suppkey 4 dbt3.supplier.

s_suppkey

40


Semi-join

SELECT s_name, s_address FROM supplier JOIN nation ON s_nationkey = n_nationkey WHERE s_suppkey IN

(SELECT DISTINCT (ps_suppkey) FROM partsupp JOIN part ON ps_partkey=p_partkey WHERE p_name like 'grey%’ AND ps_availqty >

(SELECT 0.5 * SUM(l_quantity) FROM lineitem WHERE l_partkey = ps_partkey AND l_suppkey = ps_suppkey AND l_shipdate >= '1995-01-01 AND l_shipdate < date_add( '1995-01-01’, interval '1' year)))

AND n_name = 'SAUDI ARABIA‘ ORDER BY s_name;

DBT-3 Query 20


圖示EXPLAIN: Semi-join DBT-3 Query 20


EXPLAIN: Semi-join DBT-3 Query 20

id select type table type possible keys key key

len ref rows extra

1 PRIMARY nation ALL PRIMARY 25 Using where;

Using temporary;

Using filesort

1 PRIMARY supplier ref PRIMARY,

i_s_nationkey


n_nationkey

2000

1 PRIMARY partsupp ref PRIMARY,

i_ps_partkey,

i_ps_suppkey

i_ps_suppkey 4 dbt3.supplier.

s_suppkey

40 Using where

1 PRIMARY part eq_ref PRIMARY PRIMARY 4 dbt3.partsupp.

ps_partkey

1 Using where;

FirstMatch

(supplier)

3 DEPENDENT

SUBQUERY

lineitem ref i_l_shipdate,

i_l_partkey_suppkey,

i_l_partkey,

i_l_suppkey

i_l_partkey_suppkey 10 dbt3.partsupp.

ps_partkey,

dbt3.partsupp.

ps_suppkey

3 Using where


洐生性表 (自FROM子句的查詢)

SELECT c_count, COUNT(*) AS custdist FROM (SELECT c_custkey, COUNT(o_orderkey) AS c_count FROM customer LEFT JOIN orders ON c_custkey = o_custkey AND o_comment NOT LIKE '%express%requests%’ GROUP BY c_custkey ) AS c_orders GROUP BY c_count ORDER BY custdist DESC, c_count DESC;

DBT-3 Query 13


EXPLAIN: 衍生的性表 DBT-3 Query 13

id select

type table type

possible

keys key

key

len ref rows extra

1 PRIMARY <derived2> ALL 10500000 Using temporary;

Using filesort

2 DERIVED customer index PRIMARY,

i_c_nationkey

PRIMARY 4 1500000 Using index

2 DERIVED orders ref i_o_custkey i_o_custkey 5 dbt3.customer.

c_custkey

7 Using where

SELECT c_count, COUNT(*) AS custdist

FROM (SELECT c_custkey, COUNT(o_orderkey) AS c_count

FROM customer LEFT JOIN orders ON c_custkey = o_custkey

AND o_comment NOT LIKE '%express%requests%’

GROUP BY c_custkey

) AS c_orders

GROUP BY c_count

ORDER BY custdist DESC, c_count DESC;


EXPLAINing 非Select查詢

INSERT/REPLACE

INSERT/REPLACE … SELECT FROM

UPDATE (Single- and multi-table)

DELETE (Single- and multi-table)


EXPLAINing非Select查詢

id select type table type possible keys key key len ref rows extra

1 SIMPLE t1 range a,a_2 a 5 NULL 2 Using where;

Using temporary

EXPLAIN UPDATE t1 SET a = 10 WHERE a < 10;


1 SIMPLE t1 range a,a_2 a 5 NULL 2 Using where

EXPLAIN DELETE FROM t1 WHERE a < 10;


1 SIMPLE NULL NULL NULL NULL NULL NULL NULL No tables used

EXPLAIN INSERT INTO t1 VALUES (13,'m'), (14,'n');


EXPLAIN

"query_block": {

"select_id": 1,

"table": {

"update": true,

"table_name": "t1",


"possible_keys": [ "a", "a_2" ],

"key": "a", "key_length": "5",

"rows": 2, "filtered": 100,

"using_temporary_table": "for update",

"attached_condition": "(`test`.`t1`.`a` < 10)"

} /* table */

} /* query_block */


EXPLAIN FORMAT=JSON

UPDATE t1

SET a = 10

WHERE a < 10;

結構化的EXPLAIN



UPDATE part SET p_retailprice = p_retailprice*1.10 WHERE p_partkey IN (SELECT ps_partkey FROM partsupp JOIN supplier ON ps_suppkey = s_suppkey WHERE s_nationkey = 4);

Plans may differ from similar SELECT query SELECT * FROM part WHERE p_partkey IN (SELECT ps_partkey FROM partsupp JOIN supplier ON ps_suppkey = s_suppkey WHERE s_nationkey = 4);


優化器Traces: 查詢計畫偵錯

EXPLAIN 顯示優化器所選擇的計畫

TRACE 顯示為何會選擇這個計畫:

– 可選擇的計畫

– 預估的成本

– 所下的決定

JSON格式


優化器Trace:範例

mysql> SET optimizer_trace= “enabled=on“,

end_markers_in_json=on;

mysql> SELECT * FROM t1,t2 WHERE f1=1 AND f1=f2 AND f2>0;

mysql> SELECT trace FROM

information_schema.OPTIMIZER_TRACE;

mysql> SET optimizer_trace="enabled=off";

QUERY SELECT * FROM t1,t2 WHERE f1=1 AND f1=f2 AND f2>0;

TRACE “steps”: [ { "join_preparation": { "select#": 1,… } … } …]

MISSING_BYTES_BEYOND_MAX_MEM_SIZE 0

INSUFFICIENT_PRIVILEGES 0


優化器Trace:範例

join_optimization / row_estimation / table : t1, range_analysis

"table_scan": {

"rows": 5,

"cost": 5.1085

} /* table_scan */,

"potential_range_indices": [ {

"index": "f1", "usable": true,

"key_parts": [ "f1“ ] /* key_parts */ }

] /* potential_range_indices */,

"best_covering_index_scan": {

"index": "f1", "cost": 2.093, "chosen": true

} /* best_covering_index_scan */,

"group_index_range": {

"chosen": false, "cause": "not_single_table"

} /* group_index_range */,

"analyzing_range_alternatives": {

"range_scan_alternatives": [ {

"index": "f1",

"ranges": [ "1 <= f1 <= 1“ ] /* ranges */,

"index_dives_for_eq_ranges": true,

"rows": 2, "cost": 3.41,

"chosen": false,

"cause": "cost"

}

] /* range_scan_alternatives */,

"analyzing_roworder_intersect": {

"usable": false, "cause": "too_few_roworder_scans"

} /* analyzing_roworder_intersect */

} /* analyzing_range_alternatives */


優化器Trace: 範例

join_optimization / considered_execution_plan

"plan_prefix": [ ] /* plan_prefix */,

"table": "`t2`",

"best_access_path": {

"considered_access_paths": [ {

"access_type": "scan", "rows": 3,

"cost": 2.6051, "chosen": true

} ] /* considered_access_paths */

} /* best_access_path */,

"cost_for_plan": 2.6051,

"rows_for_plan": 3,

"rest_of_plan": [ {

… next iteration … =>

} ] /* rest_of_plan */

"plan_prefix": [ "`t2`“ ] /* plan_prefix */,

"table": "`t1`",

"best_access_path": {

"considered_access_paths": [ {

"access_type": "ref", "index": "f1",

"rows": 2, "cost": 3.4698, "chosen": true

}, {

"access_type": "scan", "using_join_cache": true,

"rows": 2, "cost": 3.8087,

"chosen": true

} ] } /* best_access_path */,

"cost_for_plan": 6.4138,

"rows_for_plan": 6,

"chosen": true


優化器Trace: 控制

optimizer-trace

optimizer-trace-features

– greedy_search

– range_optimizer

– dynamic_range

– repeated_subselect

optimizer-trace-limit

optimizer-trace-offset

optimizer-trace-max-mem-size

end-markers-in-json


優化器Trace: 探索

mysql> pager less

mysql> SELECT TRACE INTO DUMPFILE <filename> FROM

INFORMATION_SCHEMA.OPTIMIZER_TRACE;

JSONView for Firefox

Pretty JSON for Chrome

A registry patch for IE


MySQL 5.7 Reference Manual

– http://dev.mysql.com/doc/refman/5.7/en/index.html

Optimizer team blog:

– http://mysqloptimizerteam.blogspot.com/

MySQL Server Blog:

– http://mysqlserverteam.com

進一步的資訊

http://mysqloptimizerteam.blogspot.com/

http://mysqloptimizerteam.blogspot.com/

http://mysqlserverteam.com/


The preceding is intended to outline our general product direction. It is

intended for information purposes only, and may not be incorporated

into any contract. It is not a commitment to deliver any material, code,

or functionality, and should not be relied upon in making purchasing

decisions. The development, release, and timing of any features or

functionality described for Oracle’s products remains at the sole

discretion of Oracle.


Q&A

mysql explain explained-norvald h. ryeng

Technology

oracle andor

type table

null null null null

estimation table

cand t2

band t2

index information uniqueness

keys key key