www.vmcd.org

MySQL 5.7 Important Features

InnoDB Read-Only Scalability

Improving the performance for Read-Only and Read-Mostly workloads. Significantly improved how InnoDB handles RO trans-actions

Removed server layer contentions related to Meta Data Locking (MDL) and removed the use of “thread locks” (thd_locks) for InnoDB

In MySQL 5.7.2, there is another optimization where all transactions are treated as a read-only transactions by default, and a transaction is only

assigned an ID and a rollback segment when the transaction attempts to do its first update. This optimization of treating all transactions as

read-only by default has the additional benefit that users do not need to use special syntax or change their applications to take advantage of it.

Reference:

http://dimitrik.free.fr/blog/archives/09-01-2013_09-30-2013.html

http://mysqlserverteam.com/mysql-5-7-over-1m-qps-with-innodb-memcached-plugin/

http://mysqlserverteam.com/transaction-life-cycle-improvements-in-5-7-3/

http://mysqlserverteam.com/mysql-5-7-3-deep-dive-into-1mil-qps-with-innodb-memcached/

www.vmcd.org

InnoDB Read-Write Scalability.

Improved the performance of Read-Write (RW) workloads. We have removed the “index lock contention” in InnoDB. The index lock that was

used to protect the entire index tree structure is now replaced by more fine grained “block locks” in the tree.

InnoDB has two "modes" when updating the Btrees. An optimistic mode and a pessimistic mode. They roughly correspond to in-place changes

and tree modification changes. If InnoDB thinks that an insert/delete will result in a tree modification change it will lock the index in X mode.

This blocks readers too and limits concurrency. The fix was to introduce a new lock mode SX. This reduces the need to lock the entire Btree

during tree modification changes.

Reference:

https://blogs.oracle.com/mysqlinnodb/entry/innodb_5_7_performance_improvements

http://mysqlserverteam.com/mysql-5-7-improves-dml-oriented-workloads/

https://blogs.oracle.com/mysqlinnodb/entry/innodb_5_7_performance_improvements

www.vmcd.org

InnoDB Faster & Parallel Flushing

InnoDB has had some sub-optimal code around traversing the dirty page and LRU list flushing. The problem was that when we release the

buffer pool and associated mutexes during the IO phase we can no longer trust the iterator that is used to scan the aforementioned lists. So, we

restart the scan from the start. The fix was to track whether the iterator was invalidated while the covering mutexes where released. If the

iterator was not invalidated then we can carry on from where we left off. If on the other hand we detect that it was invalidated we go back to

the start and do it the old way. The iterator invalidation should be very rare.

Reducing the number of pages scanned when doing flush list batches, speeding up page flushing

Recently our benchmarks showed that we are scanning excessive number of pages when

doing flush list batches. We fixed it by introducing the concept of Hazard Pointer

which reduced the time complexity of scan from O(n*n) to O(n). This WL builds on

that to extend the same concept to other scans within buffer pool. There are

significant other changes which are made as part of reduced scanning logic.

The time complexity of a scan is reduced from O(n*n) to O(n). We have also implemented parallel flushing by having multiple page_cleaner

threads

Reference:

https://blog.mariadb.org/performance-evaluation-of-mariadb-10-1-and-mysql-5-7-4-labs-tplc/

www.vmcd.org

Speeding up Connection Handling

In 5.7 we have offloaded thread initialization and network initialization to a worker thread and more than doubled MySQL’s ability to do high

frequency connect/disconnect cycles, from 26K to 56K connect/disconnect cycles per second

Reference:

http://mysqlserverteam.com/improving-connectdisconnect-performance/

Bulk Data Load Improvements

Bulk Load for Create Index – Faster index creation

This method of index creation is also known as a “sorted index build”. This enhancement, which improves the efficiency of index creation, also

applies to full-text indexes. A new global configuration option, innodb_fill_factor, defines the percentage of space on each page that is filled

with data during a sorted index build, with the remaining space reserved for future index growth. For more information, see Section 14.13.18,

“Bulk Load for CREATE INDEX”.

www.vmcd.org

Bulk INSERT operations can be sped up by creating the secondary indexes afterwards. In this way, the records will be merge sorted and then

inserted into the InnoDB index B-trees in order, using sequentially allocated pages. This will create a more optimal layout especially for indexes

whose columns are updated rarely

Resize the InnoDB Buffer Pool Online

Change “ innodb_buffer_pool_size“ dynamically.

Reference:

http://mysqlserverteam.com/resizing-buffer-pool-online/

Automatic Truncation of UNOD Logs

Need separate UNDO tablespaces have been configured.

Reference:

http://mysqllover.com/?p=1051

www.vmcd.org

InnoDB Online Alter Table

Onine rename index and enlarge varchar column size

Online Rename Index This work by Marko Mäkelä (WL#6752) and Dmitry Lenev (WL#6555) makes it possible to rename an index as an online operation. Example:

mysql> ALTER TABLE t RENAME INDEX i1 TO i2;

Enlarge VARCHAR column size online (WL#6554) This work by Marko Mäkelä makes it possible to enlarge varchar column sizes as an online operation. This is true as

long as the number of bytes required by the VARCHAR column type (VARCHAR/LONGVARCHAR) remains the same, i.e. from 0 to 255 or from 256 to higher. Example:

mysql> ALTER TABLE t1 ALGORITHM=INPLACE, CHANGE COLUMN c1 c1 VARCHAR(255);

Setting Replication Filters Without Server Restart

Dynamically change Filter behaviors without restart server

Reference:

http://mysqlserverteam.com/mysql-5-7-3-making-mysql-slave-replication-filters-dynamic/

www.vmcd.org

CHANGE MASTER Without Stopping the SQL Thread

In order to add/alter an option using the CANGE MASTER TO command, it was previously necessary to issue a STOP SLAVE command before

the CHANGE MASTER TO command. This work relaxes that constraint.

Reference:

http://mysqlserverteam.com/mysql-5-7-4-change-master-without-stopping-slave-altogether/

http://work.shivjijha.com/2014/04/change-master-without-stopping-slave.html

Improved “IN queries” With Row Value Expressions to Be Executed Using Range Scans

As of 5.7.3 this hole in the net is stitched up. The range optimizer gladly opens the door for queries with IN predicates as long as

The predicate is only IN, not NOT IN.

The row on the predicate’s left-hand side is only indexed column references, in the same index.

The rows contain only constants or come from a previously read table in nested-loops join.

Note that ‘constants’ is a pretty broad category. It consists of pre-evaluated expressions, even some sub-queries, SQL variables and similar

beings

www.vmcd.org

Improving performance when using in predicates

Reference:

http://mysqlserverteam.com/range-access-now-in-an-in-predicate-near-you/

UNION ALL” No Longer Creates a Temporary Table

In 5.7 the optimizer avoids creating a temporary table for the result of UNION ALL queries when there is no need for it, i.e., when there is no

top-level ORDER BY clause

Reference:

http://mysqlserverteam.com/state-of-the-union/

www.vmcd.org

EXPLAIN for Running Queries

This feature is useful if you are running a statement in one session that is taking a long time to complete; using EXPLAIN FOR CONNECTION in

another session may yield useful information about the cause of the delay and thus help you optimize your schema and statements.

JSON EXPLAIN

Make Use of Condition Filtering in the Optimizer

Reference:

http://mysqlserverteam.com/a-new-dimension-to-mysql-query-optimizations-part-1/

http://mysqlserverteam.com/a-new-dimension-to-mysql-query-optimizations-part-2/

www.vmcd.org

Improved ONLY_FULL_GROUP_BY SQL Mode

ONLY_FULL_GROUP_BY SQL mode was enabled by default in 5.75+

In MySQL, one can write GROUP BY queries that reference non-aggregated columns in the SELECT list that are not included in the GROUP BY

clause, even if these columns are not functionally dependent upon the GROUP BY clause. This behaviour conforms to none of the SQL standard's

versions. It is possible to avoid this behaviour by including ONLY_FULL_GROUP_BY in the sql_mode server setting, but it might make more sense

to take advantage of the ability to write only partial GROUP BY clauses.

In a nutshell:

It is completely safe to write partial GROUP BY clauses as long as all non-aggregated columns in the SELECT list are functionally dependent upon

the GROUP BY clause.

A partial GROUP BY list can result in better performance, because it keeps the server from evaluating the entire GROUP BY list.

If one does not want to write partial GROUP BY clauses, consider using MIN or MAX to 'aggregate' the functionally dependent columns in the

SELECT list rather than moving the functionally dependent columns to the GROUP BY clause.

Reference:

http://rpbouman.blogspot.co.uk/2007/05/debunking-group-by-myths.html

http://mysqlserverteam.com/mysql-5-7-only_full_group_by-improved-recognizing-functional-dependencies-enabled-by-default/

www.vmcd.org

Copying File-Per-Table Tablespaces to Another Server

Prior to MySQL 5.7.4, only non-partitioned InnoDB tables are supported. As of MySQL 5.7.4, partitioned InnoDB tables and individual InnoDB

table partitions and subpartitions are also supported.

Reference:

https://dev.mysql.com/doc/refman/5.7/en/tablespace-copying.html

InnoDB Buffer Pool Dump and Load Enhancements

This work improves both the dump and load scenarios. It is now possible to dump only the hottest N% of the pages from each buffer pool. The load

operation is also made less disruptive to user activity because the load now happens in the background while continuing to serve clients

Reference:

www.vmcd.org

http://mysqlserverteam.com/mysql-dumping-and-reloading-the-innodb-buffer-pool/

http://mysqllover.com/?p=1123

Statement Timeouts

MySQL 5.7.4 introduces the ability to set server side execution time limits, specified in milliseconds, for top level read-only SELECT statements.

This feature is introduced as part of WL#6936. It is based on a contribution submitted by Davi Arnaut with Bug#68252. Thank you, Davi!

The statement timeouts work by interrupting the execution of the statement when it takes longer than a specified number of milliseconds to

complete. After the specified number of milliseconds has passed, the server aborts the individual query without affecting the larger transaction

or connection contexts. The following error is then reported to the client when the query is aborted:

1907: Query execution was interrupted, max_statement_time exceeded.

To be clear, the execution time limit specified is really a “soft hint”, because the query interruption may not happen precisely at the specified

execution time limit. There can be a minor delay between the timer expiration and the actual query interruption.

A time limit for any SELECT statement run against a MySQL instance can be set by specifying a timeout value in milliseconds for the GLOBAL

system variable max_statement_time.

www.vmcd.org

Reference:

http://planet.mysql.com/entry/?id=673840

http://mysqlserverteam.com/server-side-select-statement-timeouts/

Multiple User Level Locks

The difference in lock acquisition behavior as of MySQL 5.7.5 can be seen by the following example. Suppose that you execute these

statements:

SELECT GET_LOCK('lock1',10);

SELECT GET_LOCK('lock2',10);

SELECT RELEASE_LOCK('lock2');

SELECT RELEASE_LOCK('lock1');

In MySQL 5.7.5 or later, the second GET_LOCK() acquires a second lock and both RELEASE_LOCK() calls return 1 (success). Before MySQL 5.7.5,

the second GET_LOCK() releases the first lock ('lock1') and the secondRELEASE_LOCK() returns NULL (failure) because there is no 'lock1' to

release.

www.vmcd.org

Waiting for More Transactions to Enter Binlog Group Commit (BGC) Queues

Solve the lag problem in MySQL slave database. We could read Booking’s article for cascade M-S architecture.

binlog-group-commit-sync-delay, binlog-group-commit-sync-no-delay-count were used to control BGC.

Reference:

http://blog.booking.com/evaluating_mysql_parallel_replication_2-slave_group_commit.html

Semi-Sync Replication

Make the Master Wait for More than One Slave to Acknowledge Back

Externalize Transactions Only after ACK is Received

Semi-Sync — Separate ACKs Collector

Reference:

www.vmcd.org

http://my-replication-life.blogspot.sg/2013/12/enforced-semi-synchronous-replication.html

http://my-replication-life.blogspot.sg/2013/12/enforced-semi-synchronous-replication.html

http://my-replication-life.blogspot.sg/2014/03/faster-semisync-replication.html

Multi-Threaded Slaves (MTS)

Intra-Schema Parallel Slave Execution

Reference:

http://geek.rohitkalhans.com/2013/09/enhancedMTS-configuration.html

Ordered Commits (Sequential Consistency)

Ensure that the commits by slave applier threads running in parallel will be done in the same order as they were on the master.

www.vmcd.org

Support Slave Transaction Retries in Multi-Threaded Slave Mode

Transaction retry feature works for both types of mutli-threaded slave(database based and logical clock based). It also works fine when you

force slave to commit transactions in same order as master by setting system variable “slave_preserve_commit_order” to ON.

By default, transaction retry is on for all types of slave. Multi-threaded slave will not break if it just encounters some temporary errors

mentioned above.

TRUNCATE TABLE Statement Becomes Atomic

This work makes the internal InnoDB TRUNCATE TABLE statement atomic by reinitializing the original table-space header with the same space

id and then physically truncating its.ibd file during the truncation of a single table tablespace.

Reference:

http://mysqllover.com/?p=1053

www.vmcd.org

InnoDB Create Tablespace in 5.7

CREATE TABLESPACE `TBS_DADOS_COMPRESS` ADD DATAFILE 'TDS_DC_DATAFILE1.ibd' [FILE_BLOCK_SIZE=4];

CREATE TABLESPACE `TBS_DADOS` ADD DATAFILE 'TDS_D_DATAFILE1.ibd' [FILE_BLOCK_SIZE=8];

CREATE TABLE ITEM_VENDA TABLESPACE=`TBS_DADOS_COMPRESS`;

ALTER TABLE tbl_name TABLESPACE=`TBS_DADOS`

DROP TABLESPACE `TBS_DADOS_COMPRESS`;

Reference:

http://mathiasbrem.com.br/innodb-create-tablespace-mysql-5-7/

InnoDB Native Partitioning

Less handles less use of memory. Easy to access partition tables

Reference:

www.vmcd.org

http://mysqlserverteam.com/innodb-native-partitioning-early-access/

InnoDB Temporary Table Performance

Temporary Tables

Faster create/drop

No redo logging

Separate tablespace for all uncompressed temporary tables

Temporary tables don't require recovery therefore there is no point in saving their definitions in the persistent data dictionary. By putting the all

the temporary tables in a special tablespace that is not redo logged we can reduce a lot of the unnecessary overhead. We do UNDO log the

changes to temporary tables, this is required for rollback to savepoint. However, the UNDO logs for temporary tables are not redo logged and

the UNDO logs for temporary tables also reside in the special temporary tablespace.

Temp tables are only visible within the connection/session in which they were created, and they are bound by the lifetime of the server. We

optimized DML for Temp Tables (WL#6470) by removing unnecessary UNDO and REDO logging, change buffering, and locking. We added an

additional type of UNDO log (WL#6915), one that is not REDO logged and resides in a new separate temp tablespace.

www.vmcd.org

Multi-Source Replication

Nearly MariaDB. MySQL 5.7 starting to support Multi-Source replication.

Generated Columns

There are two kinds of Generated Columns: virtual (default) and stored. Virtual means that the column will be calculated on the fly when a

record is read from a table. Stored means that the column will be calculated when a new record is written in the table, and after that it will be

treated as a regular field. Both types can have NOT NULL restrictions, but only a stored Generated Column can be be a part of an index.

Reference:

http://mysqlserverteam.com/generated-columns-in-mysql-5-7-5/

http://mablomy.blogspot.com/2015/03/auto-generated-columns-in-mysql-57-two.html

http://mechanics.flite.com/blog/2015/04/13/using-mysql-5-dot-7-generated-columns-to-avoid-error-1070/

www.vmcd.org

SYS schema In MySQL

New in MySQL 5.7.7, the MySQL sys schema (originally the ps_helper project) is now included by default within the MySQL server

Reference:

http://mysqlserverteam.com/the-mysql-sys-schema-in-mysql-5-7-7/

Improved alter user syntax support in 5.7

Change Alter user syntax and add more useful commands.

Reference:

http://mysqlblog.fivefarmers.com/2015/04/10/improved-alter-user-syntax-support-in-5-7/

www.vmcd.org

SSL/TLS IN MYSQL 5.7

Security is receiving more attention and MySQL 5.7 aims to be the most secure MySQL Server release ever.

Reference:

http://mysqlblog.fivefarmers.com/2015/04/09/ssltls-in-mysql-5-7/

Secondary Indexes on XML BLOBs

We can create secondary index on this Column with generated column feature

Reference:

http://mablomy.blogspot.com/2015/04/secondary-indexes-on-xml-blobs-in-mysql.html

www.vmcd.org

Emulating roles with expanded proxy user support in 5.7.7

Explain how to use proxy user in MySQL

Reference:

http://mysqlblog.fivefarmers.com/2015/04/08/emulating-roles-with-expanded-proxy-user-support-in-5-7-7/