mysql tech tour 2015 - 5.7 innodb

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |Tuesday, October 20, 2015 Oracle Confidential – Restricted

MySQL 5.7 InnoDB Update

1

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |

Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Tuesday, October 20, 2015 Oracle Confidential – Restricted 2


IntroKey parametersInnoDB 5.6 vs. 5.7InnoDB ‘Always online’GISMySQL Enterprise BackupNews From The Lab!

1

Tuesday, October 20, 2015 Oracle Confidential – Restricted

2

3

4

5

6

7


Intro1



MySQL within OracleMore Investment, More Innovation

Dez 2009 May 2015

5

... 20 Years MySQL… 10 Years InnoDB of Oracle Stewardship … 5 Years MySQL within Oracle

2x Engineering Staff 3x QA Staff 2x Support Staff


InnoDB•Default Storage Engine since MySQL 5.5 (previous as ‘Oracle’)•ACID-compliant transactions, MVCC•Row-level locking•Two phase commit•Efficient indexing•Fast DDL operations•Table compression•Automatic crash recovery•Referential integrity•Online backup

6


Key parameters2



innodb_file_per_table

• .ibd files• MySQL 5.5:Off, MySQL 5.6:On, MySQL 5.7:On• Easier to reclaim space• TRUNCATE for a table is faster• Can monitor table size on the file system• Can store specific tables on different storage devices• (5.6) Transportable tablespaces• What are you using when you’ve updated from older version?

8


innodb_flush_log_at_trx_commit

• Performance vs. Durability

• Redo logs :– innodb_flush_log_at_trx_commit = 1 // best durability (ACID Compliant), default– innodb_flush_log_at_trx_commit = 2 // better performance (not ACID Compliant)– innodb_flush_log_at_trx_commit = 0 // best performance (not ACID Compliant)

• Linux only :– innodb_flush_method = O_DIRECT // skips the FS cache

9

http://dev.mysql.com/doc/refman/5.6/en/innodb-parameters.html

http://dev.mysql.com/doc/refman/5.6/en/innodb-parameters.html

https://dev.mysql.com/doc/refman/5.6/en/server-status-variables.html


innoDB buffer pool

• Global • Caches data and indexes• innodb_buffer_pool_size

– 60 – 80% of the total RAM– Not too large: OS should never swap !– Active data <= innodb_buffer_pool_size <= 0.8 * RAM

• Use read only transactions

10


innodb_log_file_size

• Size of file on disk• Larger the file, the less checkpoint activity• Size for each log file• Max combined log file size

– MySQL 5.5 <4G – MySQL 5.6 & MySQL 5.7 = 512G (!!!)– innodb_log_file_size = 256M / 512M / 1G (‘often’ 32M default)

• Larger the log file longer recovery time➯

11


innodb_log_buffer_size

• Buffer for writing to log files on disk• Large transactions? Write heavy?

– Increasing can help with IO

• Default: 8M (16M)?

12


Tip!

• Especially for HDD, read and write, use dedicated partitions• Separate :

– datadir– innodb_data_file_path– innodb_undo_directory– innodb_log_group_home_dir– log-bin – tmpdir– backups

Random, SSD

Sequential, spinning

Random, SSD, tmpfs(NFS, ButterFS, ZFS …)


InnoDB 5.6 vs. 5.73



Scaling with InnoDB, 5.6 (up to 48 CPUs*) vs. 5.7 (beyond 72 CPUs*) * CPU Cores-HT

18 36 720

100,000

200,000

300,000

400,000

500,000

600,000

700,000

MySQL 5.7: Sysbench – OLTP Read Write

MySQL 5.7

MySQL 5.6

MySQL 5.5

CPU Cores – HT

Que

ries p

er S

econ

d

Intel(R) Xeon(R) CPU E7-8890 v34 sockets x 18 cores-HT (144 CPU threads)2.5 Ghz, 512GB RAMLinux kernel 3.16

MySQL 5.7 Sysbench Benchmark


Key-value access to InnoDB- Bypasses SQL Parsing

SQL for rich queries, JOINs, FKs, etc.

Implemented via:- Memcached plug-in to mysqld - Memcached mapped to native

InnoDB API- Use existing Memcached clients- Shared process for ultra-low latency

MySQL 5.6 – InnoDB, NoSQL With MemcachedMore Flexibility - Memcached API


Improved: MySQL 5.7: InnoDB, NoSQL With MemcachedMySQL 5.7 is 6x faster than MySQL 5.6

8 16 32 64 128 256 512 1,0240

200,000

400,000

600,000

800,000

1,000,000

1,200,000

MySQL 5.7 vs 5.6 - InnoDB & Memcached

MySQL 5.7

MySQL 5.6

Connections

Que

ries p

er S

econ

d

1 Million QPS

18

Intel(R) Xeon(R) CPU E7-4860 x86_644 sockets x 10 cores-HT (80 CPU threads)2.3 GHz, 512 GB RAMOracle Linux 6.5


InnoDB Full-Text Index support• MySQL 5.6

– InnoDB Full-Text Index support• Fully ACID compliant, MVCC search• With performance improvements over MyISAM• Easily customizable stop-word lists

• MySQL 5.7– (Significant) better Performance – Pluggable Full-Text Parser support– CJK Support

• N-gram parser for Chinese, Japanese, and Korean• MeCab parser for Japanese

19


What is it?• Search entire documents

– Character based fields • VARCHAR, TEXT, BLOB

• For a search string – Combinations of words– Phrases: “specific string to match”– Wildcards: * – Requirements: +, -, ~– Expressions: (…)– Relevancy weight characters: <, >

20


Example

21


InnoDB ‘Always online’4



MySQL 5.7: InnoDB – Always Online

• Resize the InnoDB Buffer Pool online – Allows DBAs to tune the buffer size without any downtime– Adapt in real-time to changes in database usage patterns

• Separate UNDO tablespace– With automatic online truncation

• Additional Online ALTER TABLE support– Enlarge / instant change VARCHAR, Rename / instant Index

• Dynamic configuration– Making existing settings dynamically configurable– As a design principle for new features & settings


MySQL 5.7: InnoDB – Example

• Resize the InnoDB Buffer Pool online – Allows DBAs to tune the buffer size without any downtime– Increase or decrease online, Operation is

performed in chunks (default 128MB)– Assign memory to MySQL/Apps during

runtime


• Native Partitioning– Eliminates previous limitations– Eliminates resource usage problems– Transportable tablespace support

• Native Spatial Indexes• Transparent page compression • Support for 32K and 64K pages

– Use with transparent page compression for very high compression ratios

• General TABLESPACE support – Store multiple tables in user defined shared

tablespaces

• Improved support for cache preloading– Load your hottest data loaded at startup

• Configurable fill-factor – Allows for improvements in storage footprint

• Improved bulk-data load performance

MySQL 5.7: InnoDB Improvements


MySQL 5.7: InnoDB Improvements

• Enhanced FusionIO integration– Doublewrite buffer disabled when NVMFS/DirectFS detected

• Parallel “dirty page” flushing– Higher throughput, performance, and scalability

• Partitions support for Transportable Tablespaces (TTS)– TTS support for individual partitions

• Improved crash recovery to ensure data safety


InnoDB File format– Antelope MySQL 5.1, 5.5, 5.6

vs. Barracuda MySQL 5.7• RAW performance• Lower memory and disk space footprint• Great in combination with SSD

– SHOW TABLE STATUS• Antelope Compact or Redundant• Barracuda Compressed or Dynamic

27


MySQL 5.7: InnoDB Compression

• Transparent Page Level Compression– Happens transparently in background threads– Managed entirely within the IO layer– Uses sparse file and "hole punching" support in OS kernels and File Systems

• Reduces IO – Improves MySQL performance– Improves storage efficiency– Reduces write cycles, thus increasing SSD lifespan

• Applies to all InnoDB data, including the system tablespace and UNDO logs


MySQL 5.7: InnoDB Temp Tables

• New separate tablespace for temporary tables– Improved CREATE/DROP performance– DDL changes are transient, which eliminates some disk IO

• Optimize DML operations– No REDO logging, no change buffering, less locking

• New intrinsic temporary tables– Specialized temporary tables with tailored ACID/MVCC semantics– Light weight and ultra-fast, great for intermediate query execution operations

• InnoDB as additional storage engine for temp tables– Switch optimizer from MyISAM to InnoDB (faster) for internal temp tables


GIS5



MySQL 5.7 : GIS Reborn• Dev scrapped much of the old algorithms and related code …

– Replacing it with the FOSS Boost.Geometry library (1.58)

• …and rounded out our OGC spatial function support– Standardized the namespace

• Added native GEOMETRY data type support to InnoDB– We then added SPATIAL INDEX support to InnoDB (using R-tree)

• Added Geohash and GeoJSON support• Added helper functions: ST_Distance_Sphere(), ST_MakeEnvelope(), ST_IsValid(), ST_Validate(), ST_Simplify, …

31


Spatial Indexes for InnoDB• R-tree based

– Full transactional support– Predicate locking to prevent phantoms– Records contain minimum bounding box

• Small and compact

– Currently only supports 2D data• We would like to add 3D support in the future

– Supports historical spatial index DDL syntax

32


Additional Features• GeoHash

– B-tree indexes on the generated hash values– Quick lookups for exact matches– Not very accurate for proximity searches

• GeoJSON• Additional functions

– ST_IsValid(), ST_Simplify(), ST_Buffer() …– ST_Distance_Sphere()

• Limited SRID support– Laying the groundwork for CRS support

33

{ "type": "Feature", "geometry": { "type": "Point", "coordinates": [125.6, 10.1] }, "properties": { "name": "Dinagat Islands" }}

GeoJSON Example


• Replaced custom code– For spatial calculations– For spatial analysis

• Provides OGC compliance– With improved performance

• Boost.Geometry contains– Field and domain experts – Bustling and robust communit

MySQL 5.7: GIS - Integrating Boost.Geometry

34


MySQL Enterprise Backup6


MySQL Enterprise Backup now writes tape information


MySQL Enterprise Backup 4.0

• Online, non-locking backup and recovery – Complete MySQL instance backup (data and config)– Partial backup and restore

• Direct Cloud storage backups via Swift API• Incremental backups & Point-in-time recovery• Advanced compressed and encryption• InnoDB tables with page sizes of 32 and 64K can be backed up and restored.• Optimistic backups• Support for MySQL 5.7

– General Tablespaces

New & Improved

36


• MySQL Enterprise Backup images to tape (like RMN)• Via SBT interface (standard for MMS)• Encryption, policy-based management, vault rotation• Common management, SOP for Oracle, MySQL backups• Improved SBT Backups to MMS Systems • Includes Backup/tape information

Oracle Integrations: Secure Backup


News From The Lab!7



MySQL Group Replication Plugin

Tuesday, October 20, 2015

M M M M M

Replication Group

• Multi-master update everywhere• Automatic group membership

management and failure detection.• No need for server fail-over.• Automatic reconfiguration.• No single point of failure.• Shared-nothing state machine

replication.• InnoDB compliant. Off-the-shelf

hardware friendly.

Oracle Confidential – Restricted 39


MySQL Group Replication Plugin• Great technology for deployments where elasticity is a requirement, such

as cloud based infrastructures.– Integrated with the server core through a well defined API.– It is integrated with GTIDs, row based replication.– It is integrated with performance schema tables.– It is elastic and self-healing: adding or removing servers does not need human

intervention.



• Strong development cycles and continuous community engagement through regular and frequent releases on labs.mysql.com.

MySQL Group Replication


2014-Sep-29Labs release: 0.2.0

2015-Apr-06Labs release: 0.3.0

2014-Aug-06Labs release: 0.4.0

2015-Sep-14Labs release: 0.5.0

Introduces standalone plugin,with its own release cycle!


Dependable and Scalable MySQL


Global Data Shard 1 Shard 2

MySQL Fabric Controller

SQL Queries

Server/Shard State & Mapping

Global Group HA Group

Coordinationand Control

HA Group

Group Replication cluster



MySQLRouter


Dependable and Scalable MySQL Setups


MApp

M M

MySQL Fabric (Orchestration)

MApp M M

Simple Shard Mapping, State and Extra metadata.

Control, Coordinate, Provision

...

Monitoring (MEM)

MySQL Router Group Replication – Shard 1

Group Replication – Shard NC, PHP, Perl, ...

43

MySQL Router

mysql tech tour 2015 - 5.7 innodb

Technology