mysql for oracle dbas
TRANSCRIPT
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL for the Oracle DBA
Mario Beck MySQL Sales Consulting Manager EMEA LVM GBU June 10th, 2015
Oracle Confidential – Internal/Restricted/Highly Restricted
How to run MySQL professionally
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Oracle Confidential – Internal/Restricted/Highly Restricted 2
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Adoption in Your Organization
OEM product Custom
Application
Vendor Requirement
Ad Hoc Need
Typical Early Adoption Model
• Support by ‘Web’ • Backup by ‘mysqldump’ • Monitoring by ‘script’ • Tuning by ‘log’ • Hardening by ‘password’
Usual early adoption model and mindset. Not inherently bad at the beginning, but conditions evolve…
3
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Power Curve of MySQL Deployments
Initial Deployment
‘Nice to Have’ Era
Business Value Discovered
Business Critical
Complete Operationalization
Typical Pattern
Time
Cri
tica
lity
• Not unusual to go from initial deployment to business critical in well under 12 months
• The implication is missing a budget cycle to pay for now ‘must have’ • The key is being ready for when ‘Business Critical’ hits
4
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Complete Solutions
• Best of breed components at every level of the stack
• Complete: Meets most customer requirements
• Oracle Database & MySQL Complementary
On-Premises and in the Cloud
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Powers The Web
Over 500 million Tweets/day. 143,200 Tweets/sec in Aug 2013
”Many petabytes” of data. 11.2 Million Row changes & 2.5 billion rows read /sec handled in MySQL
6 billion hours of video watched each month. 100 hours of video uploaded every minute
Globally-distributed database with 100 terabytes of user-related data based on MySQL Cluster
6
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Cloud
Web & Enterprise OEM & ISVs
Industry Leaders Rely on MySQL
7
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
5 Years of Oracle Stewardship More Investment, More Innovation
Jan 2010 Sept 2014
8
2x Engineering Staff 3x QA Staff 2x Support Staff
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
MySQL Basics
How we do Backups and Monitoring
High Availability Solutions
Security
Conclusion
1
2
3
4
5
Oracle Confidential – Internal/Restricted/Highly Restricted 9
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
MySQL Basics
How we do Backups and Monitoring
High Availability Solutions
Security
Conclusion
1
2
3
4
5
Oracle Confidential – Internal/Restricted/Highly Restricted 10
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Comparing Technical Terms
11
Controlfiles - Listener -
Database Database Server Instance Schema User Database / Schema User User (including source IP address!)
Tablespace Tablespace - Storage Engine
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Architecture
12
Support for common development language
platforms
Efficient multithreaded
session handling
Flexible storage engine options for application specific
storage neds (per table)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Storage Engine
13
• InnoDB (default >5.5.5)
– ACID, Referential Integrity, Row Locking, FullText (5.6), crash recoverable
• MyISAM (default <5.5.5)
– .MYD/.MYI portable files, GIS support, concurrent inserts, writes precede over selects
• MEMORY – fixed length rows, hash & btree indexes, table contents temporary
• NDB (Cluster)
– Specific to MySQL Cluster: HA, auto-sharding, storing on network nodes
• CSV, BLACKHOLE, EXAMPLE, ARCHIVE, MERGE/MRG_MYISAM…
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Users
14
• mysql> SELECT user,host,password FROM mysql.user; +------+-----------+-------------------------------------------+
| user | host | password |
+------+-----------+-------------------------------------------+
| root | localhost | *81F5E21E35407D884A6CD4A731AEBFB6AF209E1B |
| root | ::1 | *81F5E21E35407D884A6CD4A731AEBFB6AF209E1B |
| app | % | *462366917EEDD1970A48E87D8EF59EB67D2CA26F |
| app | 10.2.2.10 | *462366917EEDD1970A48E87D8EF59EB67D2CA26F |
| root | % | |
+------+-----------+-------------------------------------------+
• Users are always username + originating IP
– Coming from a different host may mean different access rights!
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Configuration
15
• Just one file: my.[cnf;ini] [client]
# not relevant for server, only for client tools
[mysqld]
innodb_buffer_pool_size = 12G
log_bin
# datadir = .....
# port = .....
# server_id = .....
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
• Typical locations are
– /etc;/etc/mysql;BASEDIR (check with lsof –p < mysqld.pid)
– C:\Windows;C:\;BASEDIR (check system service definition for –defaults-file=...)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL server mode
16
• SET [SESSION;GLOBAL] sql_mode = „mode1,mode2,...“
• MySQL accepts 00-00-0000 as a date -> NO_ZERO_DATE
• MySQL accepts out of range values (e.g 256 as TINYINT)-> STRICT_ALL_TABLES
• MySQL is not ANSI compliant -> ANSI
– REAL_AS_FLOAT | PIPES_AS_CONCAT | ANSI_QUOTES | IGNORE_SPACE
• Other modes: ORACLE
– ANSI | NO_KEY_OPTIONS | NO_TABLE_OPTIONS | NO_FIELD_OPTIONS | NO_AUTO_CREATE_USER
• Other modes: MSSQL, POSTGRESQL, DB2, TRADITIONAL, ...
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
The Graphical User Interface: MySQL Workbench
17
Design, Develop, Administer, Migrate
• MySQL Database IDE
• Millions of Users
• Millions of Downloads
• Windows, OS X, Linux
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
MySQL Basics
How we do Backups and Monitoring
High Availability Solutions
Security
Conclusion
1
2
3
4
5
Oracle Confidential – Internal/Restricted/Highly Restricted 18
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Backup - The Community Way
19
• Backup by ´mysqldump´
– Logical backup / export
– Creates editable SQL file
– Very slow on backup, extremely slow on recovery
– Online for small, less used databases (--single-transaction), read-only otherwise
• Snapshot on Filesystem
– Short write lock on database
– Depending on OS and other tools for copy
• Replication
– Requires additional server / storage
– Maximum flexibility for Community users
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Monitoring - The Community Way 1/2
20
– Error log file in DATADIR/<hostname>.err
– General logfile (Attention: Performance hit) • All login events
• All statements
– Enable slow-query-log to report long running queries • Stored in DATADIR/<hostname>-slow.log
• Read with ´mysqldumpslow´
– Write custom scripts to read system variables and create alarms • SHOW VARIABLES LIKE „%“
• SHOW STATUS LIKE „%“
– Integrate manually into any monitoring framework • And extend scripts after each failure that you did NOT yet check
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Monitoring - The Community Way 2/2
21
• performance_schema offers internal data
– Requires MySQL 5.6
• SYS schema offers predefined reports
– Similar to Oracle v$ catalog views
• Access data with plain SQL
• Workbench offers a GUI to performance reports
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Management Tools Advanced Features Support
• Scalability • High Availability • Security • Audit • Encryption
•Monitoring • Backup • Development • Administration •Migration
• Technical Support • Consultative Support •Oracle Certifications
MySQL Enterprise Edition
22
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Enterprise Manager for MySQL
23
Performance Security
Availability
• Availability monitoring
• Performance monitoring
• Configuration monitoring
• All available metrics collected
– Allowing for custom threshold based incident reports
• MySQL auto-detection
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise Monitor
24
• Start monitoring MySQL in 10 minutes
• Real-time MySQL performance and availability monitoring
• Visually find & fix problem queries
• Disk monitoring for capacity planning
• Cloud friendly architecture
– No agents required
– Policy driven configuration
– Easy integration with DevOps tools
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Best Practice Advisors
25
• Enforce MySQL best practices
• 14 Advisor categories
• 250+ Advisors
• Threshold-based alerts
– Exponential moving averages
– Rate change detection
• Expert problem resolution advice "I definitely recommend the MySQL Enterprise Monitor to DBAs who don't have a ton of MySQL experience. It makes monitoring MySQL security, performance and availability very easy to understand and to act on.” Sandi Barr Sr. Software Engineer Schneider Electric
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
“With the MySQL Query Analyzer, we were able to identify and analyze problematic SQL code, and triple our database performance. More importantly, we were able to accomplish this in three days, rather than taking weeks.” Keith Souhrada Software Development Engineer Big Fish Games
Enterprise Query Analyzer
26
• Real-time query performance
• Visual correlation graphs
• Find & fix expensive queries
• Detailed query statistics
• Query Response Time index (QRTi)
– “Quality of Service” (QoS) measurement for each query
– QoS measurement for a server, group, or every instance
– Single metric for query performance
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise Backup
• Online, non-locking backup and recovery – Complete MySQL instance backup (data and config) – Partial backup and restore
• Direct Cloud storage backups (S3, etc.) • Incremental backups • Point-in-time recovery • Advanced compressed and encryption • Backup to tape (SBT) • Backup validation • Integrate with Oracle Secure Backup • Cross-Platform (Windows, Linux, Unix)
27
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
MySQL Basics
How we do Backups and Monitoring
High Availability Solutions
Security
Integration with other Oracle solutions
1
2
3
4
5
Oracle Confidential – Internal/Restricted/Highly Restricted 28
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise High Availability
• HA with MySQL Fabric
• Oracle VM Templates for MySQL
• HA with DRBD and Linux Clustering
• HA with Solaris Clustering
• HA with Windows Failover Clustering
• HA with Oracle Clusterware Plugin
29
Range of options to make your
MySQL application deployments Highly Available
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Replication
MySQL Fabric
DRBD
Windows/ Solaris/ Clusterware Clustering or Oracle VM
MySQL Cluster
MySQL HA Solutions
16/06/2015
9 9 . 9 9 9 %
Copyright 2015, Oracle and/or its affiliates. All rights reserved 30
Range of options to make your
MySQL application deployments Highly Available
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Why Replicate?
• Duplicates database from a “master” to a “slave”
– Redundant copies of the data provide foundation for High Availability
– Scale out by distributing queries across the replication farm
Master
Slaves
Web / App Servers
Writes & Reads Reads
16/06/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 31
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise HA: MySQL Fabric
32
• High Availability features
– Server monitoring
– Auto-promotion
– Transparent application failover
– Dynamically scale up and down
• Fabric-aware connectors rather than a proxy
– Python, Java, and .NET
– Lower latency, bottleneck-free
• Optional sharding features
MySQL Fabric
Connector
Application
Read-slaves
SQL
HA group
Read-slaves
HA group
Connector
Application
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 16/06/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 33
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster Overview
• Memory optimized tables with durability
• Predictable Low-Latency, Bounded Access Time REAL-TIME
• Auto-Sharding, Multi-Master
• ACID Compliant, OLTP + Real-Time Analytics HIGH SCALE, READS +
WRITES
• Shared nothing, no Single Point of Failure
• Self Healing + On-Line Operations 99.999% AVAILABILITY
• Key/Value + Complex, Relational Queries
• SQL + Memcached + JavaScript + Java + HTTP/REST & C++ SQL + NoSQL
• Open Source + Commercial Editions
• Commodity hardware + Management, Monitoring Tools LOW TCO
16/06/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 34
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster Scaling
MySQL Cluster Data Nodes
Clients
Application Layer
Data Layer
16/06/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 35
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• Memory optimized tables
– Durable
– Mix with disk-based tables
• Massively concurrent OLTP
• Distributed Joins for analytics
• Parallel table scans for non-indexed searches
• MySQL Cluster 7.4 FlexAsych – 200M NoSQL Reads/Second
16/06/2015 36
MySQL Cluster 7.4 NoSQL Performance 200 Million NoSQL Reads/Second
Copyright 2015, Oracle and/or its affiliates. All rights reserved
-
50,000,000
100,000,000
150,000,000
200,000,000
250,000,000
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32
Readspersecond
DataNodes
FlexAsyncReads
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• Memory optimized tables
– Durable
– Mix with disk-based tables
• Massively concurrent OLTP
• Distributed Joins for analytics
• Parallel table scans for non-indexed searches
• MySQL Cluster 7.4 DBT2 BM – 2.5M SQL Statements/Second
16/06/2015 37
MySQL Cluster 7.4 SQL Performance 2.5M SQL Statements/Second
Copyright 2015, Oracle and/or its affiliates. All rights reserved
-
500,000
1,000,000
1,500,000
2,000,000
2,500,000
3,000,000
2 4 6 8 10 12 14 16
SQLStatements/sec
DataNodes
DBT2SQLStatementsperSecond
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
When to Consider MySQL Cluster
Scalability demands Sharding for write performance?
Latency demands Cost of each millisecond?
Uptime requirements Cost per minute of downtime? Failure versus maintenance?
Application agility Developer languages and frameworks? SQL or NoSQL?
16/06/2015 Copyright 2015, Oracle and/or its affiliates. All rights reserved 38
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
MySQL Basics
How we do Backups and Monitoring
High Availability Solutions
Security
Conclusion
1
2
3
4
5
Oracle Confidential – Internal/Restricted/Highly Restricted 39
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Block Threats
Auditing
Regulatory Compliance
Login and Query Activities
SSL/TLS
Public Key
Private Key
Digital Signatures
Privilege Management
Administration
Database & Objects
Proxy Users
MySQL
Linux / LDAP
Windows AD
Custom
Oracle Confidential – Internal 40
MySQL Security Overview
Authorization Authentication
Firewall & Auditing
Encryption
Security
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Privilege Management
• user: user accounts, global privileges columns
• db: database-level privileges
• tables_priv: Contains table-level privileges
• columns_priv: Contains column-level privileges
• procs_priv: Contains stored procedure and function privileges
• proxies_priv: Contains proxy-user
41
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Authentication
• Built in Authentication
– user table stores users and encrypted passwords
• Before MySQL 5.7 run ´mysql_secure_installation´
– Sets user passwords, removes anonymous account and test database
• Since MySQL 5.7 secure by default – if not needed install with ´mysqld –insecure´
• Password validation plugin
• Password Expiration / Rotation
• Account Lockout (in MySQL 5.7)
42
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise Authentication
43
• Integrate with Centralized Authentication Infrastructure
– Centralized Account Management
– Password Policy Management
– Groups & Roles
• PAM (Pluggable Authentication Modules)
– Standard interface (Unix, LDAP, Kerberos, others)
• Windows
– Access native Windows service - Use to Authenticate users using Windows Active Directory or to a native host
Integrates MySQL with existing security infrastructures
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise Firewall
• Real Time Protection
– Queries analyzed and matched against White List
• Blocks SQL Injection Attacks
– Positive Security Model
• Block Suspicious Traffic – Out of Policy Transactions detected & blocked
• Learns White List
– Automated creation of approved list of SQL command patterns on a per user basis
• Transparent
– No changes to application required
44
MySQL Enterprise Firewall monitoring
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise Firewall
• SQL Injection Protection with Positive Security Model
45
• Out of policy database transactions detected and blocked
• Logging & Analysis
Select *.* from employee where id=22
Select *.* from employee where id=22 or 1=1
Block & Log ✖
Allow & Log ✔
White List Applications
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise Encryption
• MySQL encryption functions
– Symmetric encryption AES256 (All Editions)
– Public-key / asymmetric cryptography – RSA
• Key management functions – Generate public and private keys
– Key exchange methods: DH
• Sign and verify data functions
– Cryptographic hashing for digital signing, verification, & validation – RSA,DSA
46
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise Audit
• Out-of-the-box logging of connections, logins, and query
• User defined policies for filtering, and log rotation
• Dynamically enabled, disabled: no server restart
• XML-based audit stream per Oracle Audit Vault spec
47
Adds regulatory compliance to MySQL applications
(HIPAA, Sarbanes-Oxley, PCI, etc.)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
MySQL Basics
How we do Backups and Monitoring
High Availability Solutions
Security
Conclusion
1
2
3
4
5
Oracle Confidential – Internal/Restricted/Highly Restricted 48
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise Oracle Certifications
• Oracle Enterprise Manager for MySQL
• Oracle Linux (w/DRBD stack)
• Oracle VM
• Oracle Solaris
• Oracle Solaris Clustering
• Oracle Clusterware
• Oracle Audit Vault and Database Firewall
• Oracle Secure Backup
• Oracle Fusion Middleware
• Oracle GoldenGate
• My Oracle Support
MySQL integrates into your Oracle environment
49
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | © Copyright Gilles Martin-Raget / BMW ORACLE Racing
Thank you! Mario Beck http://mablomy.blogspot.de