mariadb maxscale – der neue intelligente datenbank-proxy
Post on 15-Jul-2015
319 Views
Preview:
TRANSCRIPT
© 2015, MariaDB Corp. © 2015, MariaDB Corp.
MariaDB Roadshow 2015
MariaDB MaxScaleDer neue intelligente Datenbank-Proxy
Ralf Gebhardt
© 2015, MariaDB Corp.
Needs of Web Scale organization
● Ability to support growing user base ○ Scalability
● Deliver quality customer experience ○ High Availability ○ Continuity ○ Performance ○ Data Security
● Control cost ○ Development Friendliness ○ Improve DBA Productivity
● Competitive Advantage ○ Innovation Rollout ○ New Service Creation
© 2015, MariaDB Corp.
Challenges of Scalability withMariaDB & MySQL
Application Business Logic
Application Database Connection Management
MySQL/MariaDB connector
● Complex application ● Reduced Development Agility
and DBA Productivity ● Longer time to market
● For security requirements additional database firewall needed
Database Firewall Database Firewall
● Load Balance queries ● Send queries to node per load balancing ● Detect node failure and failover ● Detect node addition or removal
● Determine which shard to send the query ● If sharding schema changes, or a new shard
added, ○ update the application code
MariaDB/MySQL Replication
MariaDB Galera Cluster
1:100 101:200 201:300 301:...
Database Firewall
Data Sharding
© 2015, MariaDB Corp.
Scalability with MaxScale for MariaDB/MySQL replication
Application Business Logic
Application Database Connection Management
MySQL/MariaDB connector
MaxScale
Connection and Statement based load balancing
● Simplifies applications ○ Improved Developer productivity
● Cluster configuration transparent to applications ○ Improved DBA productivity
● Monitors backend database high availability ○ Improved Business Continuity
● Load Balance queries ● Route queries to node per load balancing algorithm ● Detect node failure and failover ● Detect node addition or removal
● Send queries over a single server connection to MaxScale
MariaDB Galera Cluster MariaDB/MySQL Replication
© 2015, MariaDB Corp.
MariaDB MaxScale: Lightweight, Modular, Intelligent, Transparent
Mon
itor Filter &
Logging
Routing
MaxScaleCore
Routing
● MaxScale core is a high performance message switch.
● Module APIs allow for multiple modules of each type.
● Modules can beclient-facing, back-endfacing, internal, or connectto other DBs.
● Eases customer-driven development.
Flexible architecture, no change to clients.
© 2015, MariaDB Corp.
Client and Back-End Protocols
● Client protocols let applicationsconnect to MaxScale ○ Initial support for MySQL connectors. ○ Future support planned for JSON/BSON, others.
● Back-end protocols allow MaxScale to connect to different database and cluster technologies. ○ Initial support for MySQL Replication, MariaDB
Replication, Galera Cluster, and MySQL Server with NDB storage engine.
Protocol
© 2015, MariaDB Corp.
Routing and Load Balancing
● Routing modules implementfundamental proxy algorithms. ○ Connection based routing: establishes route upon
connection, doesn’t examine requests. ○ Statement based routing: examines requests to
dynamically route requests to the appropriate back-end.
● Modular design allows for load balancing, HA failover, sharding, and other routing policies.
Routing
© 2015, MariaDB Corp.
Filtering, Logging, and Auditing
● Can log requests that meet filtercriteria such as time-out, accessspecific row, etc.
● Configurable firewall blocks or allows requests based on policy.
● External filtering and transformation pipeline processes requests and results as they pass through MaxScale. ○ Allows for creation of parallel streams. ○ Transforms can alter both requests and returned data.
Filter/Log
© 2015, MariaDB Corp.
Authentication and Monitoring
● Authentication modules letMaxScale proxy user credentials.
● Monitoring modules provide the
detailed back-end stateinformation MaxScale needs to make the right routing decisions.
Authentication
Monitor
© 2015, MariaDB Corp.
MariaDB MaxScale: Implementing Advanced Database Services
● MaxScale services interface to clients and servers through protocols.
● Router modules implement policy to determine which servers are best able to handle requests.
● Filters and logging modules implement a pipeline that can block, split, or transform requests. Simple clients, complex
architectures.
Routing Fi
lter/
Log
Client Protocol
Server Protocol
Message Core&
State Machine .log
© 2015, MariaDB Corp.
MariaDB MaxScale 1.0 Feature List
● Protocol ● MySQL and MariaDB client and server
● Monitoring ● MariaDB Galera Cluster ● MySQL replication cluster
● Load Balancing ● Connection based ● Statement based
● Logging ● Query performance ● Top N Query
● Query transformation based on regular expressions and time ● Query duplication to another database, storage engine or application ● Extensible plugin architecture
© 2015, MariaDB Corp.
Read Scalability: MySQL Replication + Connection Load Balancing
● Applications are “cluster aware”, split reads and writes on separate connections.
● Using connection load balancing, MaxScale routes the read/write connection to the master, and uses round-robin load balancing to route read-only connections to the slaves.
● MaxScale monitors the cluster, adjusting resource pools should a slave fail, or should MHA failover promote a slave to master.
MaxScale
© 2015, MariaDB Corp.
RW Scalability, HA Failover: Galera Cluster + Connection Load Balancing
● Applications can be simple, use one connection for reads and updates.
● Using connection load balancing, MaxScale load balances connections to each node in the Galera Cluster, round-robin.
● MaxScale monitors the cluster, routing requests only to fully synchronized nodes.
MaxScale
© 2015, MariaDB Corp.
RW Splitting, Read Scalability: MySQL Replication + Statement Load Balancing
● Applications can be simple, use one connection for reads and updates.
● Using statement load balancing, MaxScale dynamically routes writes to the master, and round-robin load balances read requests to the slaves.
● MaxScale monitors the cluster, routing requests only to fully operational nodes.
MaxScale
R/WSplitting
© 2015, MariaDB Corp.
Query Transformation with MaxScale for Legacy Application Compatibility
Modify queries from legacy applications on the fly - for example a MySQL 5.1 app:
1. MaxScale accepts a query from a MySQL 5.1 compatible client,
2. If the query matches the regular expression “/CREATE TABLE/” then MaxScale substitutes “ENGINE” for “TYPE” in that statement, else it passes the statement through the filter unchanged.
3. Forwards the transformed statement to MariaDB or MySQL 5.6.
4. Receives the result from the back-end. 5. Forwards the result to the client.
MaxScale
1 5
3 4
/CREATE TABLE/
s/TYPE/ENGINE/
2
● Application and Database can be upgraded asynchronously ○ Improves business continuity ○ Improve organizational productivity
● Faster rollout of new database versions
MySQL 5.1
MySQL 5.6 or MariaDB 10
© 2015, MariaDB Corp.
Query Logging with MaxScale For Performance Diagnostics
1. MaxScale accepts a query from a client application
2. Forwards the query to the back-end ○ Logs it into the “All Queries” ○ Logs it into the “Long Running Queries”
log files 3. Receives the result from the back-end, 4. Forwards the result to the client
○ Logs the result into the “All Queries” log ○ If it is one of the N longest-running
queries, logs the result into the “Long Running Queries” log file MaxScale
.log
Long Queries (Top N queries)
.log
All Queries
1 4
2
4
2
4 2 3
● Centralized cluster wide query performance log at MaxScale ○ No need to manually correlate query
logs across cluster nodes ○ Simplified performance tuning ○ Improved DBA productivity
● Filtering can be also used to prevent SQL injection attacks or unauthorized queries ○ Improved Security
© 2015, MariaDB Corp.
Query Duplication with MaxScale 1. MaxScale accepts a query from a client
application 2. Forwards the query to the back-end
○ Sends query to Database A ○ Send query to Database B via Tee-filter
3. Receives the result from Database A 4. Receives the result from Database B 5. Forwards the result to the client
MaxScale
1 5
4
2 2 3
● Cross-DB solutions: Duplicate queries to MariaDB Replication Cluster and Galera Cluster
● ETL: Use Tee-filter to insert into transactional database and insert into analytic database via queuing to ETL process
● Faster and smooth rollout of new services ○ Using different DB technology and
different workloads
Database A: MariaDB InnoDB Replication Cluster
Database B: MariaDB Galera Cluster or MariaDB MyISAM or CONNECT Engine for Cassandra or Spider Engine or InfiniDB Engine or RabbitMQ for another application to process
© 2015, MariaDB Corp.
Scalability Binlog relay: improved HA for master/slave replication
Schema based sharding
MySQL Cluster (NDB) connection load balancing
Named Server Routing
Ease of Use Integration with MDBE notification service
Nagios Plugin for monitoring
Hint based statement load balancing
Canonical Query Logging integrated with RabbitMQ
Security Database Firewall filter: Block SQL injection
What is new in MariaDB MaxScale 1.1?
© 2015, MariaDB Corp.
Key Use Case: Bin Log Server
● Transparent MySQL replication relay ○ Horizontally Scale Slaves without Master Overload ○ Crash Safe Disaster Recovery ○ Better Parallel Replication
● User facing application reads from slaves are most up to date
MySQL 5.6 compatible
© 2015, MariaDB Corp.
Key Use Case: Schema Sharding
● Route to sharded database ● Use Case:
A Business is hosting database for multiple customers on sharded servers. As business grows add new customers while continuing existing services ○ Each customer with its own
database schema. ○ Each Server hosts up to 600
customers. ○ Client Applications are aware of
database schema - but not the hosting server
Shard 1
MaxScale
Sharding Router
Shard 2 Shard 3 Shard 4 Shard 5
● MaxScale Schema Shard Router Solution: ○ All applications connect to single
MaxScale ○ MaxScale routes the shard server based
on query from client ○ No impact on existing client application
when ■ New client or shard server is added ■ Database schema are moved around
shard server
© 2015, MariaDB Corp.
Key Use Case: Database Security
● Database Firewall Filter ○ Block query based on date/time, where clause, wildcard, regular expression match, column
match, query type(insert, select, delete, update) ○ Prevent sensitive data from leaking to the wrong parties
● Use Case: Prevent user login data breach
● MaxScale Database Firewall Filter Solution: ○ Configure a filter that will block any SELECT queries on “logins” table with an “OR”
operator in WHERE clause
SQL injection into this query: SELECT idFROM loginsWHERE username = 'Joe'AND password = 'anything' OR 'x'='x' Result: All the rows of the logins table
Client application generated dynamic SQL: SELECT idFROM loginsWHERE username = '$username'AND password = '$password' Result: A Single row matching the user
Secure End User Data, Prevent Identity Theft, while allowing developer agility
© 2015, MariaDB Corp.
Key Use Case: Nagios Plugin
check_maxscale_resources
status and performance data
maxscaled and cli protocol configured in maxscale.cnf
Nagios plugin scripts and command configuration files that comes with MaxScale binaries are copied in appropriate directory
Nagios core
check_maxscale_resources
check_maxscale_threads
check_maxscale_monitor
Nagios: Leading Open Source Tool ● Monitoring, alerting and reporting ● More than 1 million worldwide users ● Ease to use and integrates with enterprise monitoring ecosystem
MaxScale scripts for Nagios ● Monitor MaxScale service, threads and resources ● Alert upon failure or anomalies ● Report performance data
© 2015, MariaDB Corp.
MariaDB MaxScale Advantages
● Developer and DBA friendly solution for ○ Scalability ○ High availability ○ Cluster wide performance tuning
● Continuity during database upgrades ● Data and SQL security ● New Services with extendable plugins
© 2015, MariaDB Corp.
MariaDB MaxScale Customer Benefits
● DBA ○ Handle large amount of data with scalable and highly available MariaDB clusters ○ Tune database performance with query logging ○ Continuous availability to applications during cluster administration and database
upgrades ● Developer
○ Simplified development as cluster complexity hidden from applications ○ Focus on business logic instead of cluster monitoring and tracking
● Data Architect ○ Tee-filter and Flexible Plugin architecture opens up cross-workload and cross-DB
solutions ○ Simplified and faster rollout of database versions for new services
■ Applications continue to work during database upgrades ● CIO
○ Scale the infrastructure with growth of business ○ Scale without sacrificing developer and DBA productivity ○ Improved customer satisfaction with application continuity and performance ○ Expand business horizon with newer solutions
© 2015, MariaDB Corp.
MaxScale Resources
● Product Page: https://mariadb.com/products/mariadb-maxscale ● Product FAQ: https://mariadb.com/products/product-faqs ● Data Sheet: MariaDB MaxScale Datasheet ● Blogs: https://mariadb.com/blog-tags/MaxScale ● Discussion via the Google Group: maxscale@googlegroups.com ● MariaDB MaxScale Webinar - Scalability at its best
© 2015, MariaDB Corp. © 2015, MariaDB Corp.
Questions?
mariadb.com
ralf.gebhardt@mariadb.com
"MySQL is a registered trademark of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. MariaDB is not affiliated with MySQL."
© 2015, MariaDB Corp.
MariaDB MaxScale 1.0 Supported OS and Databases
● Supported OS ○ RedHat 5, 6, 7 ○ CentOS 5, 6, 7 ○ Ubuntu 12.04 LTS, 13.10, 14.04 ○ Debian 6, 7 ○ OpenSUSE 13.1 ○ SLES 11, 12 ○ Fedora 19 64 bit ○ Fedora 20 64 bit
● Supported Backend Database ○ MariaDB Enterprise Server and Enterprise all versions ○ MariaDB Enterprise Cluster all versions ○ MySQL 5.1 and higher
© 2015, MariaDB Corp.
Feature Benefits: MaxScale Base DBA Developer Architect Organization CIO
Overall Benefits
Database Scalability, HA Simplicity Flexibility Organizational Scalability, Productivity, Customer Satisfaction, New revenue stream opportunities
Feature Component Feature Benefits
MySQL and MariaDB Client and Server Protocol
Core Simplified Application Development that can focus on business logic
Increased developer productivity
MariaDB Galera and MySQL replication cluster monitoring
Monitor Plugin Scalability and Reliability to handle large amount of data
Free from cluster configuration complexity
Ability to support growth of business Increased developer productivity
Monitor Plugin Administer Clusters(node addition, removal, replication setup) without impacting application
Continuous availability. Zero-downtime and hidden failover at application level,
Increase DBA productivity End user satisfaction with always available web applications
Connection and Statement based load balancing
Router Plugin Optimal and homogeneous cluster nodes utilization giving high performance and high throughput.
Improved application performance with high performance database queries
End user satisfaction with high performance web applications Optimized Capex resources
Query Performance Logging Top N Query Logging
Filter Plugin Database performance tuning
End user satisfaction
Query Statement Modification based on regular expressions
Filter Plugin Client Applications do not have to upgrade at the same time of database
Applications continue to work when upgrading from older DB version to newer version
End user satisfaction with continuous availability Improved Developer productivity
Query Duplication to another storage engine or application
Tee-Filter Plugin Cross Workload and Cross Technology Applications
New customer offerings with cross workload solutions
Flexible Plugin Architecture Core Architect New Applications not possible before
New Revenue Stream
© 2015, MariaDB Corp.
Feature Benefits: MaxScale Base DBA Developer Architect Organization CIO
Feature Component Feature Benefits
Query Duplication to another database
Monitoring Plugin for another database type
Database migration from another db to MariaDB
Continuous availability during database migration
Replication Relay Big Log Router Horizontally scale slaves without master overload
High Performance read scalability
Scalability and Performance to meet the end user expectation
Routing Hints Hint Filter Performance tuning via hints dynamically control statement-based routers
Improved Application performance and stability
Canonical Query logging Canonical Query Filter Application Validation Improved Developer productivity and application quality
Database Firewall Firewall Filter Prevent SQL Injection attacks
Data access security without application awareness
Prevent sensitive data from leaking to the wrong parties
Data access security
Schema based sharding Table Key based sharding
Sharding Router Scalability and Performance with Sharding
Application do not need to be aware of sharding
Scalability and Performance
top related