scaling mysql -- ny code camp

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

1


2

Insert Picture Here

MySQL Scaling§ Dave Stokes

MySQL Community Manager

[email protected]

@stoker

slideshare.net/davestokes

mailto:[email protected]


3

Safe Harbor Agreement

.The following is intended to outline our general product direction. It is

intended for information purposes only, and may not be incorporated into

any contract. It is not a commitment to deliver any material, code, or

functionality, and should not be relied upon in making purchasing decision.

The development, release, and timing of any features or functionality

described for Oracle’s products remains at the sole discretion of Oracle.


4

You want to get a bigger, badder MySQL --BUT what are the costs???

Budget

Complexity

Application

Opportunity

Sanity!


5

LAMP Stack https://en.wikipedia.org/wiki/LAMP_(software_bundle)

LAMP is an archetypal model of web service solution stacks, named as an acronym of the names of its original four open-source components: the Linux operating system, the Apache HTTP Server, the MySQL relational database management system (RDBMS), and the PHP programming language. The LAMP components are largely interchangeable and not limited to the original selection. As a solution stack, LAMP is suitable for building dynamic web sites and web applications.

Since its creation, the LAMP model has been adapted to other componentry, though typically consisting of free and open-source software. For example, an equivalent installation on the Microsoft Windows family of operating systems is known as WAMP.


6

Simple Stuff


7

Option 0

Run the Latest, Greatest MySQL

MySQL 5.5 was 20% faster than 5.1! 5.6 is another 10-15%

--Make sure your config file matches your hardware

--ANALYSE TABLE – Maintenance of indexes, statistics

--Check for SLOW QUERIES, QUERIES w/o INDEXES

--TUNE QUERIES

This is the least complex, less upsetting, and usually lowest budget cost option


8

Option 0.1

Add in a Memcached Layer

● Cache for quick access

● Session Data

● Sports Scores

● Opens up NoSQL accessto InnoDB later

This is the second least complex, less upsetting, and usually lowest budget cost

option but does make you change your application to become 'cache aware'.


9

Insert Picture Here

What database serviceLevels do you Need toProvide?


10

UpTime versus Response Time versus Access

Things to be engineered/negotiated upfront

Up Time

– How much down time can you afford?

Response Time

– How long is too long to wait?

Access

– How many do you need to serve simultaneously.


11

Vertical Scaling

● Sometimes a newer, bigger server is the next logical step

● All eggs in newer basket

● Not limitless

● Tossing $$ at problem

● Transition pain

● May not openbottlenecks


12

Insert Picture Here

Simple Level 1 -Splitting Reads /Writes


13

Use MySQL Replication to Split Read/Writes


14

MySQL Replication


15

READ Splitting

Most Common Scaling Technique

● More complex environment

● Application Changes

● Easier with PHP & Java

● Low opportunity and sanity costs

● Less risk of one server stopping operations


16

https://blog.engineyard.com/2014/easy-read-write-splitting-php-mysqlnd

● PHP Developers Have it Easy!

● Use the mysqlnd_ms Native Driver Extension

– Read/write splits

– Load balancing

– No change to application


17

Read Splitting—Easy Way

Use MySQL Utilities to copy master, set up failover

● mysqlserverclone

● mysqlfailover

– When master dies, most up to date slave (GTID) promoted automatically

● Mysqlrplcheck & mysqlrplshow


18

MySQL Fabric

● Set up highly available groups of servers or shards

● 'Smarts' in the controller, no change to application

● Automatic fail over

● Re - shard on the fly

● As easy yo set up asreplication


19

Insert Picture Here

Simple Level 2Moderately Easy, BetterAvailability


20

Distributed ReplicatedBlock Device (DRBD)

● High Availability

More complex environment

● Proven, supported

● Please practice fail over

and back

● Easy to manage but

not HA panacea


21

Cloud

High Availability, low cost (sometimes)

Slightly more complex environment

● Proven, supported

● Best for ephemeral data, may not make upper management

comfortable (or you)

● Easy to manage but not HA panacea


22

Insert Picture HereComplex Level N^NNever, Ever Down


23

MySQL ClusterFor 99.999% uptime, upper bound on response time

Extremely Complex

● No single point of

failure = lots of $$

hardware

● Not general

purpose MySQL

Auto sharding

● Add/Subtract nodes at will

● Replicate between

geographically separated

locations

● Need a smart staff


24

MySQL Cluster Shards Data Automatically

● Single or multiple data node failures do not bring down a cluster

● Easy to add/remove data nodes


25

MySQL Cluster Topology

● Nodes

– MGMT – startup/add node

– Data – Where data lives

– Mysql – the RDMS

● Can be engineered for no

single point of failure

● Easily replicate across data centers


26

Insert Picture HereBigData


27

Ask ten DBAs to define 'Big Data' and You WillGet At Least Eleven Answers

● BIG as massive amounts

– Define MASSIVE

● BIG as in fast

● Define fast

● BIG as lots of eyes

● BIG as all of the the above!!!

● How many simultaneous users


28

The Three Vs of Big Data

● Volume

● Variety

● Velocity

● (maybe) Veracity


29

End Goal of your Big Data?

● Never, ever toss any data away?

● How will be it managed, examined, maintained, documented, and used?

● Latent needs?

● Data warehouse or Digital Landfill


30


31


32

The Easiest Way to Get Speed

NoSQL/Memcached to InnoDB/NBD

● 9x faster by skipping parser, optimizer

● Key/Value pair access

and SQL access of same data


33

Easiest Way to Gain Size

Columnar Storage Engines for MySQL

● Not InnoDB

● Unique features, quirks

● Need > 26,000,000,000 rows to start seeing 'sweet spot'

● Use new server, don't scrimp

See Calpont and Infobright


34

Hadoop-- getting data from MySQL to Hadoop

80% of Hadoop Installations feed from MySQL

● Hadoop is made up of several components

● Twisty little maze, all alike

● Use Apache Bigtop

– Tested for interoperability

● Cloudera & Horton Works have great tutorials

● Kinda 'SQL-ish'

● Squoop and MySQL Hadoop Applier to move data


35


36

Partitioning and Sharding /Dividing and Conquering

Partitions built-in to MySQL

● Nice management tool

– Rolling quarters/months

– Fast drops

● No change to application but

higher complexity, low

opportunity cost

Sharding

● Much more complex

application and operations

● Not off shelf

● Hard to re-engineer

● Works


37

MySQL Fabric

● Lets you shard and reshard

● Very new

● Looking for feedback

● Http://labs.mysql.com


38

Insert Picture Here

Wrap up


39

Goals – What are you trying toaccomplishDefine problem completely

● Who produces, who consumes

data

● Negotiate performance rates

● Plan for failure, catastrophe

● Design for long life

What does end solution look/feel like?

● Platforms

● Speed

● Access areas

● Security

● Serenity


40

MySQL Marinate

● Free virtual class to learn MySQL run by the Boston MySQL Users Group

– 1 chapter a week

– Homework graded helpfully


41

MySQL Connect

● San Francisco● September 27th – October 2nd

● LA MySQL User Group will have some passes to raffle

● How do we discount for other groups?


42

Q&A

[email protected] @stoker

slideshare.net/davestokes

mailto:[email protected]


43

Insert Picture Here


44

Graphic Section Divider

scaling mysql -- ny code camp

Software

oracle andor

sole discretion of oracle

greatest mysql mysql

badder mysql

lamp model

lamp components

bundle lamp

response time