Download - Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere
![Page 1: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/1.jpg)
Sam GhodsVP Technology
![Page 2: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/2.jpg)
The simplest way for businesses to share and access data, anywhere
![Page 3: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/3.jpg)
![Page 4: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/4.jpg)
12 months ago…
400M files40M folders
One MySQL database
![Page 5: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/5.jpg)
Need to scale!
![Page 6: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/6.jpg)
NoSQL!
![Page 7: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/7.jpg)
“NoSQL” goodies
• Easy to scale– Just add machines!– Sharding handled by the database– Linearly scales, shared-nothing, no serious SPOF
• Fast, fairly simple CRUD operations• Schema-less
![Page 8: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/8.jpg)
Not so fast.
![Page 9: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/9.jpg)
We sharded MySQL instead.
![Page 10: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/10.jpg)
Why???
![Page 11: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/11.jpg)
If you use a NoSQL store, but need any advanced features in your data
store, you have to rebuild them from scratch yourself.
![Page 12: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/12.jpg)
If you are willing to partition your data yourself, you can use
MySQL’s fancy features.
![Page 13: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/13.jpg)
Inter-Row Consistency
![Page 14: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/14.jpg)
Inter-Row ConsistencyFile trees must remain consistent
• Folder A– Test File– Test File
• Solution: unique index• Solution: lock folder A
![Page 15: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/15.jpg)
Inter-Row ConsistencyModify data structure and log event
• Folder A– Test File 1
![Page 16: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/16.jpg)
Inter-Row ConsistencyModify data structure and log event
• Folder A– Test File 2
• Solution: Use transactions rename event
![Page 17: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/17.jpg)
Inter-Row ConsistencyDenormalizations
• Folder A– Test File 1
• Solution: transactions
delete this must be deleted too
![Page 18: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/18.jpg)
Indexes
• Indexes are way more awesome than people give them credit for– Guaranteed to be consistent– Extremely fast– Data locality – Only access and pull the data you
need– No maintenance required except initial ALTER cost
• SELECT files ORDER BY name (or updated time, or size, etc…)
![Page 19: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/19.jpg)
Tools
• How do you know what’s happening in your data store?– SHOW FULL PROCESSLIST– innotop
• Benchmarking tools– mysqlslap
• pt-query-digest– github.com/box/anemometer
![Page 20: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/20.jpg)
Maturity/Reliability
• Biggest companies in the world have been using MySQL for primary data storage for over a decade– Facebook, Google, Twitter, every other
company ever• When you’re dealing with the crown jewels
of your company, you can’t experiment
![Page 21: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/21.jpg)
HBase!
![Page 22: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/22.jpg)
HBase
• Currently using it as a massive event-propagation store (which can be recreated from MySQL data)
• Started a 3-person task force to learn and productionalize it
• Considering moving more to it in the future but likely need few more years of production experience
![Page 23: Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb75503460f94bc15fe/html5/thumbnails/23.jpg)
Final Thoughts
• Don’t choose a database just because “it scales”
• “Wade, don’t jump into new technologies.”• If you go with new technology, be aware that
crazy things might happen• Make sure you’re not rebuilding MySQL