techtalk #13 grokking: scaling and supercharging your online business models with elasticsearch
TRANSCRIPT
![Page 1: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/1.jpg)
SCALING & SUPERCHARGING
YOUR ONLINE BUSINESS MODELS
WITH ELASTICSEARCHTRI TRAN @ GROKKING VIETNAM
JUNE 2016
![Page 2: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/2.jpg)
ABOUT ME
• Lead Developer @ 8Bit Rockstars• Tech Lead @ Tablen & IdeasOn• Tech Mentor @ MyProClub• @moprit
I’m very happy if you call me `Mụp`
![Page 3: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/3.jpg)
WE WILL DISCUSS ABOUT
• Scaling with Elasticsearch• Good infrastructures for supercharging • ONGR & Spryker• Load Test & Evaluation
YOUR ONLINE BUSINESS
MODELS
![Page 4: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/4.jpg)
WE WON’T TALK ABOUT
• What’s Elasticsearch?• Compare Elasticsearch with the others• Back-end Framework Evaluation
![Page 5: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/5.jpg)
SCALINGWITH ELASTICSEARCH
![Page 6: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/6.jpg)
PROBLEMS?
From: ttgtmedia.com
• Large joins taking up memory• Working size exceeds the hardware limits• Full text search & NLP• Write-write conflicts
COMMON SOLUTION?
• Database Replication• Full page cache
DATABASE CONNECTION
![Page 7: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/7.jpg)
From: Google
![Page 8: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/8.jpg)
NEW APPROACH:
![Page 9: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/9.jpg)
HOW?
• Shard? What is it?• Primary shard vs. Replica shard• Shard Overallocation
![Page 10: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/10.jpg)
“”
Elasticsearch provides the ability to subdivide your index into multiple pieces called shards. Each shard is in itself a fully-functional and independent "index" that can be hosted on any node in the cluster.
From www.elastica.co
• It allows you to horizontally split/scale your content volume
• It allows you to distribute and parallelize operations across shards (potentially on multiple nodes) thus increasing performance/throughput
![Page 11: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/11.jpg)
PRIMARY VS. REPLICA
P1 R1 R2
• Can be stored on any node in a cluster• Independent Index• Distributed• Parallelize operations • A copy of primary shard
• In case failover• Never resides on a same node as
the original shard• Scaling search volume
![Page 12: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/12.jpg)
SHARD OVERALLOCATION
https://www.youtube.com/watch?v=WLiVjAELUaUBy Scott Davis
![Page 13: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/13.jpg)
SUPERCHARGINGWITH ELASTICSEARCH
![Page 14: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/14.jpg)
GOALS
• Preventing to spend expensive SQL queries to a relational database with several joins and conditions.• Or using Elasticsearch as a primary data
storage
![Page 15: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/15.jpg)
From Spryker.com
![Page 16: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/16.jpg)
From ONGR.io
![Page 17: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/17.jpg)
From ONGR.io
![Page 18: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/18.jpg)
PERFORMANCE LOAD TEST
![Page 19: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/19.jpg)
PERFORMANCE LOAD TEST
• Normal Load Condition (0-5 clients per 5 minutes | 1 dyno)
• Peak Load Condition (0-100 clients per 5 minutes | 1 dyno)
• Extra Scaling Up Load Condition (0-600 clients per 5 minutes | 10 dyno)
![Page 20: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/20.jpg)
DISCUSSROCK TOGETHER
INBACKGROUND
MUSIC
![Page 21: TechTalk #13 Grokking: Scaling and supercharging your online business models with Elasticsearch](https://reader036.vdocument.in/reader036/viewer/2022081520/588a48c01a28abd3088b580b/html5/thumbnails/21.jpg)
MOST WANTED
www.8bitrockr.com
Project Manager
Senior Front-end Developer
Senior PHP Developer8Bit Rockstars Co., LTD.L08 – R01, Vincom Center72 Le Thanh Ton, Ben Nghe, District 1, HCMC, VN