es part 2 pdf no build
TRANSCRIPT
![Page 1: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/1.jpg)
elasticsearch the missing intro
Part 2: Configuration & Deployment
by Erik Rose
![Page 2: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/2.jpg)
![Page 3: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/3.jpg)
clustering
![Page 4: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/4.jpg)
shards
curl -XPUT 'http://localhost:9200/twitter/' -d 'index: number_of_shards: 3'
![Page 5: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/5.jpg)
replicas
curl -XPUT 'http://localhost:9200/twitter/' -d 'index: number_of_shards: 3 number_of_replicas: 2'
![Page 6: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/6.jpg)
extremer extremes
![Page 7: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/7.jpg)
☁ Have at least 1 replica.☁ Make plenty of shards—but don’t go crazy.☁ 3
recommendations
discovery.zen.minimum_master_nodes: 2
![Page 8: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/8.jpg)
real-life examples
![Page 9: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/9.jpg)
☁ Protect with a firewall☁ discovery.zen.ping.multicast.enabled: false☁ discovery.zen.ping.unicast.hosts:
[“master1”, “master2”]☁ cluster.name: something_weird
too friendly
![Page 10: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/10.jpg)
downtime
discovery.zen.ping.unicast.hosts: ["a.example.com", "b.example.com"]
![Page 11: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/11.jpg)
be wary
![Page 12: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/12.jpg)
monitoring
curl -XGET -s 'http://localhost:9200/_cluster/health?pretty=true'{ "cluster_name" : "grinchyelasticsearch", "status" : "yellow", "timed_out" : false, "number_of_nodes" : 1, "number_of_data_nodes" : 1, "active_primary_shards" : 5, "active_shards" : 5, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 5}
![Page 13: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/13.jpg)
monitoring
http://karmi.github.com/elasticsearch-paramedic/
![Page 14: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/14.jpg)
optimization
![Page 15: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/15.jpg)
bootstrap.mlockall: true
![Page 16: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/16.jpg)
ES_HEAP_SIZE:half of RAM
![Page 17: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/17.jpg)
open files
/etc/security/limits.conf: es_user soft nofile 65535 es_user hard nofile 65535
/etc/init.d/elasticsearch: ulimit -n 65535 ulimit -l unlimited
✚
![Page 18: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/18.jpg)
Use default stores.
![Page 19: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/19.jpg)
RAM & JVM tuning
![Page 20: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/20.jpg)
shrinking indices% vmstat -S m -a 2procs -----------memory---------- ---swap-- -----io---- r b swpd free inact active si so bi bo 1 0 4 37 54 55 0 0 0 1 0 0 4 37 54 55 0 0 0 0 0 0 4 37 54 55 0 0 0 0
"some_doctype" : { "_source" : {"enabled" : false}}
"some_doctype" : { "_source" : {"compress" : true}}
"some_doctype" : { "_all" : {"enabled" : false}}
"some_doctype" : { "some_field" : {"include_in_all" : false}}
![Page 21: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/21.jpg)
filter caching
"filter": { "terms": { "tags": ["red", "green"], "execution": "plain" }}
"filter": { "terms": { "tags": ["red", "green"], "execution": "bool" }}
![Page 22: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/22.jpg)
dealing with the future
![Page 23: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/23.jpg)
mappings
![Page 24: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/24.jpg)
expensive updates
![Page 25: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/25.jpg)
☁ Use Bulk API.
how to reindex
☁ Turn off auto-refresh: curl -XPUT localhost:9200/test/_settings -d '{ "index" : { "refresh_interval" : "-1" } }'
☁ index.merge.policy.merge_factor: 1000☁ Remove replicas if you can.☁ Use multiple feeder processes.☁ Put everything back.
![Page 26: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/26.jpg)
☁ Small data: reindex☁ Big data: index.translog.disable_flush = true
backups
![Page 27: Es part 2 pdf no build](https://reader031.vdocument.in/reader031/viewer/2022020218/5593b4851a28ab456b8b4572/html5/thumbnails/27.jpg)
thank you
twitter: [email protected]
Background image by Tim and Julie Wilson: https://secure.flickr.com/photos/secondtree/.
This presentation is noncommercial sharealike in accordance with that image's license.