![Page 2: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/2.jpg)
Agenda
SaltStack ELK
2
![Page 3: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/3.jpg)
Filebeat
1.75 ‑> 5.3.2
SaltStack
3
![Page 4: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/4.jpg)
4
![Page 5: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/5.jpg)
5
![Page 6: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/6.jpg)
6
![Page 7: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/7.jpg)
……
7
![Page 8: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/8.jpg)
Filebeat
Packetbeat
Matricbeat/Collectd/NMon in Splunk
TBD: Kafka Driver
8
![Page 9: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/9.jpg)
ES
‑ Redis
ES Master/Data/Coordinating Node)
9
![Page 10: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/10.jpg)
10
![Page 11: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/11.jpg)
11
![Page 12: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/12.jpg)
Git
Gitlab
12
![Page 13: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/13.jpg)
Git
|____config
| |____prd
| | |____team_a
| | | |____filebeat-a-debug-log.yml
| | | |____filebeat-a-log-redis.yml
| | |____team_b
| | | |____filebeat-c-mongo.yml
| | | |____filebeat-c-redis.yml
| | | |____filebeat-c-scheduler.yml
| | | |____filebeat-c-mysql.yml
| | | |____filebeat-c-postgresql.yml
| | |____team_c
| | | |____filebeat-c-APP.yml
| | | |____filebeat-c-APP.yml
| | |____team_d
| | | |____filebeat-d-service1.yml
| | | |____filebeat-d-service2.yml
| | | |____filebeat-d-service3.yml
13
![Page 14: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/14.jpg)
Gitlab‑CI
Feature branch workflow
Gitlab‑CI vs. Jenkins
Precheck
logstash -t
filebeat -configtest
14
![Page 15: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/15.jpg)
15
![Page 16: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/16.jpg)
ELK
ELK
( /x‑pack monitoring)
16
![Page 17: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/17.jpg)
{% set appname = "metricbeat" %}
{{ appname }}:
pkg.installed:
- require:
- pkgrepo: {{ repo_name }}
file.managed:
- name: /etc/{{ appname }}/{{ appname }}.yml
- source: salt://fw-{{ appname }}/{{ appconfig }}
- template: jinja
17
![Page 18: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/18.jpg)
{{ appname }}-service:
service.running:
- name: {{ appname }}
- enbale: True
- reload: True
- require:
- pkg: {{ appname }}
- file: {{ appname }}-permission
- file: {{ appname }}
- watch:
- file: {{ appname }}
18
![Page 19: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/19.jpg)
supervisor:
pkg.installed: []
file.managed:
- name: /etc/supervisord.d/logstash.ini
- contents: |+
{% for path in paths %}{% set pname = get_filename(path) %} [program:{{ pname }}]
command=logstash --path.settings /opt/logstash/etc/ \
-f {{ path }} --path.data /opt/logstash/var/
-n {{ grains['fqdn'] }}:{{ pname }}
directory=/opt/logstash/
user=logstash
environment=LS_HEAP_SIZE=256m
{% endfor %}
- require:
- file: logstash
19
![Page 20: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/20.jpg)
elasticsearch:
file.managed:
- name: /etc/elasticsearch/elasticsearch.yml
- source: salt://es/elasticsearch-5.y/elasticsearch.yml
- template: jinja
cmd.run:
- name: elasticsearch-plugin install x-pack
- unless: elasticsearch-plugin list | grep x-pack
service.running:
- enbale: True
- reload: True
- require:
- pkg: elasticsearch-pkg
- file: elasticsearch
- file: elasticsearch-link
- watch:
- file: elasticsearch
20
![Page 21: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/21.jpg)
21
![Page 22: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/22.jpg)
Salt‑API timeout
LogStash
ElastAlert
Filebeat RHEL5
22
![Page 23: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/23.jpg)
Filebeat RHEL5
diff -urpN output_orig/etc/init.d/filebeat output/etc/init.d/filebeat
--- output_orig/etc/init.d/filebeat+++ output/etc/init.d/filebeat@@ -27,8 +27,13 @@ pidfile=${PIDFILE-/var/run/filebeat.pid}
-wrapper="/usr/share/filebeat/bin/filebeat-god"
-wrapperopts="-r / -n -p $pidfile"
+if grep -q -i "release 7" /etc/redhat-release; then
+ wrapper="/usr/share/filebeat/bin/filebeat-god"
+else
+ wrapper="/usr/share/filebeat/bin/filebeat-god-el5"
+fi
+beat_user="monitor"
+wrapperopts="-r / -n -p $pidfile -u $beat_user"
RETVAL=0
test() {
- $agent $args $test_args
+ runuser -s /bin/bash $beat_user -c "$agent $args $test_args"
}
23
![Page 24: ELK CI/CD - itdks.su.bcebos.comMetricBeat Container Monitoring ELK ‑ Dockerize 24. Q&A 25. Created Date: 5/20/2017 7:30:51 PM](https://reader035.vdocument.in/reader035/viewer/2022071511/613047671ecc51586943fe86/html5/thumbnails/24.jpg)
Kafka
MetricBeat
Container Monitoring
ELK ‑ Dockerize
24