![Page 1: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/1.jpg)
KAFKA EAGLEA EASY AND HIGH-PERFORMANCE MONITORING SYSTEM
AUTHOR : SMARTLOLI UPDATE : 2019.07.14
![Page 2: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/2.jpg)
01 Part IThe Background of Kafka Eagle
02 Part IIDesign Architecture of Kafka Eagle
03 Part IIIThe Advantages of Kafka Eagle
04 Part IVFollow-up Planning of Kafka Eagle
MENU
![Page 3: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/3.jpg)
01Part I
The Background of Kafka Eagle
![Page 4: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/4.jpg)
Background
01
02
03
04Convenience
It is difficult to configure and use the existing monitoring system.
Apache Kafka
Apache Kafka does not officially provide monitoring systems or pages.
Open Source
Open source Kafka monitoring system has too few functions or suspension of maintenance.
IM Support
Some monitoringsystem can not satisfy integration with existing IM, such as wechat ordingding.
![Page 5: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/5.jpg)
02Part II
Design Architecture of Kafka Eagle
![Page 6: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/6.jpg)
Architecture
02
01
0305
06
Visualization1. Topic list, rate, consumer2. Cluster heathy3. TBD
ACLs1. Topic create,detele etc2. User manage
Alert1. E-Mail2. WeChat3. DingDing
Collector
1. Zookeeper2. Kafka jmx & external topic3. Kafka api (2.x+)
Storage
1. MySQL2. SQLite
Metrics
1. Consumer application2. Cluster heathy
KafkaEagle
04
![Page 7: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/7.jpg)
03Part III
The Advantages of Kafka Eagle
![Page 8: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/8.jpg)
Advantages-Convenience
https://www.kafka-eagle.org/
Download
https://docs.kafka-eagle.org/
Install
ke.sh start
QuickStart
http://ip:port/ke
Visit
Step 1 Step 3Step 2 Step 4
![Page 9: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/9.jpg)
Advantages-Extensive
DashboardKafka Eagle
Topic
Cluster
Alarm
System
The kafka eagle system has favorableuser interface, powerfulfunction modules.
Consumers Metrics
![Page 10: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/10.jpg)
Dashboard
Show the scale of clusters, themes, consumers, etc.
Show kafka cluster survival nodes.
![Page 11: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/11.jpg)
Topic
1. Create topic.
2. List topic.
3. SQL query.
4. Mock data.
5. Config topic.
![Page 12: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/12.jpg)
Consumers
Display consumer group application details.
Display consumer active topics details.
![Page 13: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/13.jpg)
Cluster
1. Show details.
2. Multi-cluster.
3. ZK client.
![Page 14: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/14.jpg)
Metrics
1. Broker mbean.
2. Kafka metrics.
3. ZK metrics.
![Page 15: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/15.jpg)
Alarm
1. Add lag alarm.
2. Modify lag.
3. Add cluster.
4. Modify cluster.
![Page 16: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/16.jpg)
System
1. Add user.
2. Assign role.
3. Manage res.
![Page 17: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/17.jpg)
04Part IV
Follow-up Planning of Kafka Eagle
![Page 18: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/18.jpg)
Planning
Dimension 2Dimension 1 Dimension 4Dimension 3
Kafka Eagle follow-up plan is divided into four dimensions.
QAP
Continuous optimization of system modules of Kafka Eagle.
DSR
Collect Github issues problems and fix them.
PPL
Developing new functions of Kafka Eagle system for actual needs.
PHB
Organize and file the user manual of Kafka Eagle system.
![Page 19: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/19.jpg)
About Me
Kafka Eagle
Blog
PublishBooks
ME
Hadoop: https://item.jd.com/12455361.htmlKafka: https://item.jd.com/12371763.html
https://www.cnblogs.com/smartloli/
http://www.kafka-eagle.org/https://github.com/smartloli/kafka-eagle/
![Page 20: Kafka Eagle Architecturepdf.kafka-eagle.org/file/Kafka_Eagle_Architecture.pdf · 2019. 8. 27. · Apache Kafka does not officially provide monitoring systems or pages. OpenSource](https://reader033.vdocument.in/reader033/viewer/2022053119/609f5e833d084b5db2219372/html5/thumbnails/20.jpg)
THANK YOU FOR WATCHINGhttp://www.kafka-eagle.org/
2 0 1 9
AUTHOR : SMARTLOLI UPDATE : 2019.07.14