kafka eagle architecturepdf.kafka-eagle.org/file/kafka_eagle_architecture.pdf · 2019. 8. 27. ·...
TRANSCRIPT
KAFKA EAGLEA EASY AND HIGH-PERFORMANCE MONITORING SYSTEM
AUTHOR : SMARTLOLI UPDATE : 2019.07.14
01 Part IThe Background of Kafka Eagle
02 Part IIDesign Architecture of Kafka Eagle
03 Part IIIThe Advantages of Kafka Eagle
04 Part IVFollow-up Planning of Kafka Eagle
MENU
01Part I
The Background of Kafka Eagle
Background
01
02
03
04Convenience
It is difficult to configure and use the existing monitoring system.
Apache Kafka
Apache Kafka does not officially provide monitoring systems or pages.
Open Source
Open source Kafka monitoring system has too few functions or suspension of maintenance.
IM Support
Some monitoringsystem can not satisfy integration with existing IM, such as wechat ordingding.
02Part II
Design Architecture of Kafka Eagle
Architecture
02
01
0305
06
Visualization1. Topic list, rate, consumer2. Cluster heathy3. TBD
ACLs1. Topic create,detele etc2. User manage
Alert1. E-Mail2. WeChat3. DingDing
Collector
1. Zookeeper2. Kafka jmx & external topic3. Kafka api (2.x+)
Storage
1. MySQL2. SQLite
Metrics
1. Consumer application2. Cluster heathy
KafkaEagle
04
03Part III
The Advantages of Kafka Eagle
Advantages-Convenience
https://www.kafka-eagle.org/
Download
https://docs.kafka-eagle.org/
Install
ke.sh start
QuickStart
http://ip:port/ke
Visit
Step 1 Step 3Step 2 Step 4
Advantages-Extensive
DashboardKafka Eagle
Topic
Cluster
Alarm
System
The kafka eagle system has favorableuser interface, powerfulfunction modules.
Consumers Metrics
Dashboard
Show the scale of clusters, themes, consumers, etc.
Show kafka cluster survival nodes.
Topic
1. Create topic.
2. List topic.
3. SQL query.
4. Mock data.
5. Config topic.
Consumers
Display consumer group application details.
Display consumer active topics details.
Cluster
1. Show details.
2. Multi-cluster.
3. ZK client.
Metrics
1. Broker mbean.
2. Kafka metrics.
3. ZK metrics.
Alarm
1. Add lag alarm.
2. Modify lag.
3. Add cluster.
4. Modify cluster.
System
1. Add user.
2. Assign role.
3. Manage res.
04Part IV
Follow-up Planning of Kafka Eagle
Planning
Dimension 2Dimension 1 Dimension 4Dimension 3
Kafka Eagle follow-up plan is divided into four dimensions.
QAP
Continuous optimization of system modules of Kafka Eagle.
DSR
Collect Github issues problems and fix them.
PPL
Developing new functions of Kafka Eagle system for actual needs.
PHB
Organize and file the user manual of Kafka Eagle system.
About Me
Kafka Eagle
Blog
PublishBooks
ME
Hadoop: https://item.jd.com/12455361.htmlKafka: https://item.jd.com/12371763.html
https://www.cnblogs.com/smartloli/
http://www.kafka-eagle.org/https://github.com/smartloli/kafka-eagle/
THANK YOU FOR WATCHINGhttp://www.kafka-eagle.org/
2 0 1 9
AUTHOR : SMARTLOLI UPDATE : 2019.07.14