message queue demo
TRANSCRIPT
![Page 1: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/1.jpg)
Message Queues Demo
![Page 2: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/2.jpg)
Apache Kafka
![Page 3: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/3.jpg)
StartStart zookeeper:
bin/zookeeper-server-start.sh config/zookeeper.properties
Start brokers:
bin/kafka-server-start.sh config/server.properties
bin/kafka-server-start.sh config/server1.properties
bin/kafka-server-start.sh config/server2.properties
![Page 4: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/4.jpg)
![Page 5: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/5.jpg)
Create a topic
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic demo
bin/kafka-topics.sh --list --zookeeper localhost:2181
![Page 6: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/6.jpg)
Create a producer
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic demo
![Page 7: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/7.jpg)
Create a consumer
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --from-beginning --topic demo
![Page 8: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/8.jpg)
![Page 9: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/9.jpg)
Kill a brokerbin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic demo
Then kill a leader broker…
bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic demo
Check available messages…bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --from-beginning --topic demo
![Page 10: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/10.jpg)
Kafka Streams for data processingLet’s create a file…
echo -e "all streams lead to kafka\nhello kafka streams\njoin kafka summit" > file-input.txt
...and then create a topic…
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic streams-file-input
...and publish data to this topic…
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic streams-file-input < file-input.txt
![Page 11: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/11.jpg)
Kafka Streams for data processingLet’s run an analytics…
bin/kafka-run-class.sh org.apache.kafka.streams.examples.wordcount.WordCountDemo
And see results in output topic:
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic streams-wordcount-output --from-beginning --property print.key=true --property value.deserializer=org.apache.kafka.common.serialization.LongDeserializer
![Page 12: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/12.jpg)
![Page 13: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/13.jpg)
Kafka Streams for data processingWordCountDemo:
KTable wordCounts = textLines // Split each text line, by whitespace, into words. .flatMapValues(value -> Arrays.asList(value.toLowerCase().split("\\W+")))
// Ensure the words are available as record keys for the next aggregate operation. .map((key, value) -> new KeyValue<>(value, value))
// Count the occurrences of each word (record key) and store the results into a table named "Counts". .countByKey("Counts")
![Page 14: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/14.jpg)
RabbitMQ
![Page 15: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/15.jpg)
Startsbin/rabbitmq-server -detached
nano etc/rabbitmq/rabbitmq.config
sbin/rabbitmqctl status
Web UI:sbin/rabbitmq-plugins enable rabbitmq_management
http://localhost:15672/
Management HTTP API:http://localhost:15672/api/
![Page 16: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/16.jpg)
Sending data
![Page 17: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/17.jpg)
Receive data
![Page 18: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/18.jpg)
What happened
![Page 19: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/19.jpg)
Routing
![Page 20: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/20.jpg)
Exchange typesExchanges - entities where messages are sent.
They take a message and route it into zero or more queues. The routing algorithm used depends on the exchange type and rules called bindings.
Types:
Direct
Fanout
Topic
Headers
![Page 21: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/21.jpg)
Direct exchange
![Page 22: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/22.jpg)
Fanout exchange
![Page 23: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/23.jpg)
Topic Exchange
![Page 24: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/24.jpg)
Clients
![Page 25: Message queue demo](https://reader034.vdocument.in/reader034/viewer/2022051709/5877cea11a28ab39588b747d/html5/thumbnails/25.jpg)
Q&A