voxxed days vienna 2017 - microservices with kafka
TRANSCRIPT
albertlaszlorobert #VoxxedVienna
FAULT
TOLERANCE
PE
RF
OR
MA
NC
E
RELIABILITY SECURITY
AVAILABILITY
HIGH
SCALABILITY
REPLICATION
CH
AN
GE SELF HEALING
albertlaszlorobert #VoxxedVienna
Source: http://www.slideshare.net/dberkholz/
how-microservices-are-redefining-
modern-application-architecture
albertlaszlorobert #VoxxedVienna
Source: http://www.slideshare.net/dberkholz/
how-microservices-are-redefining-
modern-application-architecture
albertlaszlorobert #VoxxedVienna
500 million messages / day 700 billion messages / day 1.4 trillion messages / day
albertlaszlorobert #VoxxedVienna
500 million messages / day 700 billion messages / day 1.4 trillion messages / day
albertlaszlorobert #VoxxedVienna
Messaging Brokers Features
Support for multiple protocols
• Advanced Message Queueing Protocol
• Streaming Text Oriented Messaging Protocol
• Message Queuing Telemetry Transport
• Etc
Kafka ActiveMQ RabbitMQ
albertlaszlorobert #VoxxedVienna
Messaging Brokers Features
Support for multiple protocols
• Advanced Message Queueing Protocol
• Streaming Text Oriented Messaging Protocol
• Message Queuing Telemetry Transport
• Etc
Kafka ActiveMQ RabbitMQ
Java Messaging System
albertlaszlorobert #VoxxedVienna
Messaging Brokers Features
Support for multiple protocols
• Advanced Message Queueing Protocol
• Streaming Text Oriented Messaging Protocol
• Message Queuing Telemetry Transport
• Etc
Kafka ActiveMQ RabbitMQ
Java Messaging System
Java Authentication and Authorization Service
Simple Authentication and Security Layer
albertlaszlorobert #VoxxedVienna
Messaging Brokers Features
Support for multiple protocols
• Advanced Message Queueing Protocol
• Streaming Text Oriented Messaging Protocol
• Message Queuing Telemetry Transport
• Etc
Kafka ActiveMQ RabbitMQ
Java Messaging System
Java Authentication and Authorization Service
Simple Authentication and Security Layer
Transactions
albertlaszlorobert #VoxxedVienna
Messaging Brokers Features
Support for multiple protocols
• Advanced Message Queueing Protocol
• Streaming Text Oriented Messaging Protocol
• Message Queuing Telemetry Transport
• Etc
Kafka ActiveMQ RabbitMQ
Java Messaging System
Java Authentication and Authorization Service
Simple Authentication and Security Layer
Transactions
Filtering
albertlaszlorobert #VoxxedVienna
Messaging Brokers Features
Support for multiple protocols
• Advanced Message Queueing Protocol
• Streaming Text Oriented Messaging Protocol
• Message Queuing Telemetry Transport
• Etc
Kafka ActiveMQ RabbitMQ
Java Messaging System
Java Authentication and Authorization Service
Simple Authentication and Security Layer
Transactions
Filtering
Persistence
albertlaszlorobert #VoxxedVienna
Messaging Brokers Performance
Message producing
2
7
Message consumption
LinkedIn – Kafka: a Distributed Messaging System for Log Processing
albertlaszlorobert #VoxxedVienna
Producer
© ING 2016
// build a 'default' factory based on your-implicit-kafka-producer.properties
// - the properties file contains the list of brokers to connect to
ProducerFactory factory = ProducerFactory.builder().build();
// create a message producer with a Payload that sends the messages to topic ‘test’
Producer<Message<Payload>> producer =
factory.createMessageProducer("test", Payload.class);
// send a Message { header: Map<String, String>; payload: Payload }
producer.send(
Message.<Payload> builder()
.header("applicationId", "sample-producer")
.header("timestamp", now())
.payload(new Payload("/some/file/path"))
.build()
);
albertlaszlorobert #VoxxedVienna
Producer
your-implicit-producer.properties
your-security.properties
albertlaszlorobert #VoxxedVienna
Consumer // build a 'default' factory – your-implicit-kafka-consumer.properties
// - the properties contains the list of brokers from which it will receive messages
ConsumerFactory factory = ConsumerFactory.builder().build();
// obtain a consumer instance that can process Message<Payload> messages
Consumer<Message<Payload>> consumer = factory.createMessageConsumer(Payload.class);
// prepare the message (Message { header: Map<String, String>; payload: Payload } ) handler
ReceiverHandler<Message<Payload>> messageReceiver =
new ReceiverHandler<Message<Payload>>() {
@Override
public void onReceive(Message<Payload> message) {
System.out.printf("message received: %s%n", message);
}
};
// register the handler to receive messages from the topic named “test”
consumer.registerHandler("test", messageReceiver);
© ING 2016
albertlaszlorobert #VoxxedVienna
Consumer
your-implicit-consumer.properties
your-security.properties
albertlaszlorobert #VoxxedVienna
Use microservices with Kafka
… code ahead of the curve
Vienna • 16 March 2017
László-Róbert ALBERT https://www.linkedin.com/in/albertlaszlorobert