confluent and syncsort webinar august 2016
TRANSCRIPT
1Confidential
IntroductionTo Real Time Streaming with Apache Kafkaand Syncsort DMX Connectivity
2Confidential
What does Kafka do?
Producers
Consumers
Kafka Connect
Kafka Connect
Topic
Your interfaces to the world
Connected to your systems in real time
3Confidential
What is a Stream Data Platform?
KafkaStream Data
Platform
Search
NoSQL
RDBMS Monitoring
Stream ProcessingReal-time Analytics Data Warehouse
Apps
Apps
Hadoop
Synchronous Req/Response0 – 100s ms
Near Real Time> 100s ms
Offline Batch> 1 hour
Build streaming applications
Deploy streaming applications at scale
Monitor and manage streaming applications
Common Kafka Use Cases• Log data• Database changes• Sensors and device
data• Monitoring streams• Call data records
• Real-time Monitoring• Asynchronous
applications• Fraud and security• Bridge to Cloud
4Confidential
Companies Using Kafka TodayFinancial Services
Entertainment & Media
Consumer Tech
Travel & Leisure
Enterprise Tech
Telecom Retail
5Confidential
Confluent Platform: It’s Kafka ++Feature Benefit Apache Kafka Confluent Platform
3.0Confluent Enterprise
3.0
Apache Kafka High throughput, low latency, high availability, secure distributed message system
Kafka Connect Advanced framework for connecting external sources and destinations into Kafka
Java Client Provides easy integration into Java applications
Kafka Streams Simple library that enables streaming application development within the Kafka framework
Additional Clients Supports non-Java clients; C, C++, Python, etc.
Rest Proxy Provides universal access to Kafka from any network connected device via HTTP
Schema Registry Central registry for the format of Kafka data – guarantees all data is always consumable
Pre-Built Connectors HDFS, JDBC and other connectors fully Certified and fully supported by Confluent
Confluent Control Center Includes Connector Management and Stream Monitoring
Support Connection and Monitoring command center provides advanced functionality and control Community Community 24x7x365
Free Free Subscription
DMX
6Confidential
Confluent Platform with Syncsort Connectivity
Confluent Platform
Alerting
Monitoring
Real-time Analytics
Custom Application
Transformations
Real TimeApplication
s
Apache Kafka CoreConnectors
Control Center Clients & Developer Tools
Hadoop
ERP
CRM
Data Warehouse
RDBMS
Data Integration
Connectors
Database Changes Mobile DevicesloTLogs Website Events
Confluent Platform Confluent Platform Enterprise External Product
Support, Services and Consulting
Kafka Streams
DMX Source
DMX Sink
7Confidential
Connector Hub: http://www.confluent.io/product/connectors• Confluent-supported connectors (included in CP)
• Partner/Community-written connectors (just a sampling)
JDBC
Source - SupportedSink – In progress
8Confidential
Confluent Control Center
Configures Kafka Connect data pipelines
Monitors all pipelines from end-to-end