apache flink - a stream processing engine

1

Aljoscha Krettek@aljoscha

December 8, 2016Google Singapore

Apache Flink™A Stream Processing Engine

2

Original creators of Apache Flink®

Providers of the dA Platform, a supported

Flink distribution

What is Apache Flink?

3

Distributed Deployment

Stateful Stream Processing

DataStream API, Windows

Table API

Building Block

Core API

Declarative DSL

Basis

Stream Processing

4

Data Stream Data Stream

Your code

Stateful Stream Processing

5

State

Data Stream Data Stream

Your code

Stateful Steam Processing

6

Source Filter /Transform

Stateread/write Sink

Stateful Steam Processing

7

Scalable embedded state Access at memory speed &scales with parallel operators

Stateful Steam Processing

8

Re-load state

Reset positionsin input streams

Rolling back computationRe-processing

Stateful Steam Processing

9

Restore to differentprograms

Bugfixes, Upgrades, A/B testing, etc

Beam/Flink Interactions

10

Dataflow/MillWheel influenced

runs on

Code Example!

11

Based on http://data-artisans.com/robust-stream-processing-flink-walkthrough/

12

Thank you!@aljoscha@ApacheFlink @dataArtisanshttp://flink.apache.org

13

Appendix

We are hiring!

data-artisans.com/careers

15

Flink Forward 2017

San Francisco Berlin• April 10-11, 2017• The first Flink Forward

event outside of Berlin• Register at sf.flink

-forward.org/

• September 11-13• Over 350 attendees at

the last event• Registration opening

soon!