spark streaming - the simple way

Spark Streaming

What is Spark Streaming in picturesYogesh Kumar

Author - 99 Apache Spark Interview Questions for Professionals

Yogesh Kumar

99 Apache Spark Interview Questions for Professionals

Spark Application step 0

99 Apache Spark Interview Questions for Professionals

Spark Application step 1

99 Apache Spark Interview Questions for Professionals

Spark Application step 2

99 Apache Spark Interview Questions for Professionals

Streaming Step 1

99 Apache Spark Interview Questions for Professionals

Streaming Step 2

99 Apache Spark Interview Questions for Professionals

Streaming Step 3

99 Apache Spark Interview Questions for Professionals

Streaming Step 4

99 Apache Spark Interview Questions for Professionals

Streaming Step 5

99 Apache Spark Interview Questions for Professionals

What if an executor fails?

99 Apache Spark Interview Questions for Professionals

Automatic Recovery

99 Apache Spark Interview Questions for Professionals

What if the driver fails?

99 Apache Spark Interview Questions for Professionals

Recovering Driver w/ DStream Checkpointing

99 Apache Spark Interview Questions for Professionals

Recovering Driver w/ DStream Checkpointing

99 Apache Spark Interview Questions for Professionals

Recovering Driver w/ DStream Checkpointing

99 Apache Spark Interview Questions for Professionals

Received blocks lost on Restart!

99 Apache Spark Interview Questions for Professionals

Recovering data with Write Ahead Logs

99 Apache Spark Interview Questions for Professionals

Recovering data with Write Ahead Logs

99 Apache Spark Interview Questions for Professionals

Zero data loss = every stage processes each

event at least once despite any failure