aws simple workflow: distributed out of the box! - morning@lohika

Distributed Out of the Box!AWS Simple Workflow

Serhiy Batyuk Morning@Lohika

May 14, 2016

My Interests

Tech Lead at

AWS SWF 1 year>

Agenda

SWF Overview

Agenda

SWF Overview Demo

Agenda

SWF Overview Demo Our Experience

What Is AWS Simple Workflow?

AWS SWF Use Cases

https://www.youtube.com/watch?v=DYmJIQO2ZyQ&t=83

Media Processing Infrastructure Provisioning

Business ProcessesReport GenerationData Processing

https://globallogic.com.ua/speaking-event/jeeconf-2015

Preparing to Attend JEEConf

Preparing to Attend JEEConf

Preparing to Attend JEEConf

Preparing to Attend JEEConf

Preparing to Attend JEEConf

PROGRAM

STEPS

Preparing to Attend JEEConf

Preparing to Attend JEEConf

Preparing to Attend JEEConf

Preparing to Attend JEEConf

AWS SWF

https://aws.amazon.com/swf

Workflows and Activities

WORKFLOW

ACTIVITIES

SWF Application Architecture

WorkerDecider

SWF Application Architecture

WorkerDecider

AWS SWF

AWS SWF

AWS SWF Message Flow

WorkerDecider

AWS SWF

AWS SWF Message Flow

WorkerDecider

START

AWS SWF

AWS SWF Message Flow

WorkerDecider

MAKE A DECISION

AWS SWFGET

HISTORY

AWS SWF Message Flow

WorkerDecider

AWS SWF

AWS SWF Message Flow

WorkerDecider

BUY TICKET

AWS SWF

AWS SWF Message Flow

WorkerDecider

BUY TICKET

AWS SWF

AWS SWF Message Flow

WorkerDecider

TICKET CONFIRMATION

AWS SWF

AWS SWF Message Flow

WorkerDecider

AWS SWF

AWS SWF Message Flow

WorkerDecider

MAKE A DECISION

AWS SWFGET

HISTORY

AWS SWF Message Flow

WorkerDecider

AWS SWF

AWS SWF Message Flow

WorkerDecider

BOOK HOTEL

AWS SWF

AWS SWF Message Flow

WorkerDecider

BOOK HOTEL

AWS SWF

AWS SWF Message Flow

WorkerDecider

HOTEL CONFIRMATION

AWS SWF

AWS SWF Message Flow

WorkerDecider

AWS SWF

AWS SWF Message Flow

WorkerDecider

AWS SWF

AWS SWF Message Flow

WorkerDecider

RESULT

Task Lists and Routing

WorkerDecider

SWF

AWS SWF API

PollForDecisionTask GetWorkflowExecutionHistory RespondDecisionTaskCompleted

PollForActivityTask RespondActivityTaskCompleted RespondActivityTaskFailed

StartWorkflowExecution

http://docs.aws.amazon.com/amazonswf/latest/apireference

AWS SWF API

RegisterWorkflowType

RegisterActivityType

http://docs.aws.amazon.com/amazonswf/latest/apireference

AWS SWF Flow Framework

AWS SWFSWF API

.Net, Python, others

SWF Flow Framework

Java, Ruby

Java Flow Framework

Workflow Worker

Decider

Workflow Implementation

Activities Client

Activities Worker

Worker

Activities Implementations

Workflow Starter

Client app

Workflow Client

SWF

http://docs.aws.amazon.com/amazonswf/latest/awsflowguide/awsflow-basics-application-structure.html

Java Flow Framework

Workflow Worker

Decider

Activities Client

Activities Worker

WorkerClient app

Workflow Client

SWF

http://docs.aws.amazon.com/amazonswf/latest/awsflowguide/awsflow-basics-application-structure.html

Java Flow Framework

Decider

Workflow Implementation

Worker

Activities Implementations

Workflow Starter

Client app

SWF

http://docs.aws.amazon.com/amazonswf/latest/awsflowguide/awsflow-basics-application-structure.html

Java Flow Framework Classes

@Activities

@Workflow Client to schedule workflows

Client to schedule activities

Java Flow Framework Classes

@Activities

@Workflow Client to schedule workflows

Client to schedule activities

WorkflowWorker

ActivitiesWorker

Handle decision tasks

Handle activity tasks

DemoSWF

Visit Organizer Workflow

Visit Organizer Workflow

Visit Organizer Workflow

Visit Organizer Workflow

Preparing for JEEConf

Visit Organizer

Service

JEEConf Service

Travel Service

Preparing for JEEConf

Visit Organizer

Service

JEEConf Service

Travel Service

Buy tickets

Preparing for JEEConf

Visit Organizer

Service

JEEConf Service

Travel Service

Book hotelBuy

tickets

Preparing for JEEConf

Visit Organizer

Service

JEEConf Service

Travel Service

Book train tickets

Book hotelBuy

tickets

Time to see the code!

public class HelloWorld {

public static void main() { System.out.println(“Hi”); }

}

https://github.com/sbatyuk/aws-swf-sample

http://info.thinkfun.com/stem-education/mark-engelberg-inventor-of-code-master-featured-in-getting-smart

EASY, RIGHT?

FeaturesSWF

Fully Managed Service

Worker

Worker

Worker

Decider

Decider

Scalable

AWS SWF

My App

http://docs.aws.amazon.com/amazonswf/latest/awsflowguide/awsflow-basics-scalable.html

Worker

Worker

Worker

Decider

Decider

Scalable

AWS SWF

My App

http://docs.aws.amazon.com/amazonswf/latest/awsflowguide/awsflow-basics-scalable.html

Worker

Worker

Worker

Decider

Decider

Activity Retries

AWS SWF

My App

http://docs.aws.amazon.com/amazonswf/latest/awsflowguide/features-retry.html

Worker

Worker

Worker

Decider

Decider

Activity Retries

AWS SWF

My App

http://docs.aws.amazon.com/amazonswf/latest/awsflowguide/features-retry.html

Worker

Worker

Worker

Decider

Decider

Activity Retries

AWS SWF

My App

http://docs.aws.amazon.com/amazonswf/latest/awsflowguide/features-retry.html

Workflow Execution History

Exactly Once Delivery

= 1

Integration with Spring and JUnit

http://docs.aws.amazon.com/amazonswf/latest/awsflowguide/test.html

Other Features

Child workflows

Lamda tasks

Heartbeats

Continuous workflows

Timers

Signals

AWS SWF Pricing

https://aws.amazon.com/swf/pricing

10,000 workflows in a day with 3 activities each:

$1.75

Lessons learntSWF

Workflow Replays

http://docs.aws.amazon.com/amazonswf/latest/awsflowguide/awsflow-basics-distributed-execution.html

@Overridepublic void prepareForJEEConf(String name, String email) { Promise<Integer> ticket = jeeConfService.buyTicket(name); Promise<Integer> hotel = travelService.bookHotel(name, ticket); Promise<Integer> train = travelService.bookTrainTickets(name, ticket); sendConfirmationEmail(email, ticket, hotel, train);}

Workflow Replay #1Workflow Activity Client Activity Workflow History

Workflow Replay #1Workflow Activity Client Activity Workflow History

Get history

Workflow Replay #1Workflow Activity Client Activity Workflow History

History

Get history

Workflow Replay #1

Buy ticket

Workflow Activity Client Activity Workflow History

History

Get history

Workflow Replay #1

Buy ticketBuy ticket

Workflow Activity Client Activity Workflow History

History

Get history

Workflow Replay #1

Buy ticket

PromiseBuy ticket

Workflow Activity Client Activity Workflow History

History

Get history

Workflow Replay #1

Buy ticket

PromiseBuy ticket

Book hotel

Workflow Activity Client Activity Workflow History

History

Get history

Workflow Replay #1

Buy ticket

PromiseBuy ticket

Book hotel

Promise

Workflow Activity Client Activity Workflow History

History

Get history

Workflow Replay #1

Buy ticket

PromiseBuy ticket

Book hotel

Promise

Ticket confirmation

Workflow Activity Client Activity Workflow History

History

Get history

Workflow Replay #2Workflow Activity Client Activity Workflow History

Workflow Replay #2Workflow Activity Client Activity Workflow History

Get history

Workflow Replay #2Workflow Activity Client Activity Workflow History

History

Get history

Workflow Replay #2

Buy ticket

Workflow Activity Client Activity Workflow History

History

Get history

Workflow Replay #2

Buy ticket

Workflow Activity Client Activity Workflow History

History

Get history

Promise with confirmation

Workflow Replay #2

Buy ticket

Book hotel

Workflow Activity Client Activity Workflow History

History

Get history

Promise with confirmation

Workflow Replay #2

Buy ticket

Book hotelBook hotel

Workflow Activity Client Activity Workflow History

History

Get history

Promise with confirmation

Workflow Replay #2

Buy ticket

Book hotelBook hotel

Promise

Workflow Activity Client Activity Workflow History

History

Get history

Promise with confirmation

Workflow Replay #2

Buy ticket

Book hotelBook hotel

Promise

Hotel confirmation

Workflow Activity Client Activity Workflow History

History

Get history

Promise with confirmation

Activity Timeouts

Activity Heartbeats

Activity Heartbeats

Serialization of Exceptions

1 1 1 0 1 0 0 0 1 1 0 1 1 1 0 1 0 1 0 1 0 1 1 1 0 1 0 0 0 1 1 1 0 1 0 0 1 0 1 0 0 0 1 0 0 1 0 0 0 0 0 0 1 0 0 1 1 1 1 0 0 1 0 0 0 0 1 1 0 1 1 1 1 0 1 1 0 0 1 0 1 1 0 0 0 1 1 0 1 0 0 1 1 0 0 0 0 1 0 0 1 1 0 1 1 1 1 1 0 1 1 0 0 1 0 0 1 1 0 1 0 0 0 1 1 1 1 0 1 1 0 0 1 0 0 1 1 1 0 0 1 1 0 0 0 1 1 0 1 1 1 0 0 0 1 0 0 0 1 1 1 1 1 0 0 1 0 1 0 0 0 1 1 1 1 0 0 1 0 0 1 1 1 0 0 0 0 1 0 1 0 1 1 1 1 0 0 0 0 1 1 0 0 1 0 1 0 0 1 1 0 1 0 0 1 1 1 0 1 1 0 0 0 0 0 0 1 0 0 0 1 1 1 0 0 0 0 0 0 0 1 0 1 1 1 0 1 1 1 1 0 0 1 0 1 1 1 1 0 1 1 0 0 1 0 1 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 1 0 1 1 1 0 1 1 1 0 0 0 0 1 1 0 0 0 1 1 0 0 1 0 1 0 1 0 0 0 0 1 0 1 1 1 1 1 1 1 1 1 1 0 0 0 1 0 0 1 0 1 1 1 0 1 1 1 1 1 1 0 0 1 0 0 0 0 0 1 0 0 0 1 0 0 1 1 0 1 1 1 0 1 0 1 0 1 1 1 0 0 0 1 0 0 0 1 1 0 1 0 1 0 1 1 1 1 0 1 0 1 0 0 0 0 1 0 1 1 0 1 0 0 0 1 1 1 1 0 0 0 0 0 0 1 0 1 0 0 0 1 1 1 0 1 1 0 1 0 0 0 1 1 0 1 0 1 0 1 0 1 1 1 1 0 0 0 1 0 0 0 0 0 0 0 0 0 1 1 1 1 0 1 1 0 0 0 1 0 0 0 1 1 1 1 1 1 0 1 1 0 0 0 0 0 0 1 1 0 0 1 1 0 1 1 1 1 0 0 1 1 1 1 0 0 0 1 1 1 0 1 0 1 0 0 1 1 1 1 1 0 0 1 0 0 0 1 0 1 0 0 0 1 0 0 1 1 1 1 0 0 1 0 0 1 1 0 0 0 1 0 0 0 1 1 0 0 0 0 0 0 1 0 1 0 1 0 0 1 1 1 0 1 1 0 0 0 0 1 1 1 0 0 0 1 0 1 0 1 1 0 1 1 1 0 1 0 0 1 0 1 1 1 0 0 1 1 0 1 1 0 1 0 1 0 0 1 1 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0 0 1 0 1 0 0 0 0 0 1 1 0 0 1 0 1 1 0 1 0 0 0 0 0 0 1 1 0 1 1 0 1 1 1 0 1 1 0 0 1 0 0 1 0 0 0 0 1 1 0 1 0 0 0 1 0 1 1 0 1 0 1 1 1 1 0 0 0 0 0 1 1 0 0 1 1 1 1 0 0 1 0 1 1 1 0 0 0 0 0 0 1 0 1 0 0 1 1 0 1 1 1 0 0 1 1 0 1 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 0 0 0 0 0 1 1 1 0 0 1 1 0 0 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0 1 0 0 1 1 1 0 1 0 0 0 0 1 1 1 1 1 0 1 1 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 1 0 0 1 0 0 1 1 1 1 1 0 0 1 1 0 0 0 0 1 0 1 0 0 1 1 0 0 1 0 1 1 1 0 1 1 0 0 0 0 1 1 1 1 0 1 0 1 0 1 1 0 0 0 0 1 1 1 1 1 0 1 1 0 1 1 0 1 0 1 0 0

History and Data Size Limits

25K

http://docs.aws.amazon.com/amazonswf/latest/developerguide/swf-dg-limits.html

History events

32K

Message size (chars)

Summary

Programs with steps Scalability

Timeouts & Retries Java & Ruby

Questions

Serhiy Batyuk sbatyuk@gmail.com

@sbatyuk