Programming for cloud computing: Flows of asynchronous messages (swarms)

Sînică Alboaie (The Red Point S.A.) 2012,CodeCamp

Programming == Fight against complexity

Essential Complexity?

"The Tao that can be spoken is not the eternal Tao" Lao Zi

Loose couplingSingle Responsibility Principle Separation of ConcernsOpen Closed principle

Cloud Programming, Distributed and Parallel Systems

... and why should YOU care?

Point of view is worth 80 IQ points?

In Roman time you had to be very smart to multiply two numbers together.

Cloud applications

● not a 'revolution'

● an evolution of the enterprise applications

Isn't any different than traditional development.

● less tolerant of poor development practices

● attention to scalability and security● don't relay on improving hardware ● no security through obscurity

!

Cloud computing exposes bad practices

Money !?

● Programming (new APIs, complexity)● Automation in deployment, configuration● New Type of Programmer: Dev-Op● Scalability (efficient use of hardware)● High availability (install/updates, failures)● Data consistency

Insights: Programming for the cloud

Multitenancy

● PaaS● iPaaS ● ...aaS● Sharing and SLAs● Security

Insights: Programming in cloud

● use quality open sources:○ Redis, ZooKeeper, Message Queues○ Hadoop, NoSQL Databases, etc○ openstack, cloudfoundry

● node.js!

● Java,.net, php ?● Amazon, Azure, Google ?

Cloud for normal budgets

What... SWARMING?

So...

Nodes

● Cloud application == Forest● Trees are processes● Swarms are visiting trees (nodes)

Swarm concepts

Nodes

● Swarm's temporary destinations● API providers (services)● Adapters and Clients (web)

Swarm concepts

Swarm description

● variables (members)● phases (code)● "swarm" primitive

Swarm concepts

At runtime

Nothing special: Message passing!

Swarm concepts

Hello world example, execution of 2 phases in 2 nodes:

Swarm description example

Points of view

(Naive) messaging /Actors/ Erlang:Nodes are sending typed messages to each otherImplicit: Flows are in programmer's minds Application logic mixed in all nodesNodes are reacting to messagesSwarming:Simpler code in nodes(APIs)Explicit: Flows are described in filesMessages are flowing between nodesMessages are causing changes in nodes

Usage: Enterprise Integration

Code cd

Files, DatabasesPoint Point to point (Services/Messages)Busses (Services/Messages)

SwarmESB

Usage: Cloud ready architecture

Code cd

SwarmESB

● node.js ● Java Script and Flex clients● WebDAV proxy● SOLR adapter● Redis: pub/sub channels

○ messages ● Redis

○ multi-tenant storage

Current implementation

SwarmESB for you?

Familiarity! Java Script + node.jsVertical and horizontal scalability Architect your new distributed systemConsume heterogeneous services (APIs)Easy integration for future componentsParallel usage of your shiny, new CPU coresElegant programming with asynchronous messages

...

● multitenant global (redis) contexts● multitenant local (node) contexts● local sub-processes● swarm code redeployment without restart

Swarming can be for SOA's orchestration what REST is for SOAP!

More coolness

Hackers and Hustlers Thank you!

WorkerSwarm1 ctor (doWork) 3 phases● doChooseWorker● executeWork● taskDone

Load balancer example

WorkerManagment swarm ● Ctors:

○ register [worker at start]○ unregister [worker at shutdown]○ checkWorkerStatus [Balancer from time to time]

● Phases:○ doRegister [Balancer]○ doUnegister [Balancer]○ statusCheck [worker]○ statusReport [Balancer]

Load balancer example

SOA (Orchestration and choreography)EAI (Enterprise Integration Patterns)ESB (Enterprise Service Bus)Events and Asynchronous MessagesPaaS ( Platform as a Service)iPaaS (Cloud Integration)

Keywords