decomposing applications for scalability and deployability - javazone2012
DESCRIPTION
Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile devices. Applications must be highly scalable, highly available and run on cloud environments. Organizations often want to frequently roll out updates, even multiple times a day. Consequently, it’s no longer adequate to develop simple, monolithic web applications that serve up HTML to desktop browsers. In this talk we describe the limitations of a monolithic architecture. You will learn how to use the scale cube to decompose your application into a set of narrowly focused, independently deployable back-end services and an HTML 5 client. We will also discuss the role of technologies such as NodeJS and AMQP brokers. You will learn how a modern PaaS such as Cloud Foundry simplifies the development and deployment of this style of application. See the video here: http://vimeo.com/49392435TRANSCRIPT
Decomposing applications for deployability and scalability
Chris Richardson
Author of POJOs in ActionFounder of the original CloudFoundry.com
@[email protected]://plainoldobjects.com/
1
Presentation goal
2
How decomposing applications improves
deployability and scalability and
How Cloud Foundry helps
About Chris
3
(About Chris)
4
About Chris()
5
About Chris
6
About Chris
http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/
7
vmc push About-Chris
8
Developer Advocate for CloudFoundry.com
Signup at http://cloudfoundry.compromo code: JavaZone2012
9
Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?§Presentation layer design§How Cloud Foundry helps
Let’s imagine you are building an e-commerce application
10
Tomcat
Traditional web application architecture
11
Browser
WAR
MySQL Database
ShippingService
AccountingService
InventoryService
StoreFrontUI
developtestdeploy
Simple to
Apache
scale
But there are problems with a monolithic architecture
12
Users expect a rich, dynamic and interactive experience
13
Java Web Application
Browser
HTTP Request
HTML/Javascript
Old style UI architecture isn’t good enough
Real-time web ≅ NodeJS
Obstacle to frequent deployments§Need to redeploy everything to change one component§Interrupts long running background (e.g. Quartz) jobs§Increases risk of failure
Fear of change
§Updates will happen less often§e.g. Makes A/B testing UI really difficult
14
Overloads your IDE and container
15Slows down development
16
Shipping team
Accounting team
Engineering
Obstacle to scaling development
E-commerce application
17
WAR
Shipping
Accounting
InventoryService
StoreFrontUI
Shipping team
Accounting team
Inventory team
UI team
Obstacle to scaling development
18
Lots of coordination and communication required
Obstacle to scaling development
19
Requires long-term commitment to a technology
stack
20
Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?§Presentation layer design§How Cloud Foundry helps
21
The scale cube
22
X axis - horizontal duplication
Z axis
- data
parti
tionin
g
Y axis - functionaldecomposition
Scale
by sp
litting
simila
r
thing
s
Scale by splitting different things
Y-axis scaling - application level
23
WAR
ShippingService
AccountingService
InventoryService
StoreFrontUI
Y-axis scaling - application level
24
Store front web application
shipping web application
inventory web application
Apply X axis cloning and/or Z axis partitioning to each service
ShippingService
AccountingService
InventoryServiceStoreFrontUI
accounting web application
Partitioning strategies§Partition by verb, e.g. shipping service§Partition by noun, e.g. inventory service§Single Responsibility Principle§Unix utilities - do one focussed thing well
25
Something of an art
Real world examples
26
http://highscalability.com/amazon-architecture
Between 100-150 services are accessed to build a page.
http://techblog.netflix.com/
http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128
There are drawbacks
27
Complexity
28
See Steve Yegge’s Google Platforms Rant re Amazon.com
Multiple databases =
Transaction management challenges
29
When to use it?
30
In the beginning: •You don’t need it •It will slow you down
Later on:•You need it•Refactoring is painful
But there are many benefits§Scales development: develop, deploy and scale each service independently§Update UI independently§Improves fault isolation§Eliminates long-term commitment to a single technology stack
31
Modular, polyglot, multi-framework applications
Two levels of architecture
32
System-level
ServicesInter-service glue: interfaces and communication mechanismsSlow changing
Service-level
Internal architecture of each serviceEach service could use a different technology stackPick the best tool for the jobRapidly evolving
If services are small...§Regularly rewrite using a better technology stack§Adapt system to changing requirements and better technology without a total rewrite§Pick the best developers rather than best <pick a language> developers ⇒ polyglot culture
33
Fred George “Developer Anarchy”
The human body as a system
34
50 to 70 billion of your cells die each day
35
Yet you (the system) remain you
36
Can we build software systems with these characteristics?
37
http://dreamsongs.com/Files/WhitherSoftware.pdf
http://dreamsongs.com/Files/DesignBeyondHumanAbilitiesSimp.pdf
38
Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?§Presentation layer design§How Cloud Foundry helps
Inter-service communication options§Synchronous HTTP ⇔ asynchronous AMQP
§Formats: JSON, XML, Protocol Buffers, Thrift, ...
§Even via the database
39
Asynchronous is preferred
JSON is fashionable but binary format is more efficient
StoreFrontUI
wgrus-store.war
AccountingService
wgrus-billing.war
InventoryService
wgrus-inventory.war
ShippingService
wgrus-shipping.war
MySQL
40
RabbitMQ(Message Broker)
Asynchronous message-based communication
Benefits§Decouples caller from server§Caller unaware of server’s coordinates (URL)§Message broker buffers message when server is down/slow
41
Drawbacks§Additional complexity of message broker §RPC using messaging is more complex
42
Writing code that calls services
43
The need for parallelism
44
Service A
Service B
Service C
Service D
b = serviceB()
c = serviceC()
d = serviceD(b, c)
Call in parallel
Java Futures are a great concurrency abstraction
45
http://en.wikipedia.org/wiki/Futures_and_promises
Akka’s composable futures are even better
46
Using Akka futures
47
def callB() : Future[...] = ...def callC() : Future[...] = ...def callD() : Future[...] = ...
val future = for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d
val result = Await.result(future, 1 second)
http://doc.akka.io/docs/akka/2.0.1/scala/futures.html
Two calls execute in parallel
And then invokes D
Get the result of D
Spring Integration§Provides the building blocks for a pipes and filters architecture§Enables development of application components that are•loosely coupled•insulated from messaging infrastructure
§Messaging defined declaratively
48
Handling failure
49
Service A Service B
Errors happen in distributed systems
About Netflix
50
> 1B API calls/day1 API call ⇒ average 6 service calls
Fault tolerance is essential
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
Use timeouts and retries
51
Never wait forever
Errors can be transient ⇒ retry
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
Service A
Service B
Use per-dependency bounded thread pool
52http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
Runnable 1
Runnable 2
Runnable ...
bounded queue
Task 1
Task 2
Task ...
bounded thread pool
Limits number of outstanding requests
Fails fast if service is slow or down
Use a circuit breakerHigh error rate ⇒ stop calling temporarily
Down ⇒ wait for it to come back up
Slow ⇒ gives it a chance to recover
53http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
On failureReturn cached data
Return default data
Fail fast
54http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
AvoidFailing
55
Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?§Presentation layer design§How Cloud Foundry helps
Modular application
Choice of presentation layer technology
56
NodeJS is the fashionable technology
57
Why NodeJS?
58
§Familiar Javascript
§High-performance, scalable event-driven, non-blocking I/O model
§Over 13,000 modules developed by the community
§Many JavaScript client frameworks have a NodeJS counterpart, e.g. socket.io
NodeJS example
59
var http = require('http');var fs = require("fs");
http.createServer(function (req, res) { fs.readFile('somefile.txt', function (err, data) { if (err) throw err; res.writeHead(200, {'Content-Type': 'text/plain'}); res.end(data); });}).listen(1337, '127.0.0.1');
console.log('Server running at http://127.0.0.1:1337/');
Handle HTTP
request
Handle file read
NodeJS isn’t the only game in town
60
JVM-based http://vertx.io/
A modern web application
61
Browser
Service 1
Service 2
...
HTML 5Application
Socket.ioclient
Events
RESTful WS
Server Application
Socket.ioserver
Node JS
NodeJS - using RESTful WS and AMQP
62
Node JS
Service
RabbitMQ Service
REST
AMQP AMQP
RESTRequests
Events
socket.io
63
Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?§Presentation layer design§How Cloud Foundry helps
Traditional tools: monolithic applications
64
Developing modular apps is more difficult
§Many more moving parts to manage•Platform services: SQL, NoSQL, RabbitMQ•Application services: your code§Who is going to setup the environments:•the developer sandbox?•...•QA environments?
65
But Cloud Foundry helps...
Applica'on Service Interface
Data Services
Other Services
Msg Services
Easy polyglot application deployment and service provisioning
vFabric Postgres
vFabric RabbitMQTM
Additional partners services …
OSS community
Private Clouds
PublicClouds
MicroClouds
Creating a platform service instance
$ vmc create-service mysql --name mysql1Creating Service: OK
$ vmc services......
=========== Provisioned Services ============
+-------------+---------+| Name | Service |+-------------+---------+| mysql1 | mysql |+-------------+---------+
Multi-application manifest - part 1--- applications: inventory/target: name: inventory url: cer-inventory.chrisr.cloudfoundry.me framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: si-rabbit: type: :rabbitmq si-mongo: type: :mongodb si-redis: type: :redis
68
Path to application
Required platform services
Multi-application manifest - part 2 store/target: name: store url: cer-store.chrisr.cloudfoundry.me framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: si-mongo: type: :mongodb si-rabbit: type: :rabbitmq
69
Path to application
Required platform services
One command to create platform services and deploy application
$ vmc push Would you like to deploy from the current directory? [Yn]: Pushing application 'inventory'...Creating Application: OKCreating Service [si-rabbit]: OKBinding Service [si-rabbit]: OKCreating Service [si-mongo]: OKBinding Service [si-mongo]: OKCreating Service [si-redis]: OKBinding Service [si-redis]: OKUploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK Uploading (12K): OK Push Status: OKStaging Application 'inventory': OK Starting Application 'inventory': OK Pushing application 'store'...Creating Application: OKBinding Service [si-mongo]: OKBinding Service [si-rabbit]: OKUploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK 70
vmc push:•Reads the manifest file•Creates the required platform services•Deploys all the applications
Micro Cloud Foundry: new developer sandbox
71
Open source Platform as a Service project
App Instances Services
10.04
A PaaS packaged as a VMware Virtual Machine
Use as a developer sandbox• Use the services from Junit integration tests• Deploy your application for functional testing• Remote debugging from STS
Using Caldecott to tunnel into your services
72
Caldecott = TCP over HTTP
73
Your computer
Caldecott gem
Cloud Foundry
Caldecott application Service
HTTPnative
protocol
Port NNN
Service client
nativeprotocol
Using Caldecott…$ vmc tunnel1: mysql-135e02: mysql1Which service to tunnel to?: 2Password: ********Stopping Application: OKRedeploying tunnel application 'caldecott'.Uploading Application: Checking for available resources: OK Packing application: OK Uploading (1K): OK Push Status: OKBinding Service [mysql1]: OKStaging Application: OK Starting Application: OK Getting tunnel connection info: OK
Service connection info: username : uMe6Apgw00AhS password : pKcD76PcZR7GZ name : d7cb8afb52f084f3d9bdc269e7d99ab50
Starting tunnel to mysql1 on port 10000.1: none2: mysqlWhich client would you like to start?: 2
…Using CaldecottLaunching 'mysql --protocol=TCP --host=localhost --port=10000 --
user=uMe6Apgw00AhS --password=pKcD76PcZR7GZ d7cb8afb52f084f3d9bdc269e7d99ab50'
Welcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 10944342Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL),
Release 12.5, Revision 188
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
Running JUnit test with Caldecott
76
Configure your test code to use port + connection info
Summary
77
Monolithic applications are simple to develop and deploy
BUT have significant drawbacks
78
Apply the scale cube
79
X axis - horizontal duplicationZ ax
is - d
ata pa
rtitio
ning
Y axis - functionaldecomposition
§Modular, polyglot, and scalable applications§Services developed, deployed and scaled independently
Cloud Provide
r Interface
Applica'on Service Interface
Private Clouds
PublicClouds
MicroClouds
Data Services
Other Services
Msg Services
.js
Cloud Foundry helps...
Questions?
@crichardson [email protected]://slideshare.net/chris.e.richardson/
www.cloudfoundry.com promo code: JavaZone2012