building cloud castles
DESCRIPTION
A year ago, I was a committed VPS and dedicated-machine deployer. I thought the cloud imposed silly restrictions - how dare you take away my shell account! Whaddya mean I can't save files locally? Since then, I've had some interesting experiences. I've worked on big cloud-deployed systems, and certain large traditionally-deployed systems, and I've seen how a lot of the decisions that you're ... encouraged to make when designing an app to run in the cloud. Most interestingly, I've discovered how those same decisions can make for a much better app regardless of where it'll end up. In this talk, I'll share those architectural patterns with you, and show why they work. Hopefully, I'll convince all of you to build cloud castles -- even if you've got your foundation firmly on the ground.TRANSCRIPT
![Page 1: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/1.jpg)
Ben Scofield / @bscofield
and Finding Firmer FoundationsBuilding Cloud Castles
West End Ruby / 2 Feb 2011
![Page 2: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/2.jpg)
flickr: natlockwood
WORK IN PROGRESSthis presentation is a
![Page 3: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/3.jpg)
![Page 4: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/4.jpg)
![Page 5: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/5.jpg)
flickr: turtlemom_nancy
THE CLOUD
![Page 6: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/6.jpg)
flickr: sizemoresr
IS FAR AWAY
![Page 7: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/7.jpg)
LIMITED ACCESS
![Page 8: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/8.jpg)
DIAGNOSIS
![Page 9: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/9.jpg)
$ ssh [email protected] production.server #1 SMP Sat Dec 5 16:04:55 UTC 2009 i686
To access official Ubuntu documentation, please visit:http://help.ubuntu.com/Last login: Fri Jan 28 16:33:49 2011 from local.hostdeploy@production:~$ cd /var/log/apache2deploy@production:/var/log/apache2$ tail error.log[Sun Jan 23 06:25:02 2011] [notice] Apache/2.2.12 (Ubuntu) Phusion_Passenger/2.2.11... [Tue Jan 25 15:21:42 2011] [error] [client 118.129.166.97] Invalid URI in request G...[Fri Jan 28 12:01:50 2011] [error] [client 85.132.70.133] client sent HTTP/1.1 requ...[Sun Jan 30 06:25:06 2011] [notice] SIGUSR1 received. Doing graceful restart
![Page 13: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/13.jpg)
REPAIR
![Page 14: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/14.jpg)
$ ssh [email protected] production.server #1 SMP Sat Dec 5 16:04:55 UTC 2009 i686
To access official Ubuntu documentation, please visit:http://help.ubuntu.com/Last login: Fri Jan 28 16:33:49 2011 from local.hostdeploy@production:~$ cd /var/www/app/current/deploy@production:/var/www/app/current$ rails console productionLoading production environment (Rails 3.0.3)>> Article.count => 112>> Article.where(:problem => true).update_attributes(:problem => false)
![Page 15: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/15.jpg)
require 'test_helper'
class ArticleTest < ActiveSupport::TestCase context 'Broken articles' do setup do 5.times.do { Factory :broken_article } end
should 'be fixable' do assert_equal 5, Article.where(:problem => true).count Article.fix_problem_articles assert_equal 0, Article.where(:problem => true).count end endend
class Article def self.fix_problem_articles where(:problem => true).update_attributes(:problem => false) endend
![Page 16: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/16.jpg)
![Page 17: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/17.jpg)
flickr: turtlemom_nancy
THE CLOUD
![Page 18: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/18.jpg)
flickr: 93921318@N00
IS UNRELIABLE
![Page 19: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/19.jpg)
FILESYSTEMS
![Page 20: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/20.jpg)
class Comic < ActiveRecord::Base has_attached_file :cover, :styles => { :thumb => "80x120>", :medium => "300x450>" }end
$ cd public/system$ ls /covers10/ 12/ 53/ 81/$ ls /covers/10/medium/ original/ thumb/$ ls /covers/10/mediumbatman-450.png
![Page 21: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/21.jpg)
class Comic < ActiveRecord::Base has_attached_file :cover, :storage => s3, :s3_credentials => { :access_key_id => ENV['s3_key'], :secret_access_key => ENV['s3_secret'] }, :bucket => 'comicsapp', :url => ":s3_path_url", :s3_headers => { 'Expires' => 1.year.from_now.httpdate }, :styles => { :thumb => "80x120>", :medium => "300x450>" }end
![Page 22: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/22.jpg)
class Comic < ActiveRecord::Base has_attached_file :cover, :storage => s3, :s3_credentials => { :access_key_id => ENV['s3_key'], :secret_access_key => ENV['s3_secret'] }, :bucket => 'comicsapp', :url => ":s3_path_url", :s3_headers => { 'Expires' => 1.year.from_now.httpdate }, :styles => { :thumb => "80x120>", :medium => "300x450>" }end
![Page 23: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/23.jpg)
module Watchtower class Application < Rails::Application # ... require 'openid/store/filesystem' config.middleware.use OmniAuth::Strategies::OpenID, OpenID::Store::Filesystem.new('/tmp') endend
![Page 24: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/24.jpg)
module Watchtower class Application < Rails::Application # ... require 'openid/store/filesystem' config.middleware.use OmniAuth::Strategies::OpenID, OpenID::Store::Filesystem.new('./tmp') endend
![Page 25: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/25.jpg)
module Watchtower class Application < Rails::Application # ... require 'openid/store/filesystem' config.middleware.use OmniAuth::Strategies::OpenID, OpenID::Store::Filesystem.new('./tmp') endend
![Page 26: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/26.jpg)
![Page 27: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/27.jpg)
flickr: turtlemom_nancy
THE CLOUD
![Page 28: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/28.jpg)
flickr: lensonlife
IS HOSTILE
![Page 29: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/29.jpg)
EXTERNAL SERVICES
![Page 30: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/30.jpg)
class Searcher def self.search(term) Article.where(['content ILIKE ?', "%#{term}%"]) endend
![Page 31: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/31.jpg)
class Searcher cattr_accessor :index
def self.index @api ||= IndexTank::Client.new(ENV['INDEXTANK_API_URL']) @index = @api.indexes 'articles' end
def self.search(term) raw = self.index.search(term, :function => 1) results = raw['results'].to_a
article_ids = results.map {|result| result['docid'] }
unsorted = Article.published.where(:id => article_ids) results.map { |result| unsorted.find {|u| u.id.to_i == result['docid'].to_i} }.compact endend
![Page 32: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/32.jpg)
class Searcher cattr_accessor :index
def self.index @api ||= IndexTank::Client.new(ENV['INDEXTANK_API_URL']) @index = @api.indexes 'articles' end
def self.search(term) results = begin raw = self.index.search(term, :function => 1) raw['results'].to_a rescue URI::InvalidURIError # An IndexTank error occurred search_by_sql(term)['results'] end
article_ids = results.map {|result| result['docid'] }
unsorted = Article.published.where(:id => article_ids) results.map { |result| unsorted.find {|u| u.id.to_i == result['docid'].to_i} }.compact end
def self.search_by_sql(term) {'results' => Article.where(['content ILIKE ?', "%#{term}%"]). map {|a| {'docid' => a.id}}} endend
![Page 33: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/33.jpg)
class Searcher cattr_accessor :index
def self.index @api ||= IndexTank::Client.new(ENV['INDEXTANK_API_URL']) @index = @api.indexes 'articles' end
def self.search(term) results = begin raw = self.index.search(term, :function => 1) raw['results'].to_a rescue URI::InvalidURIError # An IndexTank error occurred search_by_sql(term)['results'] end
article_ids = results.map {|result| result['docid'] }
unsorted = Article.published.where(:id => article_ids) results.map { |result| unsorted.find {|u| u.id.to_i == result['docid'].to_i} }.compact end
def self.search_by_sql(term) {'results' => Article.where(['content ILIKE ?', "%#{term}%"]). map {|a| {'docid' => a.id}}} endend
![Page 34: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/34.jpg)
![Page 35: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/35.jpg)
flickr: turtlemom_nancy
THE CLOUD
![Page 36: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/36.jpg)
flickr: 3sth3r
IS RECYCLED
![Page 37: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/37.jpg)
CACHING
![Page 38: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/38.jpg)
class BooksController < ApplicationController caches_page :index def index @books = Book.all endend
![Page 39: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/39.jpg)
class BooksController < ApplicationController caches_action :index def index @books = Book.all endend
module CardCatalog class Application < Rails::Application # ... ActionController::Base.cache_store = :mem_cache_store, "memcache_host" endend
![Page 40: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/40.jpg)
class BooksController < ApplicationController caches_action :index def index @books = Book.all endend
module CardCatalog class Application < Rails::Application # ... ActionController::Base.cache_store = :mem_cache_store, "memcache_host" endend
![Page 41: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/41.jpg)
class BooksController < ApplicationController def index response.headers['Cache-Control'] = 'public, max-age=300' @books = Book.all endend
![Page 42: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/42.jpg)
class BooksController < ApplicationController def index response.headers['Cache-Control'] = 'public, max-age=300' @books = Book.all endend
![Page 43: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/43.jpg)
![Page 44: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/44.jpg)
flickr: turtlemom_nancy
THE CLOUD
![Page 45: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/45.jpg)
flickr: dev07
IS MADE OF TINY PARTS
![Page 46: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/46.jpg)
THINKING SMALL
![Page 47: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/47.jpg)
App
![Page 48: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/48.jpg)
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
![Page 49: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/49.jpg)
$ heroku createCreating empty-sword-187....... donehttp://empty-sword-187.heroku.com/ | [email protected]:empty-sword-187.gitGit remote heroku added$ git push heroku masterCounting objects: 5, done.Delta compression using up to 4 threads.Compressing objects: 100% (3/3), done.Writing objects: 100% (3/3), 285 bytes, done.Total 3 (delta 2), reused 0 (delta 0)
-----> Heroku receiving push-----> Rails app detected-----> Detected use of caches_page Installing caches_page_via_http plugin... done-----> Detected Rails is not set to serve static_assets Installing rails3_serve_static_assets... done-----> Gemfile detected, running Bundler version 1.0.7 Unresolved dependencies detected; Installing... Using --without development:test Fetching source index for http://rubygems.org/ ...
![Page 50: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/50.jpg)
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
![Page 51: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/51.jpg)
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
App
![Page 52: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/52.jpg)
HTTP and REST
![Page 53: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/53.jpg)
![Page 54: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/54.jpg)
![Page 55: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/55.jpg)
PATTERNS and VIRTUES
![Page 56: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/56.jpg)
SINGLE RESPONSIBILITY PRINCIPLE
![Page 57: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/57.jpg)
HUMILITY
![Page 58: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/58.jpg)
LAZINESS
![Page 59: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/59.jpg)
PARANOIA
![Page 60: Building Cloud Castles](https://reader034.vdocument.in/reader034/viewer/2022052523/55584540d8b42ac6078b5276/html5/thumbnails/60.jpg)
Ben Scofield / @bscofield
Thanks!
West End Ruby / 2 Feb 2011
http://spkr8.com/t/5491http://benscofield.com
http://heroku.com