building cloud castles - lrug
DESCRIPTION
Latest version of Building Cloud Castles, given at LRUG in April 2011.A year ago, I was a committed VPS and dedicated-machine deployer. I thought the cloud imposed silly restrictions - how dare you take away my shell account! Whaddya mean I can't save files locally?Since then, I've had some interesting experiences. I've worked on big cloud-deployed systems, and certain large traditionally-deployed systems, and I've seen how a lot of the decisions that you're ... encouraged to make when designing an app to run in the cloud. Most interestingly, I've discovered how those same decisions can make for a much better app regardless of where it'll end up. In this talk, I'll share those architectural patterns with you, and show why they work. Hopefully, I'll convince all of you to build cloud castles -- even if you've got your foundation firmly on the ground.TRANSCRIPT
![Page 1: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/1.jpg)
ben scofield / @bscofield / LRUG / 11 April 2011
BUILDINGCLOUD
CASTLES
![Page 2: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/2.jpg)
http://www.flickr.com/photos/natlockwood/1006351276/
WORK IN PROGRESSthis presentation is a
![Page 3: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/3.jpg)
![Page 4: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/4.jpg)
not just
![Page 5: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/5.jpg)
the cloud allowsLIMITED ACCESS
![Page 6: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/6.jpg)
FINDING PROBLEMS
![Page 7: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/7.jpg)
LOCAL$ ssh [email protected] production.server #1 SMP Sat Dec 5 16:04:55 UTC 2009 i686
To access official Ubuntu documentation, please visit:http://help.ubuntu.com/Last login: Fri Jan 28 16:33:49 2011 from local.hostdeploy@production:~$ cd /var/log/apache2deploy@production:/var/log/apache2$ tail error.log[Sun Jan 23 06:25:02 2011] [notice] Apache/2.2.12 (Ubuntu) Phusion_Passenger...[Tue Jan 25 15:21:42 2011] [error] [client 118.129.166.97] Invalid URI in ...[Fri Jan 28 12:01:50 2011] [error] [client 85.132.70.133] client sent HTTP/1...[Sun Jan 30 06:25:06 2011] [notice] SIGUSR1 received. Doing graceful restart
![Page 8: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/8.jpg)
NODE
NODE
NODE
NODE
NODE
NODE
NODE
NODE
![Page 9: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/9.jpg)
CLOUD$ heroku logs --remote production2011-04-11T01:34:22-07:00 app[web.1]: Rendered text template within layout...2011-04-11T01:34:22-07:00 app[web.1]: Completed 200 OK in 54ms2011-04-11T01:34:30-07:00 app[web.3]: Rendered text template within layout...2011-04-11T01:34:30-07:00 app[web.3]: Completed 200 OK in 111ms (Views:57....2011-04-11T08:34:42+00:00 heroku[router]: GET devcenter.heroku.com/article...2011-04-11T01:34:42-07:00 heroku[nginx]: GET /articles/facebook HTTP/1.0 |...2011-04-11T01:34:42-07:00 app[worker.1]: [Worker(host:railgun64.53370 pid:...2011-04-11T01:34:42-07:00 app[worker.1]: [Worker(host:railgun64.53370 pid:...
![Page 10: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/10.jpg)
http://hoptoadapp.com
![Page 11: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/11.jpg)
http://newrelic.com
![Page 12: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/12.jpg)
source
![Page 13: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/13.jpg)
FIXING PROBLEMS
![Page 14: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/14.jpg)
LOCAL$ ssh [email protected] production.server #1 SMP Sat Dec 5 16:04:55 UTC 2009 i686
To access official Ubuntu documentation, please visit:http://help.ubuntu.com/Last login: Fri Jan 28 16:33:49 2011 from local.hostdeploy@production:~$ cd /var/www/app/current/deploy@production:/var/www/app/current$ rails console productionLoading production environment (Rails 3.0.3)>> Article.count => 112>> Article.where(:problem => true).update_attributes(:problem => false)
![Page 15: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/15.jpg)
CLOUDrequire 'test_helper'
class ArticleTest < ActiveSupport::TestCase context 'Broken articles' do setup do 5.times.do { Factory :broken_article } end
should 'be fixable' do assert_equal 5, Article.where(:problem => true).count Article.fix_problem_articles assert_equal 0, Article.where(:problem => true).count end endend
class Article def self.fix_problem_articles where(:problem => true).update_attributes(:problem => false) endend
![Page 16: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/16.jpg)
the cloud isUNRELIABLE locally
![Page 17: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/17.jpg)
LOCALclass Comic < ActiveRecord::Base has_attached_file :cover, :styles => { :thumb => "80x120>", :medium => "300x450>" }end
$ cd public/system$ ls /covers10/ 12/ 53/ 81/$ ls /covers/10/medium/ original/ thumb/$ ls /covers/10/mediumbatman-450.png
![Page 18: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/18.jpg)
NODE
NODE
NODE
NODE
NODE
NODE
NODE
NODE
![Page 19: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/19.jpg)
EVENTUAL CONSISTENCY
![Page 20: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/20.jpg)
CLOUDclass Comic < ActiveRecord::Base has_attached_file :cover, :storage => s3, :s3_credentials => { :access_key_id => ENV['S3_KEY'], :secret_access_key => ENV['S3_SECRET'] }, :bucket => 'comicsapp', :url => ":s3_path_url", :s3_headers => { 'Expires' => 1.year.from_now.httpdate }, :styles => { :thumb => "80x120>", :medium => "300x450>" }end
![Page 21: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/21.jpg)
SERVER SERVER
![Page 22: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/22.jpg)
SERVER SERVER
STORAGE
![Page 23: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/23.jpg)
the cloud can beEXPENSIVE
![Page 24: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/24.jpg)
NODE
NODE
NODE
NODE
NODE
NODE
NODE
NODE
![Page 25: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/25.jpg)
NODE
NODE
NODE
NODE
NODE
NODE
NODE
NODE
CACHING
![Page 26: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/26.jpg)
SERVER
![Page 27: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/27.jpg)
SERVER
CACHING
![Page 28: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/28.jpg)
the cloud prefersSMALL APPS
![Page 29: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/29.jpg)
NODE
NODE
NODE
NODE
NODE
NODE
NODE
NODE
![Page 30: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/30.jpg)
NODE
NODE
NODE
NODE
NODE
NODE
NODE
NODE
![Page 31: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/31.jpg)
REST
![Page 32: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/32.jpg)
NODE
NODE
NODE
NODE
NODE
NODEEXTERNALSERVICE
![Page 33: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/33.jpg)
![Page 34: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/34.jpg)
SERVER
![Page 35: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/35.jpg)
SERVER SERVER
![Page 36: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/36.jpg)
SERVICESERVER
![Page 37: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/37.jpg)
the cloud isUNRELIABLEremotely
![Page 38: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/38.jpg)
CLOUDclass Searcher cattr_accessor :index
def self.index @api ||= IndexTank::Client.new(ENV['INDEXTANK_API_URL']) @index = @api.indexes 'articles' end
def self.search(term) raw = self.index.search(term, :function => 1) results = raw['results'].to_a
article_ids = results.map {|result| result['docid'] }
unsorted = Article.published.where(:id => article_ids) results.map { |result| unsorted.find {|u| u.id.to_i == result['docid'].to_i} }.compact endend
![Page 39: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/39.jpg)
EXPECT FAILURE
![Page 40: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/40.jpg)
CLOUDclass Searcher # ...
def self.search(term) results = begin raw = self.index.search(term, :function => 1) raw['results'].to_a rescue URI::InvalidURIError # An IndexTank error occurred search_by_sql(term)['results'] end
# ... end
def self.search_by_sql(term) {'results' => Article.where(['content ILIKE ?', "%#{term}%"]). map {|a| {'docid' => a.id}}} endend
![Page 41: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/41.jpg)
DEFENSIVE CODING
![Page 42: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/42.jpg)
the cloud isALWAYS CURRENT
![Page 43: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/43.jpg)
RAISE YOUR HAND if
![Page 44: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/44.jpg)
INFRASTRUCTURE
![Page 45: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/45.jpg)
PATTERNS and VIRTUES
![Page 46: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/46.jpg)
HUMILITY
![Page 47: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/47.jpg)
LAZINESS
![Page 48: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/48.jpg)
PARANOIA
![Page 49: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/49.jpg)
SOLIDsingle responsibility principleliskov substitution principleinterface segregation principle
![Page 50: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/50.jpg)
CLOUD PRACTICESareBEST PRACTICES
![Page 51: Building Cloud Castles - LRUG](https://reader034.vdocument.in/reader034/viewer/2022052310/5558447fd8b42acb078b529a/html5/thumbnails/51.jpg)
THANK YOU!
Ben Scofield / @bscofieldhttp://benscofield.comhttp://speakerrate.com/t/7123