hosting ruby web apps
DESCRIPTION
The options for hosting ruby web application are plentiful, all with different advantages and disadvantages, options, limitations. How to start, how to grow, what are the pitfalls? With this talk I’d first like to give a short overview of several cloud hosting alternatives such as plain VPS, AWS, EngineYard, Heroku, and provide some insights based on my experience with them – beyond just somehow getting it to run, but also how to handle continuous deployment, how to maintain and scale them. While Rails already comes with many best practices build in, there are still plenty enough traps for you. We definitely had our fair share, and I’d like to share some of them for your entertainment and learning.TRANSCRIPT
![Page 1: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/1.jpg)
Hosting Ruby Web Apps
Lessons learned from 8 years of
![Page 2: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/2.jpg)
Overview
• System architecture
• Initial setup & deploy
• Keep it running and moving forward
![Page 4: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/4.jpg)
Next Weekend! startupweekend.jp/swtokyo-personal-cloud/
![Page 6: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/6.jpg)
Build a community through your events.
![Page 7: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/7.jpg)
![Page 8: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/8.jpg)
DB (psql)
memcache
nginxunicorn
unicornRails App (unicorn)
job workerMTA (postfix)
![Page 9: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/9.jpg)
Contenders
![Page 10: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/10.jpg)
Contenders
![Page 11: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/11.jpg)
Contenders
![Page 12: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/12.jpg)
Contenders
![Page 13: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/13.jpg)
Contenders
![Page 14: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/14.jpg)
Please Note
• Not an exhaustive list of all hosting providers(it’s not even everyone we’ve been using)
• Moving targets
![Page 15: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/15.jpg)
Choosing Server Sizewww.flickr.com/photos/jonrb/7864016624
![Page 16: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/16.jpg)
Choosing Server Size
• good CPU performance
• choose 2GB RAM or more
![Page 17: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/17.jpg)
Choosing Server Size
• 1 ECU/core is a bit frustrating
• 2 ECU/core is OK
• >2 ECU/core is better
![Page 18: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/18.jpg)
Choosing Server Size
• 3 different sizes
• choose standard size if you don’t have specific requirements
![Page 19: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/19.jpg)
Hardware Failures
How can we make things robust?
www.flickr.com/photos/doegox/4551458930
![Page 20: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/20.jpg)
Hardware Failures“We’re trying to prevent failures, please
make backups for worst case”
!
• provides load balancer
• fault tolerant setup example:2 web instances + 2 DB instances
• need to configure DB replication and failover yourself
![Page 21: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/21.jpg)
Hardware Failures“Failures will happen, build your infrastructure
so they won’t impact you”
• provides load balancer (ELB)
• provides managed DB instances (RDS)
• replication support (not for psql yet)
• DB snapshots (can’t download though)
• fault tolerant setup example: 2 web instances + multi-AZ RDS
![Page 22: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/22.jpg)
Hardware Failures“Failures will happen, let us help you build an
infrastructure so they won’t impact you”
• HA proxy on web instances
• one-click setup for DB (mysql/psql)
• replication support
• DB snapshots (downloadable)
• fault tolerant setup example:2 web instances + 2 DB instances
![Page 23: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/23.jpg)
Hardware Failures“You don’t need to worry about failures”
!
!
• everything managed
• fault tolerant setup example: 2 dynos + premium DB (psql)
![Page 24: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/24.jpg)
Initial Setup & Deploy
www.flickr.com/photos/thedailyenglishshow/6013713229
![Page 25: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/25.jpg)
Initial Setup & Deploy
• manual setup feasible:
• install OS, ruby (rvm), libs, DB
• setup nginx
![Page 26: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/26.jpg)
Initial Setup & Deploy
• let’s choose Capistrano for deploying
• Capistrano config goes into source repository
• Initial deploy:
cap deploy:setup cap deploy:cold
![Page 27: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/27.jpg)
Initial Setup & Deploy
• let’s choose OpsWorks
• based on chef, provides predefined set of recipes
• more high level than Cloud Formation, more flexible than Elastic Beanstalk
![Page 28: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/28.jpg)
Initial Setup & Deploy• initial deploy:
• create stack
• define layers
• create instances
• create app
• deploy
![Page 29: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/29.jpg)
Initial Setup & Deploy
• define base layer
• assign it to all instances
• use it for any common recipes like creating swap, NewRelic, ...
Tip
![Page 30: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/30.jpg)
Initial Setup & Deploy• you need to handle asset compilation
• use deploy hook:
Tip
# deploy/before_migrate.rb !rails_env = new_resource.environment["RAILS_ENV"] Chef::Log.info("Precompiling assets for RAILS_ENV=#{rails_env}...") !execute "rake assets:precompile" do cwd release_path command "bundle exec rake assets:precompile" environment "RAILS_ENV" => rails_env end !!
![Page 31: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/31.jpg)
Initial Setup & Deploy• provides toolchain based on chef
• initial deploy:
• create an application
• select environment layout
• add plugins (NewRelic, ...)
• deploy
![Page 32: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/32.jpg)
Initial Setup & Deploy
Heroku comes with its’ own toolchain:
heroku create my-awesome-app heroku addons:add … git remote add heroku … git push heroku master
![Page 33: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/33.jpg)
What you’ll probably also need
• App configuration (for secrets and endpoints)
• Sending email
• Job queue
![Page 34: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/34.jpg)
App Configuration
• no predefined way: DIY
• put settings.yml into shared config dir
• link it into app when deploying
![Page 35: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/35.jpg)
App Configuration
• uses environment variables
heroku config:add MY_SECRET=topsecret heroku config
• in your code:
ENV[‘MY_SECRET’]
![Page 36: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/36.jpg)
Sending Email
• use an email sending service (SendGrid, AWS SNS)
• EY and Heroku provide plugins for easy installation
![Page 37: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/37.jpg)
Sending Email Yourself
• reverse IP lookup
• port 25 is restricted
• make sure your IP isn’t blacklisted
![Page 38: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/38.jpg)
Job Queue
• DIY
• define custom layer in OpsWorks, need to get 3rd party recipe
![Page 39: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/39.jpg)
Job Queue
• sample recipes are provided
• worker instances
![Page 40: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/40.jpg)
Job Queue
• provides worker dynos
• requires setup in Procfile
![Page 41: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/41.jpg)
Background mailer
• use database transaction to atomically create mailer job and related data
• rollback if something goes wrong
• reduces request response time
• job can retry sending email
Tip
![Page 42: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/42.jpg)
Up and running :-)
![Page 43: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/43.jpg)
Continuous Deployment
www.flickr.com/photos/layos/3743880081
![Page 44: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/44.jpg)
Continuous Deployment
• keep deploys cheap
• automate deploy
• easy deployment trigger
• good test coverage - use CI
• use rolling / zero downtime deploy
![Page 45: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/45.jpg)
Continuous Deployment
• deploy command:
git pull cap deploy
• unicorn for rolling deploy
![Page 46: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/46.jpg)
Continuous Deployment
• unicorn is configured for rolling deploys
• deploy command:
aws --region=‘us-east-1’ opsworks create-deployment --stack-id=‘<STACK_UUID>’ --app-id=‘<APP_UUID>’ --instance-ids=‘[“<INSTANCE_UUID>”]’ --command='{"Name": "deploy"}
![Page 47: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/47.jpg)
Continuous Deployment
gem 'aws-sdk' gem 'parseconfig' !PRODUCTION_APP_ID = "09afbde1-322b-4816-a1e9-xxxxxxxxxxxx" !config = ParseConfig.new(File.expand_path("~/.aws/config")) AWS.config( :access_key_id => config['default']['aws_access_key_id'], :secret_access_key => config['default']['aws_secret_access_key']) !@ops = AWS::OpsWorks.new.client @ops_app = @ops.describe_apps(app_ids: [PRODUCTION_APP_ID])[:apps].first @ops_stack = @ops.describe_stack_summary(stack_id: @ops_app[:stack_id])[:stack_summary] @ops_inst = @ops.describe_instances(stack_id: @ops_app[:stack_id])[:instances] @ops_inst_ids = @ops_instances.map do |instance| instance[:instance_id] if instance[:status] == 'online' end.compact !puts "Deploying #{@ops_app[:name]} to #{@ops_stack[:name]}" !deploy_options = { command: { name:' deploy' }, comment: "deploy from #{Socket.gethostname}", stack_id: @ops_app[:stack_id], app_id: @ops_app[:app_id], instance_ids: @ops_inst_ids } @ops.create_deployment deploy_options !!
or some simple ruby script:
![Page 48: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/48.jpg)
Continuous Deployment
• deploy command:
ey deploy
• in config/ey.yml:
maintenance_on_migrate: false
![Page 49: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/49.jpg)
Continuous Deployment
• deploy command:
git pull git push heroku master
• no-downtime deploys experimental:
heroku labs:enable -a myapp preboot
![Page 50: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/50.jpg)
Continuous Deployment
• “fork” in code
• on/off switch for features
• slow rollout of new features
Tip
![Page 51: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/51.jpg)
Rolling Deploys with!Database Migrations
www.flickr.com/photos/edwardshousemovers/6704586649
![Page 52: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/52.jpg)
Rolling Deploys with Database Migrations
orchestrated deployment flow:
1. copy code, keep old version
2. run migrations
3. switch to new code
➜ one-step deployments
![Page 53: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/53.jpg)
Rolling Deploys with Database Migrations
• asynchronous deployment flow
• two-step deployment
1. deploy old code + DB migrations(one instance only)
2. deploy new code(all instances)
![Page 54: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/54.jpg)
Rolling Deploys with Database Migrations
• migrations are run via:
heroku run rake db:migrate
• two-step deployment
1. deploy old code + DB migrations
2. deploy new code
![Page 55: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/55.jpg)
Down for!Maintenance
www.flickr.com/photos/metrolibraryarchive/4128611963
![Page 56: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/56.jpg)
Down for Maintenance
• Maintenance page done right:
• include contact and progress info (use 3rd party service like Twitter)
• serve page with 503 error code
• use asset host or inline all assets
Tip
![Page 57: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/57.jpg)
Down for Maintenance• add ‘capistrano-maintenance’ gem
• configure nginx
• then:
cap maintenance:enable
server { // nginx server config ! location @maintenance { rewrite ^(.*)$ /system/maintenance.html last; break; } if (-f $document_root/system/maintenance.html) { return 503; } ! error_page 503 @maintenance;
![Page 58: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/58.jpg)
Down for Maintenance
• Need custom recipe / script
• Can use similar approach as with Capistrano
• 503 won’t pass ELB health check!
• Alternative: use Route53 to fail over to instance serving maintenance page
![Page 59: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/59.jpg)
Down for Maintenance
• Part of the platform:
ey web disable
heroku maintenance:on
![Page 60: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/60.jpg)
Getting Support
![Page 61: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/61.jpg)
Getting Support
• provides ticket system for any platform related issues
• forum for anything else
![Page 62: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/62.jpg)
Getting Support
• forums for everything
• access to tickets only available if certain health checks fail
![Page 63: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/63.jpg)
Getting Support
• tickets for everything
• provides even hosting related help on application level
• “extension of your team”
![Page 64: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/64.jpg)
Conclusion• there is no silver bullet
• a lot depends on your app
• be ready to get your hands dirty
• your production environment is your app
• balance ops / dev
![Page 65: Hosting Ruby Web Apps](https://reader034.vdocument.in/reader034/viewer/2022052618/554bd70bb4c905ac708b501c/html5/thumbnails/65.jpg)
Thank you!
Contact:
Michael [email protected] @mreinsch
Questions?More awesome events:
• ijaws.doorkeeper.jp
• Startup Weekend Tokyo Personal Cloud