scalable plone hosting with amazon ec2 for rice university's rhaptos open learning platform

Scalable Plone hosting with Amazon EC2

for Rice University’sRhaptos open learning

platformNate Aune - Jazkarta.comPlone Conference 2010

Bristol, UK

1Sunday, November 7, 2010

Agenda

• Benefits of cloud computing

• What services does Amazon provide?

• 5 minute launch of Plone on EC2

• Case study: Rhaptos hosting on EC2

• Questions?


What makes the Cloud so attractive?


Abstract resourcesFocus on your needs, not on hardware specs.

As your needs change, so should your resources.


On-demand provisioningAsk for what you need, exactly when you need it.

Get rid of it when you don’t need it.


Scalability in minutesScale out or in depending on usage needs.


Pay per consumptionNo contracts or long-term commitments.

Pay only for what you use.


Efficiency of ExpertsUtilize the skills, knowledge and resources of experts.


Most applications need:

• Compute

• Storage

• Messaging

• Payment

• Distribution

• Scale

• Analytics


Flexible infrastructure

• Elastic Computing Cloud (EC2)

• Amazon Machine Images (AMI)

• Elastic IPs

• Elastic Block Storage (EBS)

• Auto-scaling

• Elastic Load Balancing (ELB)


Other services

• Simple Storage Solution (S3)

• CloudWatch (monitoring)

• CloudFront (CDN)


http://uec-images.ubuntu.com/

Text


http://uec-images.ubuntu.com/releases/lucid/release/

http://uec-images.ubuntu.com/releases/lucid/release/

mr.awsomeBuildout recipe to manage EC2 instances


Add an AWS part to buildout[buildout]

parts = aws

[aws]

recipe = zc.recipe.egg

eggs = mr.awsome

entry-points =

aws=mr.awsome:aws

assh=mr.awsome:aws_ssh

arguments = configpath="${buildout:directory}/etc"


Make a etc/aws.conf file[securitygroup:demo-server]description = Bristol Demo Serverconnections = tcp 22 22 0.0.0.0/0 tcp 80 80 0.0.0.0/0 tcp 8080 8080 0.0.0.0/0

[instance:demo-server]keypair = bristol-keypairsecuritygroups = demo-serverregion = us-east-1placement = us-east-1ainstance_type = t1.microimage = ami-480df921startup_script = startup-demo-serverfabfile = fabfile.py


Make a bash script: etc/startup-demo-server

#!/bin/bash

set -e -x

export DEBIAN_FRONTEND=noninteractive

wget http://www.enfoldsystems.com/pubkey.gpg

apt-key add ./pubkey.gpg

echo "deb http://dist.clients.enfoldsystems.com/ubuntu lucid universe" >> /etc/apt/sources.list

apt-get update && apt-get upgrade -y

apt-get install plone-default -y

/etc/init.d/plone-default


http://www.enfoldsystems.com/pubkey.gpg

http://www.enfoldsystems.com/pubkey.gpg

http://dist.clients.enfoldsystems.com/ubuntu

http://dist.clients.enfoldsystems.com/ubuntu

Set your access keys

#!/bin/bash

export AWS_ACCESS_KEY_ID="<your_access_key_id>"

export AWS_SECRET_ACCESS_KEY="<your_secret_access_key>"

Set these environment variables.

$ source setkeys.sh

$ env

File: setkeys.sh


Start the demo server$ ./bin/aws start demo-server

INFO: Instance 'demo-server' unavailable

INFO: Creating instance 'demo-server'

INFO: Instance created, waiting until it's available

....

INFO: Instance 'demo-server' available

INFO: Instance running.

INFO: Instances DNS name ec2-50-16-25-92.compute-1.amazonaws.com

INFO: Instances public DNS name ec2-50-16-25-92.compute-1.amazonaws.com


Benefits of EC2• Seamless disaster recovery with repeatable

deployments using buildout and mr.awsome

• Easy launching of testing, staging and QA servers that are isolated from your production environment (stop when not used to avoid paying for them)

• Pay-for-what-you-use metered pricing to launch servers on-demand during peak periods

• Auto-scaling to launch new Zeo client machines and shut them down when no longer needed

•


Connexions & RhaptosA case study in deploying OSS to the cloud


s

Log In Contact Us Report a Bug

SearchSearch Site

Home Content Lenses About Us Help MyCNX

FEATURED CONTENT

New features

1 2 3

MY ACCOUNT

SPOTLIGHT

Skip

ConnexionsSections

You are here: Home

Connexions is:a place to view and share educational material made ofsmall knowledge chunks called modules that can beorganized as courses, books, reports, etc. Anyone may viewor contribute:

authors create and collaborate

instructors rapidly build and share custom collections

learners find and explore content

More about us ...

Advanced Algebra IIThe world of second-yearalgebra comes alive in KennyFelder's Advanced Algebra II.Felder, a high school mathteacher in North Carolina,designed his course using anontraditional approach to a verytraditional subject. Centeredaround a Homework andActivities book, along withaccompanying Conceptual

Explanations and Teacher's Guide collections,Advanced Algebra II tackles topics by focusing oncomprehension rather than straight lecture delivery.

Advanced Algebra II was selected by California's FreeDigital Textbook Initiative to be included in a list offree textbooks available to California schools startingin Fall 2009.

Collaborative StatisticsCollaborative Statistics waswritten by two faculty members

FIND CONTENT

16547 reusable modules woven into 1007 collections.

or browse by ...

Language

Popularity

Title, author, etc.

CREATE CONTENT

Creating content in Connexions is as easy as 1, 2, 3:

Get an accountand log in to

your workspace.

Make a modulefrom scratch or

convert it from aWord doc.

Publish yourworks, sharing

them with the world.

Jump right in

Get an account

How to create a module inminutes

How to create a collectionwith existing modules

Guides and tutorials

New author guide

Connexions Tutorial andReference

Username

Password

Log in

Get an account

Forgot your password?

User feedback"I enjoy usingtheConnexionssite and thephilosophybehind itencouragesone to learnnew fields and

to network with others. Yoursite is my new homeschool onthe internet. It is helping meto grow as a writer ofacademic materials as I readwhat others have written so asto shape my own. I am alsoamazed at the range ofinstitutions where your writerscome from. Thanks for the useof these great facilities."

Go

ArtsBusinessHumanitiesMathematics and StatisticsScience and TechnologySocial Sciences

Search Content

Subject


Search

Login Preferences Help/Guide About Trac

Wiki Timeline Roadmap Browse Source View Tickets New Ticket Search

Start Page Index History Last Change

Welcome to Connexions and Rhaptos Software Development

Connexions ( cnx.org) is legos for education. It is a world-wide repository and publishing system to build textbooks, courses, lessons, andjournals. Individual authors, major content publishers, and educational organizations all contribute. Lenses provide a way for high-quality materialto be endorsed, recommended, categorized, and aligned to standards. Rhaptos is the open source software that runs the Connexions site andEnterprise Rhaptos is the software for running private content installations. Read more about Connexions and Rhaptos mission and architecturehere.

Enterprise Rhaptos Connexions and Rhaptos Development

API's for Accessing Data Quality Assurance and Testing

Enterprise Rhaptos

Enterprise Rhaptos enables the creation, sharing, modification, and vetting of educational materials accessible to anyone, anywhere, anytime viathe World Wide Web. Rhaptos' unique features of permanent versions of all materials, a simple, semantic document language, and a powerfullensing system for post-publication quality control, customized tagging, and community-based search and discovery are the engine behind a trulyreusable repository of knowledge and learning. Enterprise Rhaptos allows companies and individual organizations to host their own privateauthoring platform and publishing repository.

Installation Instructions for Enterprise Rhaptos

API's for Accessing Connexions Data

Table of ContentsWelcome to Connexions and Rhaptos Software DevelopmentEnterprise RhaptosAPI's for Accessing Connexions Data

Accessing content, feeds, and statisticsDownloading, transforming and branding contentSoftware Architecture and Application Programming Interfaces

Connexions Rhaptos DevelopmentDeveloper Resources (Getting involved)Roadmap, Designs, DesiresStarting Points for Connexions and Rhaptos Software Development

Quality Assurance and TestingTest, Release, and Configuration DocsUsing the Connexions Rhaptos Trac Bug System

Release and Configuration Details for Connexions and RhaptosMore about TRAC


Virtualization of Rhaptos

Two primary reasons to move to the cloud:

1) Hurricane territory (disaster recovery)

2) Increase adoption by other universities

Cost was also a consideration but not a primary reason for carrying out the project.


Deploy to Amazon EC2 with Python scripts

• Define server profiles with OS dependencies and turn into an AMI (Amazon Machine Image)

• Install & configure application on EBS volume

• Take snapshot of volume

• Launch new AMIs and attach EBS volume created from saved snapshot.


Plone Conference 2009 – Federico C. Guizzardi – 21 Aprile 2009

Distributed Architecture

Internet

Webserver

Cache Sys

Load Balancer

s1

s3

s4 s5

s2

10

proxy

frontend1

frontend2

frontend3

backend


[macro:base-securitygroup]connections = tcp 22 22 0.0.0.0/0 tcp 80 80 0.0.0.0/0

[securitygroup:backend]<= macro:base-securitygroupdescription = Bristol backend machine running Zeo server

[securitygroup:frontend1]<= macro:base-securitygroupdescription = Bristol frontend machine running Zeo client 1connections = tcp 8080 8080 0.0.0.0/0

[securitygroup:frontend2]<= macro:base-securitygroupdescription = Bristol frontend machine running Zeo client 2connections = tcp 8080 8080 0.0.0.0/0

[securitygroup:proxy]<= macro:base-securitygroupdescription = Bristol proxy machine running Apache, Varnish, HAProxy

aws.conf - define security groups


[macro:base-instance]image = ami-480df921keypair = bristol-keypairregion = us-east-1placement = us-east-1cinstance_type = t1.microfabfile = fabfile.py

[instance:backend]<= macro:base-instancesecuritygroups = backendstartup_script = backend-startup.ship = xxx.xxx.xxx.xxx

[instance:frontend1]<= macro:base-instancesecuritygroups = frontend1 startup_script = frontend-startup.ship = xxx.xxx.xxx.xxx

[instance:frontend2]<= macro:base-instancesecuritygroups = frontend2 startup_script = frontend-startup.ship = xxx.xxx.xxx.xxx

[instance:proxy]<= macro:base-instancesecuritygroups = proxy startup_script = proxy-startup.ship = xxx.xxx.xxx.xxx


[buildout]parts = backend-script frontend-script proxy-script

[backend-script]recipe = collective.recipe.templateinput = deployment/templates/zope-startup-script.shoutput = deployment/backend-startup.shsvn_url = http://svn.jazkarta.com/.../bristol-buildout/trunkbuildout_config = backend.cfg

[frontend-script]recipe = collective.recipe.templateinput = deployment/templates/zope-startup-script.shoutput = deployment/frontend-startup.shsvn_url = http://svn.jazkarta.com/.../bristol-buildout/trunkbuildout_config = frontend.cfg

[proxy-script]recipe = collective.recipe.templateinput = deployment/templates/proxy-startup-script.shoutput = deployment/proxy-startup.shsvn_url = http://svn.jazkarta.com/.../bristol-buildout/trunkbuildout_config = proxy.cfg


http://svn.jazkarta.com/training-deployment/bristol-buildout/trunk






#!/bin/bashset -e -xexport DEBIAN_FRONTEND=noninteractiveapt-get update && apt-get upgrade -y

echo "----- Install system packages for Plone"apt-get -y install python2.6-dev subversion build-essential

echo "----- Create an unprivileged user zope with password disabled"adduser zope --disabled-password --gecos ""

echo "----- Check out the buildout"cd /home/zopeecho 'p\n' | svn co http://svn.jazkarta.com/.../.../trunk buildout

echo "----- Make the zope user the owner of this directory"chown -R zope:zope buildout

echo "----- Run the buildout as user zope"cd buildoutsudo -u zope python2.6 bootstrap.py -c backend.cfgsudo -u zope ./bin/buildout -v -c backend.cfg

echo "----- Start Zeo processes"sudo -u zope ./bin/supervisordsudo -u zope ./bin/supervisorctl status

echo "----- Make sure that Supervisor starts on reboot"ln -s /home/zope/buildout/etc/supervisor /etc/init.d/supervisorchmod +x /etc/init.d/supervisorupdate-rc.d supervisor defaults




[buildout]backend-dns-name = ec2-174-129-192-63.compute-1.amazonaws.comfrontend1-dns-name = ec2-174-129-116-33.compute-1.amazonaws.comfrontend2-dns-name = ec2-184-73-240-104.compute-1.amazonaws.com

Must define the FQDN of the Zeo clients and

Zeo servers

When these names are resolved *on* the server, the DNS server returns the private IP, which means you avoid paying

bandwidth charges for outgoing/incoming traffic.31Sunday, November 7, 2010

$ ./bin/aws start backend$ ./bin/aws start frontend1$ ./bin/aws start frontend2$ ./bin/aws start proxy

Start up the servers with one command


Fault tolerantEliminating single points of failure.


Plone Conference 2009 – Federico C. Guizzardi – 21 Aprile 2009

Distributed Architecture

Internet

Webserver

Cache Sys

Load Balancer

s1

s3

s4 s5

s2

10


02/21/08

The following diagram summarizes this information.

Heartbeat monitors Zeo server and moves to backup in case of failure

DRDBDistributed storage system similar in principle in RAID,but which works over the network.


document jazhar_diagrams.graffle

modified 26 Oct 2009 20:51+0100 page 2

logical server

architecture

...

plone-fe-1.seas.harvard.eduplone-fe-0.seas.harvard.edu

plone-zope-0.web.private plone-zope-1.web.private

plone-zeo-0.web.private plone-zeo-1.web.private

Zope Client

ZEO Server ZEO Server

Zope Client Zope Client Zope Client

Apache

Varnish

Deliverance

Pound

browser clients

backend

frontend

internet

standard

connection

failover

connection

Apache

Varnish

Deliverance

Pound Supervisor

Supervisor

• HTTP:8221

• HTTP:8222 • HTTP:8223

• HTTP:8220

• HTTP:8889

• HTTP:9999• HTTP:80

• HTTPS:443

• HTTP:8000

• HTTP:9999

ldap

LDAPSupervisor

• HTTP:9999 • LDAP:389• HTTP:8222 • HTTP:8223

Supervisor

• HTTP:9999

host hostpassive

resource

active

resource

Supervisor

• HTTP:9999

• HTTP:8220

• HTTP:8889

• HTTP:80

• HTTPS:443

• HTTP:8000


Improvements to the deployment process

• time to launch a new site was days

• brittle non-repeatable installation process

• delay in procuring hardware and installing OS dependencies

• time to launch a new site was minutes

• streamlined and automated installation process

• virtual machines are launched instantly and OS dependencies are already installed on the disk image.

Before After


FabricScripts to deploy to remote hosts.

http://www.fabfile.org 38Sunday, November 7, 2010

http://www.fabfile.org

http://www.fabfile.org

Fabricset(fab_user='plone',

fab_hosts=['localhost'],

local='/home/plone/prod/',

remote='/opt/zope/prod/',

site='budapest.buildout')

def deploy():

local('cd $(local)$(site)')

local('tar cvfz $(site).tgz buildout --exclude=.svn --exclude=*.pyc')

run('cd $(remote)$(site); ./bin/supervisorctl shutdown')

run('rm -rf $(remote)$(site)')

put('$(site).tar.gz', '$(remote)$(site).tar.gz')

run('cd $(root)$(site) && tar zxf $(site).tar.gz')

start()

def start():

run('cd $(remote)$(site)')

run('./bin/supervisord')


Deploy

$ fab deploy

• Provides a local packaging of the source into a tarball

• Removes the old remotely deployed code

• Puts the source on the remote host

• Restarts the site

What does this one command do?


Benefits of Fabric

• Chain local-command, remote-command, and remote-copy commands

• Substitute variables, and the splitting of distinct operations (like in the case of the 'restart' command) so you run them separately, with a fab restart, for example.


Example Fabric file

• See Alex Clark’s Fabric file:

• http://github.com/aclark4life/Debian-Deploy-Plone/blob/master/fabfile.py


http://github.com/aclark4life/Debian-Deploy-Plone/blob/master/fabfile.py




collective.hostoutBuildout-based recipe for deploying to remote host

http://plone.org/products/collective.hostout




[buildout]extends = buildout.cfg

parts += hostout prod

auto-checkout += collective.hostout

[sources]

collective.hostout = svn https://svn.plone.org/svn/collective/collective.hostout/trunk

[hostout]recipe = collective.hostout

user = root

#password = blahidentity-file = /Users/nateaune/id_rsa-gsg-keypair

pre-commands =

# ${buildout:directory}/bin/supervisorctl shutdown || echo 'Unable to shutdown'# post-commands =

# ${buildout:directory}/bin/supervisord

#effective-user = zope

[prod]

recipe = collective.hostout

extends = hostouthost = ec2-75-101-211-135.compute-1.amazonaws.com

buildout = buildout.cfg

path = /opt/zope/prod


https://svn.plone.org/svn/collective/collective.hostout/trunk

https://svn.plone.org/svn/collective/collective.hostout/trunk

Run the deploy commands

$ ./bin/buildout -NInstalling prod.Generated script '/bin/hostout'.

$ bin/hostout deployInvalid hostout hostouts are: prod

$ bin/hostout deploy prod...


Go to Dylan’s talk

• Deploying with collective.hostout with Dylan Jay.

• Today at 16:10-16:55 in Duchess 1+2.


Sprint idea

• Make a site to launch demo Plone sites using :

• t1.micro instances (free for 1st year)

• Enfold’s Ubuntu package

• xdv to show multitude of themes

• Could later include bundles of popular add-ons: PloneEdu, multimedia, etc.


Questions?

• http://rhaptos.org

• http://pypi.python.org/pypi/mr.awsome

• http://fabfile.org

• http://pypi.python.org/pypi/collective.hostout

•


http://rhaptos.org

http://rhaptos.org

http://pypi.python.org/pypi/mr.awsome

http://pypi.python.org/pypi/mr.awsome

http://fabfile.org

http://fabfile.org

http://pypi.python.org/pypi/collective.hostout

http://pypi.python.org/pypi/collective.hostout

scalable plone hosting with amazon ec2 for rice university's rhaptos open learning platform

Technology

instance demoserver

demoserver keypair

demoserver description

demoserver region

aws aws recipe

server fabfile

amazon ec2

ec2 instances