considerations for building your private cloud.pdf

October 12, 2012

Considerations for Building a Private Cloud

Ryan RichardOpenStack Engineer

[email protected]@rackninja

Thursday, October 18, 12

mailto:[email protected]

mailto:[email protected]

RACKSPACE® HOSTING | WWW.RACKSPACE.COM

Whaaa???

source: http://ken.pepple.info/openstack/2012/09/25/openstack-‐folsom-‐architecture/


http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/

http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/


What is a Private Cloud?

Generally considered to be smaller than a “public” cloud

Less than 100 physical servers (for this talk)

API endpoints may not be publicly accessible

Limited inbound connectivity. Use floating IPs to allow for inbound connectivity

Can be customized for specific workloads (hardware/network/etc)

Company may leverage multiple private clouds



Build with the End in Mind

What are you building for?

A. Are you building for 10 servers? 20? 100?

B. Or are you building 500 instances? 1000? 2000?

C. Or are you building 400 CPUs? 3TB RAM? 100TB disk?



Build with the End in Mind

What are you building for?

A. Are you building for 10 servers? 20? 100?

B. Or are you building 500 instances? 1000? 2000?

C. Or are you building 400 CPUs? 3TB RAM? 100TB disk?

d. ALL OF THE ABOVE



Build with the End in MindExample hardware

12 Physical Cores -‐ 24 w/ Hyperthreading -‐ 48 vcpus w/ 2:1 overcommit ratio

128GB of RAM -‐ 1:1 overcommit ratio

8 x 300GB drives RAID 10 -‐ ~1.2 TB usable disk space

How many instances can I run on this physical host?



Build with the End in MindExample hardware

12 Physical Cores -‐ 24 w/ Hyperthreading -‐ 48 vcpus w/ 2:1 overcommit ratio

128GB of RAM -‐ 1:1 overcommit ratio

8 x 300GB drives RAID 10 -‐ ~1.2 TB usable disk space

How many instances can I run on this physical host?

(total VCPUs / smallest flavor #VCPUs) = maximum # of instances

Double or quadruple this to account for growth -‐ size of fixed network range



Build with the End in MindNetworking

We can build a cloud with 2 networks (3 if using floating IPs)

Host Network (physical machine access, OpenStack services)

Fixed Network (instance network)

Floating network







Floating network

Networking is the important part, get it right!







Floating network


Easy to add physical nodes and/or networks







Floating network



Don’t try to change the fixed network once in production







Floating network



Don’t try to change the fixed network once in production

Easy to add additional floating networks



Build with the End in MindGlance

Disk space on server acting as glance backend (file based) will be a limiting factor.

Good alternatives: Swift, CloudFiles, NFS (locally mounted)

Local disk is considerably faster than the alternatives

Will you be leveraging snapshots? If so, disk space will need to be a serious consideration

If using qcow2, set “snapshot_image_format=qcow2“ to help limit disk usage



Build with the End in MindGlance Performance

Network throughput is a limitation

1000Mb/s = 125MB/s max (expect ~112MB/s realistically)

Large sequential read/writes -‐ RAID5 may be preferred

Lean towards disk bandwidth over raw IOPs

Reduce # of images to allow for more efficient local caches on compute nodes (dramatically increasing performance of instance creation)









Image Size Not Cached Cached

1.4GB 20secs 1sec

16.4GB 2min 21secs 1sec









Image Size Not Cached Cached

1.4GB 20secs 1sec

16.4GB 2min 21secs 1sec

*times from “creating image” to “qemu-‐img

create”



To Swift or not to Swift?



To Swift or not to Swift?Pros

Scalable object storage that works great as a backend for Glance

Can be leveraged as object storage for other parts of the business

Ability to quickly increase the amount of storage available

Extremely stable if designed correctly



To Swift or not to Swift?Pros

Scalable object storage that works great as a backend for Glance

Can be leveraged as object storage for other parts of the business

Ability to quickly increase the amount of storage available

Extremely stable if designed correctly

Cons

Additional expertise needed to run Swift

Architecture (network/swift components) design is important to get right

Depending on initial usage, there may be high up front costs to populate 5 zones



Architecture Examples and Thoughts

1 -‐ 20 physical servers

Single controller (single API endpoint, single scheduler, etc) should suffice

Single network (1Gbps) for instance connectivity and OpenStack services is sufficient

Rackspace “Alamo” installer

20-‐50 physical servers

Single controller (single API endpoint, single scheduler, etc) should suffice

Investigate Swift as a glance backend.

Start looking into ways to break apart various controller services




50-‐100 servers

Keep an eye on the scheduler to make sure it’s not a bottleneck

Strongly consider swift especially for snapshots

Consider Availability Zones/Cells (didn’t make it into Folsom)

Consider “frontend” and “backend” networks for instances




50-‐100 servers

Keep an eye on the scheduler to make sure it’s not a bottleneck

Strongly consider swift especially for snapshots

Consider Availability Zones/Cells (didn’t make it into Folsom)

Consider “frontend” and “backend” networks for instances two or more instance networks?

Set “use_single_default_gateway” in nova.conf



Performance Considerations and Bottlenecks

IO

20-‐40 instances per physical server causes high random IO

Reduce IO as much as possible -‐ i.e. centralized logging

Can be further mitigated with Cinder



Performance Considerations and Bottlenecks

IO

20-‐40 instances per physical server causes high random IO

Reduce IO as much as possible -‐ i.e. centralized logging

Can be further mitigated with Cinder

0" 200" 400" 600" 800" 1000" 1200" 1400" 1600"

compute/host"

compute/host"(no"ht)"

compute/host"(deadline)"

Rs/speed/test13"(deadline"cache=none)"

Rs/speed/test13"(cfq,"cache=none,"no"ht)"

Rs/speed/test12"(cfq"cache=none)"

Rs/speed/test12"(noop"cache=none)"

rs/speed/test13"(cfq,"cache=writeback)"

Rs/speed/test13"(noop,"cache=writeback)"

rs/speed/test12"(cfq,"host"deadline,"cache=none)"

Async&Random&IO&

randW"(direct)"

randR"(direct)"

randW"

randR"

0"

2000"

4000"

6000"

8000"

10000"

12000"

14000"

randR" randW" randR"(direct)"

randW"(direct)"

seqR" seqW" seqR"(direct)"

seqw"(direct)"

Host&vs.&Instance&

compute/host"

Rs/speed/test12"(cfq"cache=none)"



Final Thoughts

Lessons learned

Standardize on a design that works for your organization

Find the right questions to ask

Important to understand OpenStack as a whole

OpenStack is still changing often, keep up to date with current state of the projects



But....

But this is a design summit also

Open to discussions/thoughts/questions


RACKSPACE® HOSTING | 5000 WALZEM ROAD | SAN ANTONIO, TX 78218US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM

RACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM

Rackspace is hiring

www.rackertalent.com


considerations for building your private cloud.pdf

Documents

physical host

network instance dont

physical servers

physical cores

networks openstack services

architectu rackspace

aboverackspace hosting

public cloud