bca2053-virtualizing ms exchange 2010 at general mills - architecture details and lessons...

7/27/2019 BCA2053-Virtualizing MS Exchange 2010 at General Mills - Architecture Details and Lessons Learned_Final_US.pdf

1/27

Virtualizing MS Exchange

2010 at General Mills -Architecture Details andLessons Learned

Aaron Kopel, General Mills

Curtis Witt, VMware, Inc.

APP-BCA2053

#vmworldapps


2/27

2

Disclaimer

This session may contain product features that are

currently under development.

This session/overview of the new technology represents

no commitment from VMware to deliver these features in

any generally available product.

Features are subject to change, and must not be included in

contracts, purchase orders, or sales agreements of any kind.

Technical feasibi lity and market demand will affect final delivery.

Pricing and packaging for any new technologies or features

discussed or presented have not been determined.


3/27

3

Session Objectives & Agenda

General Mills Background

Exchange 2007 Architecture


Business Drivers and Benefits

VMware Environment and Configuration

Key Takeaways

Q & A


4/27

4

General Mills Background

Global Company

Headquartered in

Minneapolis, MN 35,000 employees

across 80+ countries

$16.6B annual revenue

Virtualization Highlights

Virtualizing since 2007

94% virtual at HQ,74% virtual globally

VMware in 80+ sitesacross 20+ countries

vSphere ESXi 4.1 and 5.0

2200+ server VMs on 250+ hosts

1200+ View 5.1 users

Lab Manager


5/27

5

General Mills Exchange Landscape

32,000+ Mailboxes

Highly mobi le user base 10,000+ Mobile devices

Utilize Microsoft Unified Messaging

Regional centralization Exchange infrastructure in 8 sites

US: 25,735

UK: 3,030

Australia: 593

China: 824

Hong Kong: 838

India: 561S. Africa: 103

Argentina: 958


6/27

6


Started physical and migrated to virtual

Virtualized everything except the Cluster Continuous Replication

(CCR) clusters

Server roles in Minneapolis

4 Client Access (CAS) servers

2 Hub Transport (HUB) servers

2 Unified Messaging (UM) servers

3 pairs of CCR clusters (Physical Servers with DAS)

Server roles at remote sites

9 all-in-one servers


7/27

7


Started 100% virtual never ran physical

Server roles in Minneapolis

6 CAS servers

4 Hub servers

3 UM servers

One 7-node Database Availability Group (DAG)

Server roles at remote sites

7 all-in-one servers

Performance and sizing

Microsoft Exchange 2010 Mailbox Server Role Requirements Calculator

VMware Exchange 2010 on VMware Best Practices Guide


8/27

8

Exchange 2010 DAG Architecture

Single 6-node DAG (split across 2 sites), each host ing 8 active and

8 passive databases

Approximately 4,200 mailboxes per DAG node (25,000+ Total)

2GB Mailbox Quota

All mail groomed with 1-year max retention

Each nodes active databases replicate 2nd copy to other DAG node

VMs, never on the same host or in the same datacenter

7th DAG node houses 3rd copy of all 48 passive databases for

backup processing


9/27

9

Exchange 2010 DAG Database Layout


10/27

10

Virtualization Business Drivers and Benefits

Agility

Right size VMs initially, and hot-add vRAM and vCPU as needed

Easily expand shared storage to dynamically grow

Ability to quickly add additional servers

Operational Simplicity and Efficiency

No need to create one-off architecture

Core Compute, Network & Storage is the same as the rest of the environment

Easier planned maintenance


11/27

11

VMware Environment

Started on ESXi 4.1, now on ESXi 5.0

8 ESXi blade hosts dedicated for Exchange

Blade enclosures are shared with other general purpose ESXi hosts

No Exchange-purposed blade hosts share the same enclosure

No two DAG nodes share the same blade host

Everything located at 2 sites, interconnected with private fiber


12/27

12

Compute / Memory Config


13/27

13

HA / DRS / vMotion

HA Admission Control enabled with 25% CPU / Memory reserved

for failover capacity (2 hosts)

DRS Set to Fully Automated priority 1 & 2 recommendations

DAG Nodes set to Manual (no DRS)

Using DRS Groups / Rules to pin VMs to Primary or DR Datacenter

and to separate like Exchange server roles.

Succeeding with vMotion and DAG nodes

SameSubnetDelay increased from 1000ms to 2000ms

SameSubnetThreshold left at default of 5

vSphere 5 improved vMotion performance for DAG nodes

Work very closely with Exchange team to test / qualify this!!


14/27

14

Compute / Memory Lessons Learned

VMs created with hot-add CPU and Memory enabled

Reserved all memory for DAG nodes

Not using CPU reservations

Most server roles required increases from original CPU/memory

configurations

DAG: 46 vCPUs and 24GB48GB

CAS: 816GB

UM: 24 vCPUs and added another server

Make sure to size CPU / Memory for failover / maintenance capacity


15/27

15

Datacenter Network Config


16/27

16

Host Network Config


17/27

17

Networking Results and Lessons Learned

Using VMXNET3 (default settings)

Mailbox VMs have 2 vNICs connected to Exchange and Backup

Networks

Overall daytime bandwidth averages ~2Gb/s

Separated vMotion / Backup network to relieve production network

bandwidth


18/27

18

Datacenter FC SAN Config


19/27

19

Disk / LUN / VMFS Layout


20/27

20

DAG Virtual Machine Storage Config


21/27

21

Storage Results and Lessons Learned

You can succeed with shared storage!

Make sure your SAN has the required capacity and throughput.

(Run Jetstress!)

Plan for Background Database Maintenance

(runs 24x7 @ 5MB/s/Database)

Total Exchange IOPS (all roles) is around 8000IOPS + 3000 for

7th

DAG node Needed to dedicate 4 FA ports per site for Exchange on Tier-1 SAN

Each FA port could handle about 4000IOPS

Increase VMFS heap size to 256 MB

Allows for increase from 20 TB to 64 TB of open VMDK capacity per ESXi host Needed for the gigantic 7th DAG node

Details at: http://kb.vmware.com/kb/1004424


22/27

22

Uber Diagram


23/27

23

Key Takeaways

1. Be on the good side of your Exchange and Storage Teams!

2. Enable hot-add vCPU and vRAM for all VMs easily adjustsizing for all server roles.

3. Design storage so it is quick and easy to grow!

4. Compared with Exchange 2007, disk I/O is definitely reduced,but at the expense of CPU and Memory

5. PowerCLI is handy for configuration of complex VMs(lots of controllers and VMDKs).

6. Review the VMware E2K10 Best Practices Guide(s)

http://www.vmware.com/files/pdf/Exchange_2010_on_VMware_-_Best_Practices_Guide.pdf

http://www.vmware.com/files/pdf/using-vmware-HA-DRS-and-vmotion-with-exchange-2010-dags.pdf


24/27

24

After The Conference

APP-BCA1684 - Virtualizing Exchange Best Practices

By Alex Fontana, VMware

(Monday, 1:00 PM 2:00 PM and Tuesday, 3:30 PM 4:30 PM)

Review content for this session and Alexs session

after the conference, at http://www.vmworld.com


25/27

25

THANK YOU

Questions?


26/27

FILL OUT

A SURVEY

EVERY COMPLETE SURVEY

IS ENTERED INTO

DRAWING FOR A

$25 VMWARE COMPANY

STORE GIFT CERTIFICATE


27/27

Virtualizing MS Exchange

2010 at General Mills -Architecture Details andLessons Learned

Aaron Kopel, General Mills

Curtis Witt, VMware, Inc.

APP-BCA2053

#vmworldapps

bca2053-virtualizing ms exchange 2010 at general mills - architecture details and lessons...

Documents