sizing exchange on dell servers with fc storage …€¦ · web viewthis document explains the...

Sizing Exchange 2000 on Dell PowerEdge Servers

and Dell/EMC Fibre Channel Storage

SystemsEnterprise Systems Group (ESG)

Dell White PaperBy Guy Westbrook

January 2003

Contents

Executive Summary.............................................................................4Consolidation......................................................................................5Tools Used to Analyze Performance....................................................6Microsoft Exchange 2000 Architecture and Implementation.............8

Hardware and Software..................................................................8Sizing the Hardware..........................................................................10

Exchange Database Recommended Best Practices......................10Optimize Transaction Log............................................................10Optimize Database File.................................................................11RAID Level and Stripe Size.....................................................................................11

Sizing the Hardware.....................................................................13Server Components.......................................................................14Peripheral Component Interconnect (PCI) Bus...........................................................14Processor Scalability..............................................................................................14Greater than 2GB User Memory Support...................................................................14

Windows 2000 Memory Management Setting..............................15Exchange Server Memory Considerations....................................15Windows 2000 Server Disk Subsystem Setup...............................16Exchange Server I/O Subsystem Setup.........................................16DELL | EMC Fibre Channel Storage System FC4700..................................................16

System Health and Performance Monitoring....................................18OS Monitoring..............................................................................18Windows 2000 Performance Monitor...........................................18Exchange Server Monitoring........................................................18Basic Counters for Server Monitoring..........................................19CPU Performance Monitor Objects and Their Meaning................................................19

Memory Utilization, Monitoring and Optimization......................20Window 2000 Server Paging: Detection and Avoidance...............................................20

January 2003 Dell Enterprise Systems Group

Disk Subsystem Utilization, Monitoring and Optimization..........20Diskperf...............................................................................................................20

Disk Performance Monitor Objects and Their Meaning...............21Network Utilization, Monitoring and Optimization........................................................22

Basic Counters for Exchange 2000 Monitoring...........................22Conclusion.........................................................................................23

Contacts........................................................................................23APPENDIX A: Reference Documents................................................24

FiguresFigure 1: Messaging Infrastructure Diagram Storage Attached Network (SAN).......7Figure 2: Average Response Time (ART) with Five or Nine Drives Running RAID 5

for X users (response times in milliseconds).......................................................13

TablesTable 1: Average Disk Access Times.........................................................................21

January 2003 Dell Enterprise Systems Group

Section 1Executive Summary

This paper is designed as a guide to assist with decisions relevant to deploying and sizing hardware for a Microsoft® Exchange 2000 messaging infrastructure. This document explains the basics of how to size servers and storage for an Exchange 2000 messaging infrastructure rollout. It includes considerations related to sizing the hardware, tuning the operating system (OS) and Exchange 2000 Server application, and monitoring performance related issues. Finally, this paper provides best practices on choosing hardware, configuring software, and deciding which performance counters to monitor. As a prerequisite, Dell recommends that the reader be familiar with the installation and basic administration of Microsoft Windows® 2000 Advanced Server and Exchange 2000. Additionally, this paper touches on Dell’s Server and Storage Consolidation tools, and makes recommendations of RAID configurations based on application function and cost.

January 2003 Dell Product Group

Section 2Consolidation

Organizations are constantly looking for ways to work more efficiently, and do more with less. This applies to messaging solutions as well, where the major concern is return on investment (ROI) along with results and better performing messaging solutions. Server and storage consolidation is a means used by many businesses to help increase productivity. Server consolidation is a way to restructure an organization's infrastructure in order to reduce costs, centralize administration, and optimize available resources. Dell offers tools that help evaluate an environment for server consolidation and determine the estimated return on investment by consolidating server and storage resources.The Dell Server Consolidation ROI Analyst Tool helps users to rapidly and comprehensively assess server and storage consolidation needs. This tool performs a financial assessment of consolidating an infrastructure, and generates reports that demonstrate the financial advantages of consolidation. Dell Professional Services offers assistance in planning, implementing, and supporting Microsoft-based consolidation projects. Dell account managers work with customers to enter information about their organizations, assess risks, and develop comprehensive analysis.The potential benefits of server consolidation are as follow: Reduced Costs Improved control of manageability of servers Improved service levels and efficiency of operationsDell has designed server and storage consolidation programs that offer the resources you need to maximize the return from your IT investments, including the right tools, products, and services. Our proven expertise goes into every step of the consolidation process, including the critical first step of preparing a business case. The Server Consolidation ROI Analyst Tool helps evaluate the impact of consolidation. For a quick estimate of the projected benefits and savings of a server and storage consolidation solution, Dell has a Server ROI Calculator and a Storage ROI & TCO Calculator. Please contact a Dell Account Manager for further information.


http://www.dell.com/us/en/esg/topics/products_storage_cons_main_pedge_roi_form_storage.htm

http://www.dell.com/us/en/esg/topics/products_server_cons_main_pedge_roi_form_server.htm

http://www.dell.com/us/en/esg/topics/products_roi_main_pedge_000_roi.htm


Section 3Tools Used to Analyze Performance

There are tools available that can help size the hardware needed to support a messaging infrastructure. Benchmarking tools can assist in comparative analysis of hardware, the selection of hardware, and the sizing of the hardware components. Loadsim, developed by Microsoft, is one of the most popular tools for Microsoft Exchange 2000 Server analysis. Loadsim is used to compare vendor hardware for Exchange 2000 performance. The new Loadsim MAPI Messaging Benchmark (MMB2) measures throughput in terms of a specific profile of user actions, executed over an 8-hour working day. Results should be interpreted as a benchmark for messaging throughput and should not be confused with deployment recommendations. Factors such as server and network topology, backup and restore requirements, and other issues should be considered when planning a deployment. All of these factors can have a significant affect on hardware decisions and system performance. For example, in tests conducted at Dell’s performance lab in Austin, Texas in July 2002, the Dell™ PowerEdge™ 6650, used as the System Under Test, demonstrated the ability to support over 11,000 MMBs. Microsoft’s LoadSim tool was used to simulate 11,304 Exchange (MAPI) e-mail users running on a Dell PowerEdge 6650 Enterprise Server. During this period, the PowerEdge 6650 provided a weighted 95th percentile response time of 137 ms for 11,304 users, with overall CPU utilization of 67%. Results are based on 4 hours of steady state running. The most current performance data and comparative analysis for the MMB2 benchmark can be found at the following URL:http://www.microsoft.com/exchange/techinfo/planning/2000/PerfScal.aspIn addition, sizing tests on the PowerEdge 6650 were conducted at the Solutions Enablement and Showcase Lab in Austin, Texas in September 2002. Loadsim was used to stress test the Exchange server. Figure 1 depicts the complete configuration tested, with fully redundant paths for high availability.


The Dell PowerEdge 6650 (PE6650) is a 4U, rack-mountable 4-way server that incorporates high performance, scalability, and availability features for enterprise users. The PowerEdge 6650 is targeted as a business-critical enterprise system to be deployed in data centers, for clustering, and in storage area network (SAN) environments.

The PowerEdge 6650 supports large on die cache providing up to 1.25MB per processor, for large footprint applications like OLTP databases and data warehouses. For memory intensive applications, the PowerEdge 6650 offers up to 16GB synchronous dynamic random access memory (SDRAM), which includes "Chipkill," "Spare Bank," and "Memory Mirror" support.

http://www.microsoft.com/exchange/techinfo/planning/2000/PerfScal.asp


ow rVa lt56F

16 port FC switch

Dell/EMC FC4700 Fibre Channel Storage

16 port FC switch

PowerEdge 6650 System Under Test

Pwe E ge4 0

Client 1 Client 2

P we Ege4 0

Client 8

Pwe E ge4 0

ow rVa lt56F ow rVa lt

56F

16 port FC switch


16 port FC switch


Pwe E ge4 0

Client 1 Client 2

P we Ege4 0

Client 8

Pwe E ge4 0

ow rVa lt56F ow rVa lt

56Fow rVa lt

56F

16 port FC switch


16 port FC switch



Pwe E ge4 0

Client 1 Client 2

P we Ege4 0

Client 8

Pwe E ge4 0

Pwe E ge4 0

Client 1

Pwe E ge4 0

Pwe E ge4 0

Client 1 Client 2

P we Ege4 0

Client 2

P we Ege4 0

P we Ege4 0

Client 8

Pwe E ge4 0

Client 8

Pwe E ge4 0

Pwe E ge4 0

ow rVa lt56F

ow rVa lt56F

Figure 1: Messaging Infrastructure Diagram Storage Attached Network (SAN)

Note: While benchmarks are useful guidelines, they are based on controlled lab simulations rather than a specific environment and needs.


Section 4Microsoft Exchange 2000 Architecture and Implementation

With significantly enhanced features and newly developed functions, Microsoft Exchange 2000 is a major revision of Exchange 5.5. Architecturally, Exchange 2000 does not use a self-contained messaging system like Exchange 5.5. Exchange 2000 separates many components and tightly integrates them with the underlying Windows 2000 operating system and Microsoft Internet Information Server (IIS). These improvements can help reduce the overhead on the Exchange servers, enable consolidation of the users onto fewer servers, and unify the network and messaging administration, all of which contribute to lower total cost of ownership (TCO).Dell and EMC have partnered to provide fibre channel storage products that offer high performance, flexibility, manageability, and scalability in a backend storage system that can be optimized for Microsoft Exchange 2000 database technology. Dell’s Storage Area Network (SAN) uses fibre channel and RAID technologies to provide the protection, speed, flexibility, and data-storage capacity necessary to meet the performance requirements of today’s complex server networks. SANs provides great ability for storage side consolidation.

Hardware and SoftwareDell tested an Exchange 2000 configuration with the Dell PowerEdge 6650 as the Exchange 2000 server. Eight PowerEdge 6450 systems were used as load simulator clients. Each PowerEdge 6450 was configured with four Intel® Pentium® III Xeon™ 900MHz/2MB L2 cache CPUs and 4GB of memory. One Dell/EMC FC4700 and two Dell/EMC DAEs were used as the fibre channel storage subsystem for the Exchange database. All enclosures were populated with 36GB, 10K rpm fibre channel disks, and the FC4700 had two storage processors, each with 1GB on-board configurable cache and two Intel Pentium III 733MHz processors. For storage subsystem connectivity, the Exchange server contained two Emulex 9002 Host Bus Adapters (HBAs). Two Dell/EMC 16DBS fibre channel 2 switches were used for the switched fabric.Dell has partnered with Microsoft for over 15 years, and is the #2 provider worldwide of Exchange-based servers, with 21% server hardware unit share, according to IDC.1 This relationship has helped

1 IDC's Server Workloads 2002 Model - Server Customer Revenue ($M) and Shipments, 2001 by Collaborative Workload, Operating System, Vendor, ISV and Priceband


the two companies join together to provide integrated solutions with high availability, reliability, scalability and performance. Microsoft Exchange 2000 on Dell’s PowerEdge Severs has been at the forefront in performance. Dell servers have consistently been among the top performing servers running Exchange 2000, as mentioned on page 6.Dell/EMC Storage Area Network (SAN) products not only provide redundant but also competitive price/performance for small, medium, and enterprise level users.

(August, 2002)


Section 5Sizing the Hardware

Exchange Database Recommended Best PracticesThe disk I/O subsystem is the most critical component in sizing the Exchange environment. The goal in tuning the Exchange environment for performance is to decrease response time yet support more users. This can be done by adding more physical disk drives or spindles. Exchange 2000 Server uses random I/O and 4KB to 7KB transfer sizes for the database and 100% sequential I/O, 4KB transfer writes for the transaction logs. Efficiency of the writes can be improved through the use of write-back cache. The DELL/ EMC storage subsystems can provide onboard cache of up to 1GB per Storage Processor. The cache is write-mirrored protected and both read and write cache can be used for the read I/O. Since I/O patterns for the database and the transaction log are different, they should be configured and tuned differently.

Optimize Transaction LogTransaction logs are important since they are key to the recovery of the Exchange database. For small user environments of up to a few hundred users, for example, the effect of transaction log optimization is not significant. But for large user environments, optimization of transaction log files can provide performance gains and prevent bottleneck issues relating to the transaction logs. Transaction logs bottlenecks can degrade performance of the entire Exchange environment. What follows are recommended best practices for optimizing transaction logs: Use a dedicated LUN for the transaction log files Do not put multiple transaction log files on the same LUN Enable write-back mode on the LUN Use RAID 1 (2 spindles) or RAID 5 (3 spindles) for

transaction logsOne optimization parameter that can be tuned to increase transaction log performance is the transaction log buffer size. This is from the server side, and does not affect the cache on the backend storage system. Exchange 2000 comes with a default of 84 buffers set for transaction logs. These buffers are used by the database engine to store transactions until they can be committed to the disk. In a large environment where I/O might be busy, the buffers can fill up, causing write delays. The Transaction log buffer is storage group-based. To optimize the log buffer, use ADSIEDIT to find the storage group, and then find the msEXchESEParamLogBuffers object and change the size from 84 to 9,000. See the section on System Health and Performance


Monitoring to determine which counter to examine to determine if the log buffers need tuning.

Optimize Database FileThe I/O patterns for the Exchange database are different than that of the transaction logs. Although read cache is not significant for database access performance based on this pattern, the use of the write cache will lower response time. Users can change the distribution of the memory of each of the storage processors (SP) in the DELL/EMC FC4700 based on this pattern. The FC4700 has 1GB of cached memory. Dell recommends setting 50MB for read and the rest for write caching. This is because I/O request uses both read cache and write cache to fetch data from disks. In addition to the need to increase spindle count to increase the number of concurrent supported users, there other factors in sizing Exchange to determine the total storage requirement. First, it is necessary to figure out how many users will be supported and how big each user mailbox can grow. Next, the deleted items retention policy requires storage. Dell recommends 25% additional space for deleted items retention. Lastly, whether circular logging is turned on or off will have to be factored into the total space requirements.

RAID Level and Stripe Size RAID not only can provide protection for data, but it also can improve performance by adding more spindles to the array group. Both RAID 5 and RAID 10 provide data protection, but come with different costs. For the same amount of disks, RAID 5 provides more storage space than RAID 10. Both have the same read performance, but on write I/O, traditionally RAID 10 outperforms RAID 5 because RAID 5 needs to calculate and write parity information to the disk. This characteristic of RAID 5 can be a performance burden if there are a lot of small I/Os. For the Exchange I/O environment, since databases are typically averaging 4K random I/Os, RAID 10 typically performs better than RAID 5. However, the new DELL/EMC storage subsystem has a more efficient RAID 5 algorithm that decreases the write penalty for RAID 5. Based on the above assessment, Dell recommends RAID 5 for Exchange environments that require fault tolerance. Dell used RAID 5 for sizing tests. Although some recommend using the default stripe size, which is 64KB stripe (128 elements), Dell tests show that 32KB performs slightly better than 64KB, so Dell recommends 32KB stripe size. Figure 2 shows a test with 5 drives with RAID 5, in a 4+1 RAID set. The maximum number of users Dell recommends for this number of physical drives or spindles is 2,000 MMB2 users with a sub-second response time. With more than 2,000 MMB2 users, the response time increased to greater than 4 seconds. With Exchange 2000, once the threshold is reached, the queues will continue to grow. This will cause unpredictable behavior on the Exchange Server and the system may have to be rebooted to recover to a stable state. As a result of this testing, Dell recommends limits that will preclude reaching these thresholds and thus causing serious problems.


Figure 2 also shows a test with nine drives with RAID 5 in an 8+1 RAID set. The maximum number of users recommended for this number of physical drives or spindles was shown to be 4,000 with a sub-second response time. For implementations that require over 4,000 MMB2 users, Dell recommends that the LUN design be planned to correspond with user requirements. A generic formula can be used to determine the number of drives required for a specific number of users. The formula is:

150 I/Os per drive times number of drives = total I/Os per second

Total I/Os per second divided by .35 = number of users .35 I/Os per user is the number of I/Os for each user Dell recommends for planning purposes. This number was derived from MMB2 testing, and represents the total number of users that can be supported by RAID 5 and number of drives and still achieve a sub-second response time.Although disk drives are rated by the manufacturers for higher throughput rates, our recommended number of I/Os per disk drive is 150. Tests have shown that over 150 I/O per drive disk performance results in degradation, causing a significant increase in the Average Response Time (ART) and it can also result in unstable system behavior, see Figure 2.


Figure 2: Average Response Time (ART) with Five or Nine Drives Running RAID 5 for X users (response times in milliseconds) (Testing done by Dell in July 2002)

Sizing the HardwareWhen sizing hardware we first need to take a look at server capabilities. There are a number of requirements for determining the appropriate hardware, including the following: Number of users and their usage patterns Concurrent vs nonconcurrent Typical concurreny rate for messaging servers is 30-35% of

total number of users Number and speed of Processors Hyperthreading Support Level 2 and L3 cache size Maximum amount of RAM possible Number of PCI slots The number and speed of network interface cards Number of drive controllers Number, capacity, and speed of hard drives Stripe size for data - recommended 32k RAID configuration and/or fault tolerant requirements


Failover requirements (clustering) High Availability and Scalability

The backup and restore requirements will be a major factor in determining the number of users to put on a single server.

Response Time Criteria

Server ComponentsMost performance problems occur from a lack of sufficient hardware, improper hardware configuration, or an Exchange Server configuration that is not optimized. Sizing the system usually involves three main components: CPU, memory, and I/O subsystem, as well as other related hardware components. Utilizing the PowerEdge 6650 as a guide for sizing, this section will cover each of those components.

Peripheral Component Interconnect (PCI) BusThe PCI bus is a high-performance, 32 or 64-bit local bus(s) that provides an interface for high-speed transfers to peripheral components without overloading the host processor. Generally, when installing an even number of network or array controllers, it is best practice to split the controllers evenly between the buses. In general, network controllers consume more bandwidth than array controllers.

Processor ScalabilityWith the rapid growth of processor technology, systems today are performing tasks faster and faster. The scalability of systems has increased along with processing power. The front side bus and three levels of cache, available in the latest MP Xeon processors, have greatly improved within the processor subsystem. The processor subsystem gives the server the capability to handle an increasing number of users without compromising response time. Dell recommends getting the fastest processor with the largest amount of cache options. This will allow for better initial performance today and it will also support future growth.

Greater than 2GB User Memory SupportBy default, Microsoft Windows 2000 Advanced Server permits a maximum 2GB of the installed memory (the PowerEdge 6650 supports up to 16GB) to be allocated as user memory. With the advent of Microsoft Windows 2000 Advanced Server, user applications like Exchange 2000 can utilize up to 3GB of memory. With a 4 GB system configuration, 1GB of memory will be reserved as system memory for use by the Windows 2000 Advanced Server system and is unavailable as user memory. After installing Microsoft Windows 2000 Advanced Server, it is possible to modify a system to use the full 4GB of memory. To enable the 3GB user memory feature you will need to edit the system hidden file c:\boot.ini and add the command line option /3GB to the boot up string and reboot your system.


[boot loader]timeout=30default=multi(0)disk(0)rdisk(0)partition(0)\WINNT[operating systems]multi(0)disk(0)rdisk(0)partition(0)\WINNT="Windows 2000 Advanced Server [3GB user]" /3GB4.00 [VGA mode]" /basevideo /sos

Windows 2000 Memory Management SettingTo help reduce excessive paging, check the Windows NT Registry setting for LargeSystemCache under HKEY_LOCAL_MACHINE, System, CurrentControlSet, Control, Session Manager, Memory Management. This value should be set to zero to instruct Windows 2000 to favor the process working set over the system cache.Refer to the Microsoft Windows 2000 Advanced Server, Concepts and Planning Guide that’s included with your software for a more in-depth discussion of the paging file.

Exchange Server Memory ConsiderationsWith Exchange 2000 and Windows 2000 Advanced Server, it is possible to allocate 3GB of RAM to Exchange 2000 Server. Windows 2000 can now take advantage of greater than 4GB of RAM with Physical Address Extension (PAE) support. The current release of Exchange 2000 Server does not support PAE. For a server with a large number of users, it is possible to specify that Windows 2000 needs approximately 64MB for itself. Included in this 64MB will be some memory allocated for the System Cache. Minimize the System Cache size by selecting Maximum Data Throughput for Network Applications. With this setting, Windows 2000 will page memory used by the System Cache before it will page memory used by Exchange 2000 Server. In the case where the system is running low on memory resources, the Exchange 2000 Server processes will begin to have their memory paged. This is usually the first warning sign that it is necessary to add memory to the server. “How much RAM do I really need?” is a frequently asked question. Dell recommendations vary between 300KB to 1MB per user depending on the users’ activities. The following formula has been used during benchmarking for determining the memory requirements:

256 + (#concurrent users/5)MBUsage patterns, and whether the users work directly on the server rather than replicating databases to their own computers and freeing processing resources on the server, will factor into determining the amount of required memory. A key optimization parameter that can be set is the Mailstore Database Cache Size on Exchange 2000, which has a default value of 900MB. On servers with more than 2GB of memory, it may be beneficial to increase the size of this cache. Because of virtual


address space limitations, this value should never be set higher than 1,200 MB.The 900MB default limit is in place to ensure that the Store.exe process always has ample virtual address (memory) from which to allocate. Increasing this value too much can lead to system instability. The information gathered from the Performance Monitor will give you an accurate value for the virtual address space the Store.exe process has allocated. On a server with the /3GB switch set in the Boot.ini file, the value seen in the performance monitor should be below 2.8 GB. On a server without the /3GB switch set, the value should be below 1.8GB. If you see values that are higher than those above for either configuration, do not increase the size of your maximum cache size. If you see values that are lower for either configuration, you can safely increase the size of your database maximum cache size. To modify, use the ADSI tool and select msExchESEParamCacheSizeMax attribute and adjust the value.

Windows 2000 Server Disk Subsystem SetupFor the Windows 2000 Server Operating System setup, Dell recommends that two drives are set up as a RAID 1 mirrored set. These two drives allow for these drives to be used as redundant boot devices, which frees the remaining drives for storage. Also, the Exchange 2000 executables can be placed on this mirrored set.

Exchange Server I/O Subsystem Setup

DELL | EMC Fibre Channel Storage System FC4700As a write intensive application, the RAID configuration Dell recommends for Exchange Server is RAID 5 on the drives containing the mail database files. This approach provides the performance benefits of the DELL/EMC RAID-5 implementation.Recommended RAID configurations differ based on unique customer environments and situations. If performance is the only factor to be considered, and fault tolerance is a non-issue, RAID 0 will best meet the user’s needs. Dell recommends RAID 0 for areas such as disk backup, because RAID 0 can provide faster I/O than other RAID levels and fault tolerance is not a requirement. However, most messaging application deployments demand fault tolerance for mail databases and transaction logs. In these cases, RAID 5 provides an effective combination of performance, economy, and reliability.Scaling refers to the method of distributing the load to multiple devices. RAID uses multiple drives functioning as one logical unit to complete requests faster than a single drive. In the case of hard drives, it is possible to scale by adding more identical drives, or by dividing a desired storage capacity between a different number of drives. Adding drives to an array provides greater throughput regardless of storage capacity and fault tolerance level. Carefully choose what parameters to maximize for the system: cost, performance, capacity, replaceability, or future expandability. Saving money now may cost more in the long run when a failed drive


becomes harder to find, or when there are not enough spindles available to handle growing I/O demands. It is up to the individual users to determine which type of drive scaling is right for their server workload. For environments with more random and small-block transfers, such as messaging, they will enjoy greater performance benefits with more drive spindles.


Section 6System Health and Performance MonitoringOS Monitoring

Windows 2000 Server Performance Monitor can be used to monitor the performance and health of an Exchange Server. The results from monitoring can be used to determine system bottlenecks, plan for future hardware expansion, or simply to help understand the complex interactions between the Windows 2000 operating system, Exchange Server, and the hardware.

Windows 2000 Performance MonitorThe Windows 2000 Performance Monitor can collect information from the performance counters and display them in real time or store them in a data file for later review. This information can be used to determine system level and application level bottlenecks, or to determine long term trends in system wide resource utilization. Please refer to the Microsoft Windows 2000 Advanced Server Resource Kit, which is a separate purchase from your software, for a concise and in-depth discussion of the performance counters and their meaning. For information on performance monitoring of the network, please refer to the section entitled Using Performance Monitor with TCP/IP Services in the Microsoft Windows 2000 Server Resource Kit. Additionally, an optional piece of software can be used to gather historical and statistical data with Dell/EMC storage systems. Data Analyzer can be purchased separately and used to predict trends, detect bottlenecks, optimize storage components, and proactively manage a storage system.

Exchange Server Monitoring Exchange Server has its own useful statistics that are generated for monitoring the performance of Exchange. These help with tracking performance or system behavior because they provide specific performance information on components within the Exchange server environment. There are numerous counters and the Exchange administrator should determine which statistics are critical for monitoring in their environment.


Basic Counters for Server Monitoring

CPU Performance Monitor Objects and Their MeaningThis section highlights important Performance Monitor objects that can be monitored to gain insight into the system processor utilization.Object: ProcessorCounter: % Processor TimeThe % Processor Time counter monitors the percent of CPU utilization of all threads (Privileged Mode and User Mode threads) running on the system. This counter is useful in determining whether or not there is a CPU bottleneck. Consistent CPU utilization of over 75% on all processors is a safe indicator of a CPU bottleneck. In the instance of a bottleneck, add another system processor or upgrade the existing processor(s) to the next generation processor(s). It may also be necessary to review the logical database design to make sure it is as efficient as possible. Hardware tuning and upgrades can only partially compensate for an inefficiently designed database.Object: ProcessorCounter: % Privileged Time, % User TimeIn a dedicated messaging environment, % Privileged Time is typically the percentage of time the system processor is spending executing NT kernel commands; much of this time is associated with processing Exchange I/O requests. % User Time is the percentage of time the system processor is spending executing user applications, such as Exchange.If Exchange finds all (or most of the frequently used) objects in the data cache, and does heavy processing with these objects (such as table joins), very little I/O is generated. As a result, % Privileged Time can be low, in the 5-15% range, whereas % User Time can be as high as 85-95%. On the contrary, if Exchange generates a large amount of I/O, % Privileged Time will be substantially higher (30-40%) and % User Time will be substantially lower (60-70%). Both of these counters are useful in determining how different types of operations are utilizing the system processor(s). If a system is spending too much time doing I/O, it may be necessary to further investigate the disk subsystem and determine how to relieve it of some I/O. And it also may be necessary to add more memory. Object: Processor Queue LengthCounter: Processor Queue LengthThis is the number of threads contending for the processor; basically threads in ready state. Windows 2000 Server schedules processors using symmetric multiprocessing with soft affinity and preemptive multitasking. On a single processor system this means that the highest priority thread that is ready to run will execute processor instructions. If a computer has multiple processors, a ready thread can run on any of them. The system attempts to run a thread on the same processor it last ran on, all other things being equal. (This is soft affinity.)


Memory Utilization, Monitoring and OptimizationThis section provides information on parameters that influence performance based on system memory, and the tools available to monitor the behavior of the system.

Window 2000 Server Paging: Detection and AvoidanceIn a dedicated Exchange 2000 environment, Windows 2000 Server paging can occur if there is too much memory dedicated to Exchange and the remaining memory does not satisfy Windows 2000 Server.Object: MemoryCounter: Page Faults/secThis counter monitors total page faults by all running processes. A page fault occurs when a process makes a reference to a virtual memory page that is not in its working set in the memory. It is also possible to monitor Page Faults/sec per individual process, such as the number of page faults per second Exchange generates. On a dedicated Exchange system, Exchange-generated page faults and total page faults per second will often be identical.There should be very little or no paging once the system reaches a steady state; occasional paging prior to steady state is often acceptable.Object: MemoryCounter: Pages/secPages/sec is the number of pages read from the disk or written to the disk to resolve memory references to pages that were not in memory at the time of the reference. This is the sum of Pages Input/sec and Pages Output/sec. This is the primary counter to observe to monitor excessive memory pressure.

Disk Subsystem Utilization, Monitoring and OptimizationThis section provides information on parameters that influence performance from the disk subsystem perspective and tools available to monitor the behavior of the system.

DiskperfThe diskperf option allows for the monitoring of the disk subsystem activity, and is especially useful for monitoring performance of the drive subsystem. If this option is disabled, performance monitor will not be able to monitor low level disk-related activity, such as LogicalDisk and PhysicalDisk counters. Having this option enabled slightly degrades performance. Enable diskperf only when needed, then disable it to get maximum performance. To enable or disable diskperf, issue the diskperf command from the system prompt. It is necessary to restart the system for the diskperf option to become effective.


Disk Performance Monitor Objects and Their MeaningIt is possible to monitor LogicalDisk and PhysicalDisk related objects, such as Avg. Disk sec/Transfer or Disk Transfers/sec. Monitoring LogicalDisk and PhysicalDisk related objects requires that the diskperf option be enabled, which slightly reduces performance. Note that resetting the diskperf option requires a system reboot.Object: Logical DiskCounter: Avg. Disk sec/Read, Avg. Disk sec/WriteThe Avg. Disk sec/Read and Avg. Disk sec/Write counters monitor the average numbers of seconds for read or write operations from or to a disk, respectively. Use the following guidelines when optimizing the disk subsystem. Do not have more average disk seconds per read or write operation than the values in Table 1.

Avg Disk sec/Read Avg Disk sec/Write

Transaction Log N/A <=10msec

Information Store <=10msec <=10msec

Table 1: Average Disk Access Times

If the values are much higher than those suggested in the above table, increase the speed of your disk subsystem by adding more drives or using faster drives.Object: Logical DiskCounter: Disk Transfers/secThe Disk Transfers/sec counter monitors the rates of read and write operations on the disk. It is important to realize that if there is a fibre channel Array controller and several drives allocated to a logical volume, this counter monitors the total number of disk transfers per logical volume. To calculate number of disk transfers per second per drive, divide the Disk Transfers/sec value by the number of drives in the logical volume. Object: Logical DiskCounter: Disk Queue LengthDisk Queue Length is the number of requests outstanding on the disk at the time the performance data is collected. It includes requests in service at the time of the snapshot. This is an instantaneous length, not an average over the time interval. Multi-spindled devices can have multiple requests active at one time, while other concurrent requests are awaiting service. For everyday use the average should be less than two for good performance.

Network Utilization, Monitoring and OptimizationThe network performance characteristics of a system can be determined by using several different tools including the Windows


2000 Server Performance Monitor, the Microsoft Network Monitor, and software based network sniffers. Note: Any discussion of using third party hardware- and software-based network sniffers is beyond the scope of this document.

Basic Counters for Exchange 2000 MonitoringThis section provides information on parameters that can be set to influence performance from the Exchange 2000 Server perspective, and available tools to monitor the behavior of the system.Send Queue Length

MsExchangeIS Mailbox: Send Queue SizeThis is the queue of items waiting to go from the Exchange store to IIS (SMTP) for “categorization” (determination of the appropriate server to which the mail must go).

Expected: Average should be less than 1% of the number of usersCategorizer Queue Length

SMTP SERVER: Categorizer Queue LengthThis is the queue of items waiting to be categorized in IIS.

Expected: Average queue size should not grow continuouslyLocal Queue Length

SMTP SERVER: Local Queue LengthThis is the queue of items waiting in IIS to be delivered in the local store database after categorization.

Expected: Average queue size should not grow continuously Database:

Log Record Stalls/secThis is the buffer that indicates whether the transaction log buffer is processing data efficiently.

Expected: Average < 1


Section 7Conclusion

Dell tests show that Dell PowerEdge servers and Dell/EMC fibre channel storage in a Storage Attached Network (SAN) configuration are very well suited for Exchange 2000 in an enterprise environment, for both server and storage consolidation. Exchange 2000 server takes full advantage of the high availability and scalability delivered by such a configuration. With the new database design, more users can be put onto a single server, which reduces the number of servers needed. With up to 8.4TB storage space, and configuration flexibility, the Dell/EMC fibre channel FC4700 makes growing the size of the Exchange database much easier than with direct-attached storage systems. The switched fabric also provides backend storage sharing for a heterogeneous environment. With the right design, configuration, and continuous monitoring, Dell server and storage products can be combined with Microsoft Exchange 2000 to create a robust messaging environment. It can be tailored to suit the needs of a variety of environments by using sizing techniques and monitoring performance.

ContactsFor questions about this paper or the implementation of Microsoft Exchange Server 2000 with Dell products, please contact a Dell sales representative. For comments and feedback about this paper, please email [email protected] Enablement Lab and ShowcaseEnterprise Systems Group/Storage System GroupDell Computer CorporationOne Dell WayRound Rock, Texas USA 78682+1-(800) WWW-DELL (999-3355) or +1-(512) [email protected]


mailto:[email protected]

mailto:[email protected]

Section 8APPENDIX A: Reference Documents

1. Microsoft Exchange Web Site http://www.microsoft.com/exchange

2. Dell computer Corp. Online http://www.dell.com3. Dell’s ROI Tool


4. Exchange Resource Kit, Microsoft Press5. EMC CLARiiON Best Practices for Fibre Channel Devices, EMC

White Paper6. Running an Optimized Microsoft Exchange 2000 Server with

Dell | EMC SAN, Richard Hou, Power Solution, May 20027. Published Benchmarks for Dell Servers running Exchange

2000 can be viewed at: http://www.microsoft.com/exchange/techinfo/planning/2000/PerfScal.asp

8. Intel Xeon MP Cache Size Impact on Exchange 2000 Server MMB2 Workloads, Scott Stanford, September 2002, Dell Power Solutions Magazine

THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF ANY KIND.

Dell and PowerEdge are trademarks of Dell Computer Corporation. EMC is a registered trademark of EMC Corporation. Microsoft and Windows are registered trademarks of Microsoft Corporation. Intel, Pentium and Xeon are registered trademarks of Intel Corporation. Other trademarks and trade names may be used in this document to refer to either the entities claiming the marks and names or their products. Dell disclaims proprietary interest in the marks and names of others.

©Copyright 2003 Dell Computer Corporation. All rights reserved. Reproduction in any manner whatsoever without the express written permission of Dell Computer Corporation is strictly forbidden. For more information, contact Dell.

Information in this document is subject to change without notice.






http://www.dell.com/

http://www.microsoft.com/exchange