Enterprise Network Services Overview

Online Services for the Enterprise

Published: January 2010

For the latest information, see www.microsoft.com/online.

The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of

publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of

Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication.

This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO

THE INFORMATION IN THIS DOCUMENT.

Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document

may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical,

photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this

document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any

license to these patents, trademarks, copyrights, or other intellectual property.

©2009 Microsoft Corporation. All rights reserved.

Microsoft, Bing, Hotmail, MSN, and Windows Live are trademarks of the Microsoft group of companies. All other trademarks are property of their

respective owners.

Microsoft Online Services | Network Service Description 2

ContentsIntroduction................................................................................................................................................ 4

Network Architecture................................................................................................................................ 5Inter-DC Network (LAN).......................................................................................................................... 6Microsoft Backbone Network................................................................................................................... 7Edge Network.......................................................................................................................................... 8Packet Flow............................................................................................................................................. 8

Connectivity Design Principles..............................................................................................................10

Network Security..................................................................................................................................... 11Internet Security.................................................................................................................................... 11Separation (Compartmentalization).......................................................................................................11

Appendix A: Read More About Microsoft Online Services Standard Offerings.........14

Microsoft Online Services | Network Service Description 3

IntroductionThis document describes the Microsoft Online Services networking infrastructure components and security features that support delivery of all Online Services for the enterprise that use the Internet for transport. These include all of the offerings that are part of the Microsoft Business Productivity Online Standard Suite (Microsoft Exchange Online, Office Live Meeting, Office Communications Online, and SharePoint Online), Microsoft Dynamics CRM Online, and many others. The document is intended for network engineers and system integrators who work with Microsoft Online Services customers. (Dedicated offerings are covered in a separate downloadable document, Microsoft Online Dedicated Service Descriptions and Service Level Agreements).

The components and features that are described include:

Enterprise network architecture for Microsoft Online Services

Microsoft Internet connectivity

Network security

Microsoft is constantly investing in new technologies, expansion, and innovation of its network infrastructure. This document is updated regularly to reflect changes that are deployed to the network to support Microsoft Online Services.

Microsoft Online Services | Network Service Description 4

Network ArchitectureThe network architecture for Microsoft Online Services was designed specifically to support enterprise-level services and applications. It was built around the pillars of performance, stability, security, redundancy, and scale. The network can be separated into three distinct functional sections: the intra-Data Center (intra-DC) local area network (LAN) environment, the global backbone, and the edge network for Internet connectivity. Some of the highlights of this architecture are noted here:

Based on a layered LAN architecture that allows for best-in-class technologies and equipment to be deployed in an independent (of layer) manner.

Provides the necessary functionality to enable virtualization for each Online Service within a data center.

Enables the sharing of multiple services on the same physical hardware, thus optimizing costs and utilization.

Uses the Microsoft global backbone to enable connectivity to thousands of Internet service providers (ISPs). This means that packets exit closer to the customer and in an optimized manner.

Utilizes multiple means of abstraction and checkpoints to help ensure that only desired traffic is allowed.

Uses redundant, very high-capacity links throughout, to help ensure stability and performance.

The overall network is illustrated in Figure 1.

Anchor Site

Anchor SiteInternet

Internet

Anchor Site

Data Center

Anchor Site

... .

Internet

Switch A Switch B

AccessRouter BAccess

Router A

Data Center Router B

Data Center Router A

CoreRouter A

Edge

Core

EdgeRouter

EdgeRouter

Data Center

AccessLayer 3

Layer 2 Aggregation

Top of Rack/ Servers

CoreRouter B

Data Center

Data Center

Firewall A

Load Balancer A Load Balancer B

Firewall B

SERVERS

SERVERS

TOR SwitchesTOR Switches

SERVERS

TOR Switches

Figure 1: Microsoft Online Services network architecture

The primary components of the Microsoft Online Services network architecture are discussed in detail in the following sections.

Microsoft Online Services | Network Service Description 5

Intra-DC Network (LAN)The intra-DC network provides connectivity to the servers that host the applications that make up a given Online Service. This is illustrated in Figure 2 below.

The first layer consists of servers that host the Online Service applications, which are located in racks that contain two top-of-rack (TOR) switches. Each server is connected via separate network interface controllers (NICs) to each switch. Every switch has two connections into the Layer 2 Aggregation.

Layer 2 Aggregation exists to consolidate many racks, and also to host shared services such as load balancing and firewalls. The advantage of hosting these devices in this layer is that they can be shared across racks and services with no change in physical topology.

The next layer, Layer 3 Aggregation, is the main routing layer for all virtual LANs (VLANs); it is where IP address blocks are configured. Each service terminates within a dedicated Virtual Routing and Forwarding (VRF) instance, and the routers are virtualized using VRF Lite. The routers also connect to the backbone network to provide connectivity to internal Microsoft administrative networks and to the Internet. Note that each Online Service resides on dedicated LANs and is physically separated from all other Microsoft services such as Windows Live Hotmail Web-based e-mail service. Security checkpoints to these LANs treat internal Microsoft traffic in the same manner as Internet traffic; in other words, internal traffic is not trusted any more than external traffic. This provides an additional level of security and abstraction.

Microsoft Online Services | Network Service Description 6

Figure 2: Intra-DC network architecture

As can be seen in Figure 2, redundancy and high availability are central themes. Two devices are used for routing and switching functions, and all connections are on a redundant basis. Firewall and load-balancer deployments use duplicate systems with automatic failover. Each service rack has two separate network connections and two individual power feeds to help ensure availability. Each data center network stamp has redundant, high-capacity (n x 10GE) links into the Microsoft backbone. These links provide protected connectivity to the Internet edge network and to other Microsoft locations.

Microsoft Backbone NetworkThe Microsoft backbone network, also known as the core network, provides high-bandwidth, low-latency connectivity to other Microsoft data centers and to the edge of the Microsoft network. Microsoft has a global network that takes advantage of modern technologies such as multi-protocol label switching (MPLS) and dark fiber with dense wavelength division multiplexing (DWDM) to deliver this level of connectivity. The backbone also has connectivity to many major "carrier hotels," data centers that are used to connect to ISPs, carriers, and other enterprises. These are called anchor sites in Figure 1. The backbone network is designed to handle the massive amount of traffic that is generated, getting it to its destination as fast as possible. The backbone network also carries inter-DC traffic, in addition to Internet and other external traffic.

Edge NetworkThe final component of the network is the edge network, which is used for Internet connectivity. Microsoft is one of the largest traffic destinations on the Internet due to the broad range of Microsoft hosted services such as the MSN network of Internet services, Microsoft.com, and Bing. Given the enterprise's desire for high performance and redundancy, Microsoft has an aggressive and open policy to solicit and connect with as many ISPs and enterprises as possible. This has been done on a global basis using direct, private connections and via membership in public exchange points such as LINX, PAIX, and Equinix. All of these efforts have gained Microsoft a position as one of the "top five best connected networks in the world," according to FixedOrbit.com. The advantage this brings to our customers is being close (in hops) to the Microsoft services that they are using. In addition, service quality is continuously being improved by provisioning multiple links to ISPs in different geographies and implementing optimal routing policies. Finally, link utilization is constantly monitored and capacity upgraded as needed. Figure 3 illustrates the connectivity strategy for a given data center.

Microsoft Online Services | Network Service Description 7

Figure 3: Edge network architecture

As mentioned before, anchor sites are carrier hotels that are used to connect to ISPs and exchanges. Because the main data center may not be in a favorable location for connectivity to a broad range of ISPs, a metro network is used to transport traffic between the anchor site and the DC. This is provisioned using multiple redundant high-capacity links.

Packet FlowFigure 4 presents a logical view of the Microsoft Online Services network architecture, and depicts how packets flow through it.

Backbone & Edge Routing

Customer

Enterprise Service

Internet

Management

Network Security Policy Enforcement

Point

Network Security Policy Enforcement

Point

Multi-Service Access

Router (MAR)

Load Balancing

Figure 4: Logical network architecture of Microsoft Online Services

A Microsoft Online Service is provided out of the Enterprise Service cloud, which is made up of the racks of servers, TORs, and aggregation switches. Flows coming into this cloud can be load-balanced if needed. Note that the load balancer can also provide additional functionality such as network address translation (NAT) and Secure Sockets Layer (SSL) offload. The Management cloud contains the servers and applications that are used by Microsoft to administer and manage servers in the Enterprise Service cloud. Security features help ensure that only trusted flows are allowed.

Microsoft Online Services | Network Service Description 8

For packets bound for the customer, the first ones arrive in the multi-service access router (MAR) inside of the VRF instance for the service. Customer-bound packets are sent to a firewall that provides for deep packet inspection of flows. If allowed, the packets are sent to the customer via the backbone and edge networks, which provide connectivity to the Internet. Routing optimizations are implemented to help ensure that the best path is used to reach the customer.

Microsoft Online Services | Network Service Description 9

Connectivity Design PrinciplesMicrosoft Online Services customers should keep in mind the design factors of reliability, capacity, and latency when planning network connectivity to Microsoft data centers. Note that all services (including those that are not specifically Online Services) are accessed over the Internet with no specific transport requirements such as dedicated circuits or virtual private networks (VPNs).

Reliability: Microsoft has very robust and broad connectivity to most Tier 1 and Tier 2 ISPs globally. This means that multiple paths are available to reach a given destination network. In addition, redundancy is implemented at all levels of the network, including equipment and links. We strongly recommend that the customer connect to at least two separate ISPs for access to Microsoft Online Services. Multiple ISP connections provide the redundancy required to help ensure that users have uninterrupted access to critical services at all times.

Capacity: Regardless of transport method, it is critical that the customer perform initial planning and ongoing capacity analysis to help ensure that adequate bandwidth is available for reaching Microsoft Online Services at all times. These processes require accurate prediction of bandwidth demand and ensuring that proper measuring tools are in place to monitor usage. Access to Microsoft Online Services may be impacted if the same link is used for access to Microsoft as well as for general Internet traffic. For example, flash traffic may overwhelm traffic that is destined for Microsoft Online Services, which can cause degraded network service or lack of access. We recommend that the customer provision separate links for Internet access and Online Services access.

Latency: Latency is a critical network factor that directly affects perceived and actual performance of a given Microsoft Online Service. Each Online Service provides general guidance for acceptable round-trip time (RTT) between the customer and Microsoft Online Services. When provisioning, tests must be conducted ahead of time to help ensure that RTT is within acceptable tolerances.

Microsoft Online Services | Network Service Description 10

Network SecurityBecause Microsoft Online Services manages multiple customer environments from a single management space, network infrastructure controls are specifically designed to help ensure the confidentiality and integrity of customer data through strict compartmentalization. Under no circumstances is access permitted between one environment and another. Any traffic that is not part of the Online Service is treated in a non-trusted manner, including internal Microsoft traffic. The Online Services network also enables reliable data availability through equipment redundancy, resiliency, and industry-standard high-availability design practices.

Internet SecurityMicrosoft Internet connections are used to transport traffic for various Microsoft Online Services. Microsoft applies a rich set of security controls and optimizes routing to help ensure the desired level of performance. In particular, three levels of security are implemented to prevent unwanted traffic from entering the Microsoft network or the VLAN of the service.

1. As traffic heads toward the VLAN, two sets of network filters allow only authorized networks on given ports and protocols to reach the servers for a given Online Service.

2. At the router, security by abstraction obscures the routes and allows only authorized traffic to pass through. Because virtualization is used on the router level, only the needed routes are present in the routing table of the Online Service. Thus, routing is not available to any other destination and must pass through the firewall for validation.

3. All unrecognized traffic is routed to the firewall, where specific rules govern the type of traffic that is allowed to pass through on a stateful basis. Any traffic that does not meet the firewall rule list is simply dropped.

In addition to this three-tiered security, there’s a final checkpoint in data centers: only servers that are managed by Microsoft and configured for Internet access can receive Internet traffic.

Separation (Compartmentalization)One key strategy that Microsoft Online Services uses to maintain the confidentiality and integrity of customer data is compartmentalization. Multiple techniques are used to control information flows between the various clouds shown in Figure 5:

Network separation: Network segments are physically separated by virtualized routers that are configured to prevent communications between Online Services unless otherwise desired. Routing for all Online Services is, in effect, on a "need-to-know" basis. Further, all networks outside of a given Online Service, including other Microsoft Online Services, are treated as an external environment, just as the Internet is.

Logical separation: VLAN technology is used to further separate communications between Customer Network and Managed Network segments.

Firewalls: Firewalls and other network security enforcement points are used to limit data exchanges with servers that are exposed to the Internet, and to isolate systems from the back-end systems that are managed by Microsoft.

Protocol restrictions: Only known and required services and applications can be used to access servers on an Online Service network from the Management network. The access is restricted by strict policy filters.

Figure 5 illustrates these connections and associated restrictions.

Microsoft Online Services | Network Service Description 11

Figure 5: Online Services network communication flows

Figure 6 illustrates the separation of Microsoft Online Services from other networks and enforcement points.

Microsoft Online Services | Network Service Description 12

Admin/Management

- Imaging- Backup- Patching

- Server management

All Other Services

- Includes services such as Bing and Hotmail

Microsoft Online

Services

- Purpose-built, enterprise network architecture

- All services exist inside separate virtualized instances

- Redundant, high-capacity uplinks- All external traffic (including

Microsoft) is treated as non-trusted

Microsoft Backbone

Internet

- Global reach- Very high performance

- Redundant, scalable, high-capacity links

Internal Traffic

Network Security Enforcement Point

Network Security

Enforcement Point

Network Security Enforcement Point

Network Security Enforcement Point

Figure 6: Separation of the Microsoft Online Services network

Microsoft Online Services | Network Service Description 13

Appendix A: Read More About Microsoft Online Services Standard OfferingsThe following links provide additional detail about Microsoft Online Services Standard offerings.

Main Web site detailing the Standard offerings [Link]Contains useful information about the Microsoft Business Productivity Online Standard Suite.

Get Started with Business Productivity Online Standard Suite [Link]Contains useful information about the Microsoft Business Productivity Online Standard Suite.

Microsoft Office Live Meeting Service DescriptionOffice Live Meeting is an enterprise-class Web conferencing service. With Office Live Meeting, organizations can engage customers through real-time meetings, training sessions, and events presented over the Internet. Office Live Meeting operates in an infrastructure separate from Microsoft Online Services. However, Microsoft Online Services provides consulting services to help organizations efficiently adopt and begin using the Office Live Meeting service.

Security in the Business Productivity Online Standard Suite from Microsoft Online Services [Link]This white paper describes the security and reliability features of the Business Productivity Online Standard Suite from Microsoft Online Services. It details the capabilities, technologies, and processes that are used, and examines how the experience of Microsoft in building and operating enterprise software has led to the demonstrated reliability of its Microsoft Online Services offerings.

Guidance for Microsoft Online Services Multinational Customers [Link]This white paper is targeted at IT professionals in a multinational company who are interested in evaluating Microsoft Online Services. It provides the tools and guidance for using Microsoft Online Services from multiple locations worldwide, both during an evaluation stage and after purchase, when Microsoft Online Services are introduced for the first time to the multinational company’s branch offices in different regions.

Microsoft Online Services | Network Service Description 14