mail migration to office 365 factors that impact mail migration performance part 2#4

of 16 | Mail migration to Office 365| Factors that impact mail Migration throughput

| Part 2/4

Written by Eyal Doron | o365info.com | Copyright © 2012-2015

Mail migration to Office 365 | Factors

that impact mail Migration performance

| Part 2/4

When we implement a project of mail migration to the cloud (Exchange Online), one

of the most important measurement values that we use is the “throughput value”

of the mail migration. In simple words, we want and need to know, how much time

it will take us to copy data (mailbox content) from the organization to the “cloud”

(Exchange Online).

This “value” is important to us because based on this value, we can provide an

estimation for How long it will take to finish the project, what are the costs that

involved in the project (a longer project will cost more than shorter time project)

and so on. However, before we can provide the required estimation, we need a

throttling policy\Throttling mechanism to know what are the “entity” or the factors

that are involved in the mail migration process and how these factors impact the

mail migration process.

http://o365info.com/mail-migration-office-365-factors-impact-mail-migration-throughput-part-24/


https://il.linkedin.com/in/eyaldoron1

http://o365info.com/


| Part 2/4


Note – In the next article (Office 365 mail migration | Optimizing the throughput |

Part 3/4) of this series, we will review these factors and the options that are

available for use for improving and optimize these factors.

Mail migration to Office 365 | Optimizing the Mail Migration

throughput | The article series

The article series includes the following articles:

Mail migration to Office 365 | Mail Migration methods | Part 1/4

Mail migration to Office 365 | Factors that impact mail Migration performance

| Part 2/4

Mail migration to Office 365 | Optimizing the Mail Migration throughput |

Part 3/4

Mail Migration to Office 365 | Measure and estimate Mail Migration

throughputs | Part 4/4

General mail migration questions

Before we begin, I would like to ask two questions (that will be answered in the

current article and in the next articles in this series).

Q1: Is there any kind of a formal answer about a “fixed number” that we can use

when we need to estimate the throughput of mail migration (the transfer rate of

data)?

A1: Yes, it is true that there is not much information on the subject but, there is a

Microsoft official article that provides “numbers” and estimation for a mail

migration throughput.

Q2: Why do we need to use an “estimation” or “average range of values” for the

mail migration throughput? Why can’t we use a simple a mathematical formula for

calculating the required time for transfer information from point A to point B?

A2: The answer is that – when we migrate a mailbox from point A to point B, there

are many components that are involved in the process. Each of these “components”

has a specific character that impact the result of the mail migration throughput.





http://o365info.com/mail-migration-office-365-optimizing-mail-migration-throughput-part-34

http://o365info.com/mail-migration-office-365-optimizing-mail-migration-throughput-part-34

http://o365info.com/mail-migration-office-365-mail-migration-methods-part-14/



http://o365info.com/mail-migration-office-365-optimizing-mail-migration-throughput-part-34/


http://o365info.com/mail-migration-office-365-measure-estimate-mail-migration-throughputs-part-44/

http://o365info.com/mail-migration-office-365-measure-estimate-mail-migration-throughputs-part-44/


| Part 2/4


When we want to migrate the content of a user’s mailbox to the cloud (Point B), we

need to get through many elements. Each of these “elements” has the ability to

“hold” the process or just let the data get through.

Each of the “elements” or the factor has resource’s limitations that affect the

performance. For example, in case that the “element” is the local Exchange On-

Premise server, the mail migration throughput is depended or based on the

Exchange On-Premise server resources such as: CPU, RAM, Hard disk and so on.

Additional factors are the type of the mail migration that we use, limitation of the

communications line, Throttling policy, the size of the mailbox, the number of mail

items and much more.

Factors that impact mail migration throughput

In the following section, we will review some of the factors that impact the

throughputs (transfer rate) of the mail migration. Additional and important

information about the performance factors that is not included in the current

articles appears in the article: Exchange Online Migration Performance and Best

Practices





https://technet.microsoft.com/library/dn592150(v=exchg.150).aspx



| Part 2/4


Network infrastructure

1. Communication line

One of the most important factors that relate directly to the result of the mail

migration transfer rate is obvious, the existing communication line.

1.1 The “speed” (The bandwidth) of the organizational communication line

Regarding the rate or the speed of the communication line, don’t forget that many

times the communication line technology is providing different transfer rate for

“upload’” verses “download” (asynchronies). When we migrate existing mail

infrastructure to the cloud, the operation is considered as “upload” (moving data

from the local network to the public network) and many times, the transfer rate for

“upload” is slower or less than the download rate.

1.2 The free or available percentage of the organization communication line

A very important factor is the “load” that exists on the communication line or the






| Part 2/4


“free percentage bandwidth” of the communication line. For example, it’s not

enough to say that the existing communication line bandwidth is 20 MB, but

instead, the real question is how many percentages of the bandwidth are free or

available for the task of the mail migration project.

1.3 Working hours and weekends

The additional element that affects the communication line “free or available

bandwidth” is – the factor of working hours and weekends. The basic assumption is

that during the “working hours,” the amount of the “free or available bandwidth” of

the communication line, is lower than the rate that we can get after the working

hours and especially at weekends.

2. Geographic location of the organization’s network

Office 365 data centers and mail infrastructure are physically located in different

geographical location worldwide. There are three “main points” in which Office 365

Data centers are located.

Here is a quotation from an article that I have found:

The way I’ve always answered this question is that Microsoft divides the world into

three regions: North & South America, Europe & Africa, and Asia & Australia. We

have an undisclosed number of datacenters in each of those regions, and we select

a primary datacenter within the region where the organization’s headquarters

address is located

[Source of information: Where is my Office 365 data located? ]

The additional important question that we can ask is: in case that our organization

has multiple sites that are located worldwide, can we assume that each location is

configured to access the nearest Office 365 data center?

The answer is: “No”. All the organizational resources such as user’s mailboxes will

be located in one data center and will not be “distributed” between the various

Office 365 data centers worldwide.

Microsoft has a regionalized data center strategy. The customer’s country or region,

which the customer’s administrator inputs during the initial setup of the services,

determines the primary storage location for that customer’s data.





http://blogs.technet.com/b/uspartner_ts2team/archive/2013/03/29/where-is-my-office-365-data-located.aspx


| Part 2/4


For specific details about where are the Office 365 data center geographically

located, you can read more information in the article: Where is my data?

The point of this information is that- there is an impact from the geographic

“distance” between the organization’s network and the Exchange Online data

center.

(I will provide more information about how to check the quality of the

communication and provide a recommendation in the last article of this

serious Office 365 mail migration | Optimizing the throughput | Part 3/4)

3. Network devices

Network devices such as a Firewall or Proxy servers can impact the results of the

throughput (data transfer rate) in a couple of ways:

The load factor- The load or the performance of these devices: for example, in

case that the existing Firewall or Proxy server has very high utilization, this could

lead to a reduction of the mail migration throughput.

Number of session restrictions- Another factor could be: the feature of Firewall

and especially Proxy server that limits or restricts the number of sessions in the

communication process.

For example: the process of mail migration could create tens or hundreds of

sessions, between the Exchange on-Premises server and Exchange Online. In case

that the Firewall or the Proxy server is configured with a maximum number of

session’s value, and the mail migration process reaches this limit, the throughput of

the mail migration is reduced.

Exchange On-Premise infrastructure

1. Data source (Exchange Server) – Performance of the local Exchange

On-Premise servers.

Mail migration is based on the concept in which the mailbox data is copied or

moved to the cloud (Exchange Online). In a scenario of multiple mailboxes, the

amount of the data that is transferred can easily become huge and the operation of

accessing to each of the mail server mailboxes and copy the data creates a

considerable load on the Exchange on-Premises server.





http://www.microsoft.com/online/legal/v2/?docid=25

http://o365info.com/mail-migration-office-365-optimizing-mail-migration-throughput-part-34/#SUB-6


| Part 2/4


In a scenario of Hybrid configuration, the server load could impact the Exchange

on-Premises server mailbox server who hosts the user’s mailboxes or to the

Exchange on-Premises server hybrid server, which serve as a router between the

internal Exchange on-Premises servers and the cloud.

2. Exchange sites

Mid and large organization, will usually have more than one Exchange site. The

subject of Exchange sites can impact the data transfer rate of the mail migration

from two perspectives:

The geographic location of the Exchange site versus the geographic location

of the Exchange Online data center. In case that the Exchange site is “closer”

to the Exchange Online data center, we would probably have better results.

Performing mail migration by using multiple Exchange sites

In case that the mail migration is performed from multiple Exchange sites, we






| Part 2/4


can migrate more mailboxes in a specific time window and the result of the

mail migration throughput is significantly improved.

3. Hybrid migration & MRSProxy

When using the option of Hybrid migration, the Exchange on-Premises server

component that is responsible for – implementing and managing the move mailbox

operations is the MRSProxy.

The task of moving mailbox from source server to the destination server described

as: MRSProxy connections.






| Part 2/4


In Exchange on-Premises server, the default value for the MRSProxy for the

maximum number of connections is 100.

In case that we want to increase this value, we can change the default value by

using a PowerShell command (we will review the exact syntax in the Office 365 mail

migration | Optimizing the throughput | Part 3/4)

4. Exchange Mailbox charters

Most of the time, when we mention the term “mailbox,” we use this term in general.

In the reality, the term “mailbox” is used for describing many different types of

mailboxes such as:

Small-sized mailbox with a small amount of mail items

Average mailbox with the average number of mail items

Small-sized mail box with a large number of mail items

Large-sized mailbox that usually characterizes as a mailbox with large-size +

large amount of mail items.

So what is my point?

My point is – that most of the time we relate only to the “mailbox size” and we

understand that logically, it will take more time to migrate Large-sized mailbox

versus a Small-sized mailbox.

An important factor that is not mentioned most of the time that has a tremendous

impact on the mail migration throughput is: the number of mail items that exists in

the user mailbox.

Common migration performance factors

One 4GB mailbox with 400 items, each with 10 megabytes (MB) of attachments, will

migrate faster than one 4-GB mailbox with 100,000 smaller items.

[Data source: Exchange Online Migration Performance and Best Practices]

Number of mail items and third party mail migration tools

In the article: Exchange Online Migration Performance and Best Practices, We can

find a data table that focuses on the mail migration throughput results in a scenario









of 16 | Mail migration to Office 365| Factors that impact mail Migration

throughput | Part 2/4


on a mailbox with an average amount of mail items versus a mailbox with a lot or

above average of mail items when using an RPC mail migration.

The little “catch” is that the data table relates to a result of mail migration that was

created when using a third party mail migration tools.

(I must admit that I’m wondering why the article includes information about third

party mail migration tools and doesn’t include a related information to the Office

365 built-in migration options)

Although the table compares the migration throughput in a scenario of third party

mail migration products think that we can use the information as a derivative for all

of Office 365 mail migration options such as: Cutover and Stage migration because,

these methods are also based on an RPC connection that is created between the

Exchange on-Premises server and the Exchange Online server.

So what is the data telling us?

If we take a closer look at the data that appear in the table, we can observe very

inserting issue: when we migrate an Exchange mailbox (small size mailbox in our

scenario) that have multiple numbers of mail items the process will create a huge

amount of RPC transactions.

To emphasize the conclusion form the data that appear in the table, I have created

the following diagram.








In this scenario, we can see information about the mail migration throughput for

two mailboxes.

One mailbox is quite small (size of 249 MB), and the other mailbox is larger (377

MB). The interesting fact is that – the time that we need to move the smaller

mailbox is significantly longer (11~ hours for the smaller mailbox verse 4.5~ hours

for the larger mailbox).

The answer for this “strange result” is that the smaller mailbox, includes 13,000~

mail items versus the larger mailbox that includes 4,100~ mail items.

The conclusion is that the factor of the number of mail items in a mailbox, has a

huge impact of the mail migration throughput.

5. Exchange on-Premises server throttling policy

Additional factor that we can relate to is: the Exchange on-Premises throttling

policy. In case that there is an implementation of: Exchange on-Premises throttling

policy, the throttling policy could impact the performance of the mail migration

process.

You can read additional information in the following articles:

Throttling Policy Associations in Exchange 2010 SP1

Exchange Online Throttling and Limits FAQ

Office 365\Exchange Online factors

1. Throttling policy\Throttling mechanism





http://blogs.technet.com/b/exchange/archive/2010/08/02/3410563.aspx

http://blogs.msdn.com/b/exchangedev/archive/2011/06/23/exchange-online-throttling-and-limits-faq.aspx




The term “Throttling policy” is a term that describes a similar concept as the term

QOS (Quality Of Service). A Throttling policy is basically a predefined restriction or a

limitation which is enforced by the administrator on a specific system.

For example: in the Exchange server environment, we can use Throttling policy for:

limiting the number of concurrent PowerShell remote sessions, the maximum

amount of data that can be accepted or sent and so on.

The question that relates to our scenario of mail migration to Office 365 is:

Does Office 365 use Throttling policy that restricts the amount of data that can be

“transferred” from the on-Premises mail server to Exchange Online?

And the answer is Yes and No.

In Office 365, there is an implementation of three types of throttling policy that

relates to mail migrations:

Office 365 migration-service throttling

Office 365 user throttling

Resource health-base throttling

Office 365 migration-service throttling

The migration-service throttling define the maximum number of concurrent

mailbox move to the Exchange Online server.

For example:

The default value for concurrent mailboxes “moves” when using Cutover, Stage

or Hybrid migration is 20.

The default value for concurrent mailboxes “moves” when using IMAP migration

is 4.

Q1: can I change this default value?

A1: Yes, very easily from the GUI interface or by using a PowerShell command.

Q2: can I assume that the obvious conclusion is: if I double the default value from

20 concurrent moves to 40 concurrent moves, I will double the mail migration

throughput?

A1: In theory the answer is: “yes.”

We use the term “In theory,” because when we increase the number of maximum

mailbox move on the Exchange Online server side (and in the Exchange on-

Premises server side), the results could be: overwhelming of the Exchange on-








Premises server because, the task of “move mailbox” is consuming resources from

the server who “deliver” the mailboxes (Exchange on-Premises server) and the

server who “accept” the mailboxes (Exchange Online).

My point is that we can assume that Exchange Online has an unlimited amount of

resources, but we cannot apply the same conclusion to the Exchange on-Premises

server because the basic assumption is that the Exchange on-Premises server

has limited hardware resources.

Office 365 user throttling

The term “user throttling” describe an Office 365 Throttling policy, which is

implemented or “enforced” in two scenarios:

Scenario 1: when using the option of “PST Migration”

Scenario 2: when using Third party MAPI Migration (client-uploading migration

method. That is implemented by using RPC over HTTPS).

Q1: What is the reason for using the user throttling policy?

A1: I could not find a formal answer for this question, but I can assume that the

reason for implementing the user throttling policy when using PST migration (client-

uploading migration) is that this type of migration, is not very effective and cause a

high load on the side that “export the data” (The client side) and the side the

“accept” the data (Exchange Online).

The Office 365 user throttling policy was designed for “protecting” the Exchange

Online from a scenario in which we implement tens or hundreds of PST migration

and by doing so, “overwhelming” the Exchange Online servers.

User throttling affects most third-party migration tools and the client-uploading

migration method. These migration methods use client access protocols, such as

RPC over HTTP, to migrate mailbox data to Exchange Online mailboxes. These tools

are usually used to migrate data from platforms such as IBM Lotus Domino and

Novell GroupWise.

User throttling is the most restrictive throttling method in Office 365. Because user

throttling is set up to work against an individual end user, any application-level

usage will easily exceed the throttling policy and result in slower data migration.






http://technet.microsoft.com/library/dn592150(v=exchg.150).aspx




Resource health-base throttling

The Resource health-base throttling was created to protect the health of the

Exchange Online server in special scenarios. The use of the Resource health-base

throttling will be implanted only on special occasions, when the monitoring system

identifies a special load or specific problem that relates to the Exchange Online

server infrastructure.

All migration methods are subject to the governance of availability throttling, but

Office 365 service throttling doesn’t affect Office 365 migrations as much as the

other types of throttling described in the previous sections.

Resource health-based throttling is the least aggressive throttling method and

occurs only when there is a service availability issue that affects end users and

critical service operations.


Summary

The following screenshot was taken from the Microsoft article: Exchange Online

Migration Performance and Best Practices

The table describes the “matrix” of the Office 365 Throttling policies and in which of

the migration scenario a specific Throttling policy is implemented.











In the following diagram, we can see a different representation of the information

that appears in the table: we can see that the Resource health-base throttling is

implemented for all the mail migration types without exception.

When “server to server migration” such as: Stage or Cutover migration, the Office

365 migration-service throttling is used to define a maximum number of concurrent

mailbox move.








Additional reading

Exchange Online Migration Performance and Best Practices

Exchange Online Throttling and Limits FAQ

EWS throttling in Exchange






http://blogs.msdn.com/b/exchangedev/archive/2011/06/23/exchange-online-throttling-and-limits-faq.aspx

http://msdn.microsoft.com/en-us/library/office/jj945066(v=exchg.150).aspx

mail migration to office 365 factors that impact mail migration performance part 2#4

Documents