1© copyright 2012 emc corporation. all rights reserved. emc performance optimization for microsoft...

1© Copyright 2012 EMC Corporation. All rights reserved.

EMC PERFORMANCE OPTIMIZATION FOR MICROSOFT FAST SEARCH SERVER 2010 FOR SHAREPOINTEMC Symmetrix VMAX, FAST VP, Microsoft Hyper-V


Agenda

Solution overview: Objectives

Why this solution

Solution architecture (physical and virtualized)

Results and findings (physical and virtualized)

Backup and restore results

Summary


Solution overview: Objectives Implement Microsoft FAST Search Server 2010 for SharePoint in

a virtualized environment on a Symmetrix VMAX storage system

– Validate and show a FAST Search environment on Physical servers for 4 TB SharePoint content and 5 TB file share document

– Showcase how to configure and deploy FAST Search servers

– Provide guidelines for virtualizing the FAST Search servers

– Validate the benefit of FAST VP for the SharePoint farm environment as well as the FAST Search server environment

– Validate the SharePoint and FAST Search full farm backup, content database restore and FAST Search full farm backup and restore


Why this solution Microsoft FAST Search Server 2010 for SharePoint delivers an

exceptional intranet and people search experience and a platform for building custom search-driven applications.

The deployment of the FAST Search Server 2010 for SharePoint needs careful planning and understanding of both SharePoint and FAST Search servers. This solution can provide some guidance to the implementation team.

Virtualization is often called for from many customers, but there is no guideline available. Questions such as “Is it possible?” “How could it be done?” “What kind of performance can I get?” remained unanswered.

This solution addresses the questions above with EMC VMAX as the storage platform and Microsoft Hyper-V as the virtualization platform.


Solution architecture


FAST Search for SharePoint topology overview (I)

FAST Search Server 2010 for SharePoint provides enterprise search infrastructure for the SharePoint server farm.

The search solution is hosted on two farm topologies:

– SharePoint Server farm hosts the query interfaces and content collection (crawling) interfaces for the search solution.

– FAST Search Server 2010 for SharePoint farm provides back-end indexing and query matching capabilities and administration of these features.


FAST Search for SharePoint topology overview (II)

FAST Search for SharePoint farm topology and work flow


Design considerations Use minimum servers in a FAST Search farm

Create a redundant Web analyzer to ensure service availability whenever it is possible

Keep storage out of the administrator server for easy recovery

Install FAST on the non-operating system LUN to divert the logging activities and make LUN based replication of the FAST Search farm possible


Solution architecture:

Physical FAST Search farm design


Solution architecture with physical FAST Search servers (I)


Solution architecture with physical FAST Search servers (II)

Physical FAST Search server farm configuration


Solution architecture:

Virtualized FAST Search farm design


Solution architecture with virtualized FAST Search servers (I)


Solution architecture with virtualized FAST Search servers (II)

Virtualized FAST Search server farm configuration


Results and findings


Testing profiles User load profile

– Heavy (60 requests/hour)– Required response time

Content profile

Test type Details Required response timeBrowse User browse Less than 3 secondsSearch Unique value search Less than 3 secondsModify Browse and metadata modify Less than 3 seconds

Item ValueContent type doc, docx, pdf, xls, xlsx, ppt, pptx, txtAverage content size 1.65 MBSharePoint - Total data 4 TB SharePoint – Document size range 20 KB – 50 MBSharePoint – Number of documents 2,000,000 SharePoint – Total site count 220 SharePoint content database size (each) Up to 200 GB

SharePoint – Total site collections 20 SharePoint – Sites for each site collection 11 SharePoint – Total user count 22,000 SharePoint -- User concurrency 10% File share – Total data 5 TB File share – Number of documents 3,000,000


Key results (I)

Environment Available CPUsDocument processors

Full crawl rate ( items/minute)

SharePoint content

File share

Physical 60 42 1513 1667

Virtualized 20 24 770 978

Server typeAverage read size (KB)

Average write size (KB)

Average read/write

Total IOPS

SQL server 26 24 218:1 1434

FAST index 275 605 3:1485

FAST servers 13 26 1:66

Environment

Profile ( browse/ search/ modify)

Average response time (second)

Queries per second

FAST IOPS total

Browse Search Modify

Physical 80/10/10 0.94 0.60 2.43 7.1

550

Virtualized 80/10/10 0.95 0.47 2.83 8.5


Key results (II) The maximum total disk IOPS is around 3,500 on the back end for full

crawls and around 1,900 for the query testing.

Crawl rate is 150 GB/hour in the physical environment, and 80 GB/hour in the virtualized environment with less hardware resources.

Search response time is less than 1 second in both physical and virtualized environments supporting 22,000 users with 10% concurrent access.


Key results (III)

With Replication Manager-engaged TimerFinder snapshot backup: – The whole SharePoint farm, with a total of 9 TB data, can be

backed up within 15 minutes. – The FAST Search farm, with a total of 5 TB FAST Search

volumes, can be backed up within 8 minutes.

With Replication Manager-engaged TimerFinder snapshot restore:– The SharePoint content database (4 TB) and file share data

(5 TB) can be restored and recovered within 1 hour and 30 minutes.

– The FAST Search farm, with a total of 5 TB FAST Search volumes, can be restored within 13 minutes.

Replication Manager-engaged TimeFinder/Snap backup and restore


Key results (IV)FAST VP With FAST VP enabled, the environment can better handle peak

I/O request.

The search performance improved about 20% after enabling FAST VP on Symmetrix VMAX storage.

QPS result of profile 80/10/10 (percentage of browse/search/modify)

FAST VP used? (Yes /No) Queries per second (QPS)

No 7.1

Yes 8.5

Before FAST VP was enabled After FAST VP was enabled

100% FAST volumes on FC virtual pool

1.3% on Flash virtual pool

94.6% on FC virtual pool

4.3% on SATA virtual pool


Test results:

Physical FAST Search farm


Test result: Full crawl for SharePoint content in the physical FAST Search farm (I)

Full crawl performance of the SharePoint content

IOPS and CPU usage of the SharePoint full crawl

Duration (hh:mm:ss)

Items crawled Crawl rate (items /min)

Crawl rate (MB/min)

Crawl type

24:15:00 2,190,954 1,513 2,111 Full

Item FAST Search servers SharePoint servers

Average total IOPS 485 1,434

Maximum total IOPS 1,809 4,749

Average CPU usage 41% 26%

Available CPUs in the farm 60 N/A

Document processors 42 N/A

FAST Search index volume I/O characteristics and read/write ratio

Server role Average IOPSIOPS (maximum)

Read I/O size

Write I/O size

Read:Write

Primary 345 1,809 275 KB 605 KB 3:1

Secondary 47 334 45 KB 664 KB 1:50


Test result: Full crawl for SharePoint content in the physical FAST Search farm (II)

SharePoint content database full crawl rate

• The crawl rate of smaller documents was higher with the peak reaching 8,000 items per minute.

• The crawl rate of larger documents was lower, which was 1,516 items per minute or 2111 MB per minute on average.


Test result: Full crawl for SharePoint content in the physical FAST Search farm (III)

Back-end storage Symmetrix VMAX performance

• Disks and thin pools performed well• The disk utilization of SharePoint and FAST Search

volumes was less than 50 percent • The total IOPS on the back end was 2,500 to 4,500

Item Average Maximum

IOPS 3,340 4,730

Throughput (MB)

210 275

Disk group utilization

Percentage

Content DB 5%–35%

FAST index volumes 10%–47%


Test result: Full crawl for file share in the physical FAST Search farm (I)

Full crawl performance of the file share content

IOPS and CPU usage of the file share full crawl

Duration (hh:mm:ss)

Items crawled Crawl rate (items/min)

Crawl rate (MB/min)

Crawl type

35:15:00 3,124,223 1,668 2,602 Full


Average total IOPS 208 339

Maximum total IOPS 1,706 930




FAST Search index volume IO characteristics and read/write ratio

Server roleAverage IOPS

IOPS (maximum)

Read I/O size

Write I/O size

Read:Write

Primary 284 1,706 313 KB 617 KB 5:2

Secondary 45 186 18 KB 756 KB 1:25


• The crawl rate was higher than the content database

• The crawl rate was largely related to the size of the files peak while processing the smaller files

Test result: Full crawl for file share in the physical FAST Search farm (II)

File share full crawl rate in the physical environment


• Disks and thin pools performed well

• Disk pools utilization was low

• Total IOPS on the back end was around 1400-2800


Test result: Full crawl for file share in the physical FAST Search farm (III)


IOPS 2,200 2,880

Throughput (MB)

234 322


Percentage

Content DB 5%–25%

FAST index volumes 20%–55%


Test result: Query performance on the physical FAST Search farm (I)

VSTS (browse/search/modify)testing

Profile (percentage of browse/search/modify)

Requests per second


Queries per second


80/10/10 35.7 0.94 0.60 2.43 7.1

70/20/10 36.5 0.93 0.63 2.14 9.5

50/30/20 38.0 0.90 0.83 2.26 16.7


Test result: Query performance on the physical FAST Search farm (II)

Back-end storage performance with VSTS testing

Symmetrix VMAX back end IOPS during 80/10/10 (percentages for browse/search/modify) profile

Maximum IOPS

Total SharePoint and FAST Search servers 1,900

FAST Search servers only 550


Test results:

Virtualized FAST Search farm


Test result: Full crawl for SharePoint content in the virtualized FAST Search farm (I)

Full crawl performance of the SharePoint content

IOPS and CPU usage of the SharePoint content full crawl

Duration (hh:mm:ss)


Crawl rate (MB/min)

Crawl type

47:47:00 2,190,954 770 1,395 Full


Average total IOPS 675 900

Maximum total IOPS 6,592 4,257






IOPS (maximum)

Read I/O size

Write I/O size

Read:Write

Primary 483 6,592 184 KB 345 KB 3:1

Secondary 103 13,346 42 KB 582 KB 1:5


Test result: Full crawl for SharePoint content in the virtualized FAST Search farm (II)

Full crawl rate of the SharePoint content

• The crawl rate of smaller documents was higher with the peak reaching 12,500 items per minute.

• The crawl rate of larger documents was lower, which was 770 items per minutes or 1395 MB per minutes on average.


Test result: Full crawl for SharePoint content in the virtualized FAST Search farm(III)

• Disks and thin pools performed well

• Disk pools utilization low

• Total IOPS on the back end was around 2,800-4,300



IOPS 3,500 4,300

Throughput (MB)

217 300


Percentage

Content database 20%–30%

FAST index volumes

30%–50%


Test result: Full crawl for file share in the virtualized FAST Search farm (I)

Full crawl performance of the file share content

IOPS and CPU usage of the file share full crawl

Duration (hh:mm:ss)


Crawl rate (MB/min)

Crawl type

60:14:32 3,124,223 978 1383 Full


Average Total IOPS 384 193

Maximum Total IOPS 7,869 1,263


Available CPUs in Farm 20 N/A

Document Processors 24 N/A



IOPS (maximum)

Read I/O size

Write I/O size Read:Write

Primary 463 7,869 186 KB 373 KB 3:1

Secondary 29 8,704 36 KB 560 KB 1:2


• The crawl rate was faster than the content database.

• The crawl rate was largely related to the size of the files peak while processing the smaller files; same pattern with the physical environment.

Full crawl rate of the file share content

Test result: Full crawl for file share in the virtualized FAST Search farm (II)

Tiffany Li

here


Test result: Full crawl for file share in the virtualized FAST Search farm(III)

• The total IOPS on the back end was between 2,500 and 4,300.

• The disks and thin pools performed well. Disk utilization of SharePoint and FAST Search volumes was less than 50 percent.



IOPS 3,500 4,300

Throughput (MB)

217 300


Percentage

Content database 20%–30%

FAST index volumes

30%–50%


Test result: Query performance on the virtualized FAST Search farm (I)

VSTS testing

Profile (percentage of browse/search/modify)

Requests per second


Queries per second


80/10/10 38.2 0.95 0.47 2.83 8.5

70/20/10 41.6 0.90 0.55 2.50 14.4

50/30/20 38.5 0.88 0.67 2.08 12.9


Test result: Query performance on the virtualized FAST Search farm (II)

Back-end storage performance with VSTS testing

Symmetrix VMAX back end IOPS during 80/10/10 (browse/search/modify) profile

Maximum IOPS

Total SharePoint and FAST Search servers 3,400

FAST Search servers only 850


FAST VP test results• FAST VP policy:

– Flash: 20 percent

– Fibre Channel: 30 percent

– SATA: 50 percent

• Before FAST VP was enabled: 100 percent FAST Search volumes were on FC virtual pool.

• After FAST VP was enabled: About 1.3 percent of data was promoted to the Flash tier; about 4.3 percent of data was demoted to the SATA tier; and 94.6 percent of data resided on the FC tier.

In the test above, the search performance improved about 20 percent with FAST VP enabled.


Test result: Replication Manager-engaged FAST farm backup and restore (I)RM-engaged backup/restore summary• Backup:

– A snapshot for the whole SharePoint farm took only 11 minutes to complete.

– A snapshot for file share completed in less than 2 minutes.

– A snapshot for the FAST Search farm took about 8 minutes across three servers.

• Restore:

– Restoring a single SharePoint database took only a few minutes to process. Restoring all 20 SharePoint database across two SQL Server database servers took about 1.5 hours, most of which was spent recovering the databases (log replay).

– Restoring file share took less than 2 minutes to finish.

– Restoring the FAST Search farm took less than 13 minutes to finish.


Test result: Replication Manager-engaged FAST farm backup and restore (II)Replication Manager-engaged TimeFinder/Snap backup

Replication Manager-engaged TimeFinder/Snap restore

*Including database recovery time.

Item Total sizeBackup duration (hh:mm:ss)

SharePoint farm backup 4 TB 0:11:33

File share backup 5 TB 0:01:53

FAST Search farm backup

5 TB0:07:50

Item Total sizeRestore duration (hh:mm:ss)

SharePoint content database restore*

4 TB1:24:33

File share restore 5 TB 0:01:53

FAST Search farm restore

5 TB0:12:49


Configuration tipsThe following settings are recommended by Microsoft for better crawl performance.

• SharePoint crawler registry changes:

– HKLM\SOFTWARE\Microsoft\Office Server\14.0\Search\Global\Gathering Manager• FilterProcessMemoryQuota

– Default 100 MB– Changed to 200 MB

• DedicatedFilterProcessMemoryQuota– Default 100 MB– Changed to 200 MB

• FolderHighPriority– Default 50 – Changed to 500

• Turn off TCP/IP Chimney: http://support.microsoft.com/kb/2570111

• Apply hotfix KB976462

• Document processors could be adjusted in the run time

http://support.microsoft.com/kb/2570111


Summary (I)• SharePoint farm design

– The SharePoint farm used three Hyper-V servers supporting 12 virtual machines, which formed the virtualized SharePoint farm.

– The SharePoint farm contained 9 TB content source (4 TB content database and 5TB file share).

– The SharePoint farm supported 22,000 users to operate with mixed profile (browse, search, and modify) with 10 percent concurrent access. The average search response time was less than 1 second.

• FAST Search farm design

– The physical FAST Search farm used five physical servers.

– The virtualized FAST Search farm used two physical hosts with Hyper-V enabled.

• FAST Search performance in a virtual environment

– A full crawl against a 5 TB file share can be completed in 60 hours and 15 minutes in the virtualized FAST Search servers (35 hours and 15 minutes in the physical FAST Search servers).

– A full crawl against 4 TB of SharePoint content can be completed in 47 hours and 15 minutes in the virtualized FAST Search servers (24 hours and 15 minutes in the physical FAST Search servers).


Summary (II)• Consolidation ratio in a virtualized environment

– The migration of FAST Search servers to a virtualized environment reduced the number of physical host servers from five to two (60 percent reduction). This also leveraged all the management advantages of a virtualized environment while achieving comparable query performance of a physical environment and acceptable crawl performance.

• I/O characteristics of FAST Search components

– The read and write I/O size were around 300 KB and 600 KB on the primary index volume of the FAST Search server.

– The read and write I/O size were around 40 KB and 700 KB on the secondary index volume of the FAST Search server.

• Enabling FAST VP improved the query performance of FAST Search by 20 percent


Summary (III)• Crawl rate

– A full crawl against a 5 TB file share completed in 35 hours and 15 minutes in the physical FAST Search servers, or 60 hours and 15 minutes in the virtualized FAST Search servers.

– A full crawl against 4 TB of SharePoint content completed in 24 hours and 15 minutes in the physical FAST Search servers or in 47 hours and 15 minutes in the virtualized FAST Search servers.

• Search Rate

– Mixed workload with 30 percent query achieved 16.7 queries per second (QPS) in the physical FAST Search farm.

– 14.4 QPS were processed in the virtualized FAST Search farm while supporting 22,000 users with 10 percent concurrent access.

– Pure search test achieved 205 QPS while keeping search the mean query latency response time less than 0.072 seconds.

• Backup and Restore

– Replication Manager ensures that the environment is protected. The SharePoint content and FAST Search farm were replicated and recovered with minimal impact on the users.


Demonstration


Q&A

1© copyright 2012 emc corporation. all rights reserved. emc performance optimization for microsoft...

Documents

fast search environment

fast farm

fast search serversvalidate

benefit of fast vp

virtualized sharepoint

sharepoint farm environment

emc vmax

sharepoint new