![Page 1: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/1.jpg)
SQL Server and SharePoint - Best Practices
Steffen Krause Snr. Technical Solution Prof. – Data Platform
Microsoft Germany http://blogs.technet.com/steffenk
![Page 2: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/2.jpg)
Agenda • Introduction
• SharePoint Architecture
• SharePoint Design
• SQL Server Best Practices
![Page 3: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/3.jpg)
Introduction • What is SharePoint?
– 2007 • WSS (Windows SharePoint Services) • MOSS (Microsoft Office SharePoint Services)
– Builds on WSS.
– 2010 • SharePoint Foundation • SharePoint Server
• Why is SQL Server so important – Central store for most SharePoint data – Stress on SQL Server causes stress on front-end servers
![Page 4: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/4.jpg)
Heterogeneous Workloads…
Employee Portal
Project “X” Site
Business Intelligence Dashboard
Extranet Collaboration
Site
Custom SAP
Front-End
Geneva Office
Site
Knowledge Management
Portal
R&D Community
Regulatory Compliance Repository
Corporate Web
Presence
Team “ABC” Site
Sales Division Portal
My “Facebook”
![Page 5: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/5.jpg)
Architecture 9000 Foot View
Web Web Web
Load Balancing
…
App Server (XLS)
App Server (People)
App Server (Search)
SQL
…
Content DB … Content DB Content DB
Content DB
![Page 6: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/6.jpg)
SharePoint Storage – One Pager • Flexible, user extensible types
– Announcement, Contacts, Document Types – Support 10th of a million types in a single DB – A few types that may have 100’s of properties
• Millions of instances of multiple types in a list • Efficient display of “all items in a folder”
– “Why do we need that long clustered index”?
• End-user queries over multiple lists in multiple sites which is mapped onto a single table – Show me all recent articles in this Site Collection where priority = 1 and HBI =
true
![Page 7: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/7.jpg)
Wide, and Widely Sparse Tables • Classic problem for modeling extensible type system on SQL
– Tables Per Type (SP V1) and UDTs (WinFS) don’t work – e.g Survey and Form on SQL table
MyProductSurvey MyTAPSubmissionForm
Product Single Line of Text Title Single Line of Text
Years of Experience Number Description Multiple lines of Text
Employee Id Number Pre-Release Prod? Bool
C-Sat Index Float Technical Evidence? Bool
Location Single Line of Text IT Budget Number
Employee Size Number Size of IT Org Number
![Page 8: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/8.jpg)
Userdata table (e.g. AllUserData)
Namespace Table (e.g. AllDocs)
Container Tables Id Quota Other Metadata
Sites
SiteId Id Url Title ScopeId Metadata Webs
WebId Id Title ItemCount ScopeId Fields Metadata
AllLists
Other Metadata
1…64 1...32 1..8 1..16 1..12 1..8 1…16 ~35
Url
Architecture: Content DB Schema
![Page 9: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/9.jpg)
SharePoint Wraps a Record Into Multiple Rows
• Morgan Stanley InfoPath integration case-study – # of properties 430 – # of ints > 90 – # of floats > 70
• Design Choices – Widen the wide table arbitrarily OR – Wrap the rows so we pay penalty only for wide types
• Resolution – Wrap rows
![Page 10: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/10.jpg)
SharePoint Maintains Its Own Index
• Multiple types in the same table makes SQL indexing untenable
TypeId Ntext Int Int Float Ntext Int nvarchar Bool Bool
ProdSrv MOSS 5 181870 78.9 Tokyo 90
TapForm HR Project 500,000 A LoB integration project for our HR Site
Yes No
• Design Challenges
• How do I put a SQL index for a given property in all instances of a given type?
• Do you suggest 1000’s of index on a table?
• Resolution - Maintain Name-Value pairs and SQL index NVP table
![Page 11: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/11.jpg)
DB Changes Not Supported • Workloads w/ conflicting Platform
needs • Single data platform for all
workloads • Change for one may adversely affect
another
• Upgrade and Servicing expects solid DB contract
• App logic is heavily dependent on DB specifics
• App enforces constraints and integrity!
Corporate Web
Presence
Team Collab Site
• Mostly Read • Article Pages • Search and structured
queries
• Read Writes • Office Documents • Organization and ad-hoc
queries
Note: DB Changes are possible, you need SharePoint Support in the loop
![Page 12: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/12.jpg)
SharePoint Databases The Basics
• No access to DBs except thru SharePoint! • Exception: Logging DB in 2010 • Separate servers for SharePoint and SQL • SQL Server: 64 bit with lots of RAM • Gigabit Ethernet between SQL Server and frontend servers • High availability:
– Cluster or mirroring for local HA • Mirroring directly supported in 2010 admin UI
– Mirroring for redundant data centers
![Page 13: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/13.jpg)
Data Scale Improvements – 2010 style • Enabling new deployment scenarios in 2010
Examples: – 100 Million Items per Search Index (1 Billion with Fast 2010) – Tens of Millions of Documents/Items in a single List – View/Query 5000 items at a time
• Many recommendations/limits stay the same – Site Collections per WebApp (150,000) – Site Collections per Content DB (50,000) – Content DB Size (100 GB)
The Product has its limits!
![Page 14: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/14.jpg)
If you want more than 100 GB per Content DB...
• Test I/O subsystem for decent performance
• Avoid having more than one site collection in a Content DB that grows over 100 GB
• Test backup solution throughout
• Determine backup & restore times
• If possible use efficient backup solution like DPM
![Page 15: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/15.jpg)
Storage Optimizing SQL Server I/O Subsystem
• Ensure correct HBA driver and firmware versions
• Use SQLIO.exe to measure I/O performance
• Configure correct NTFS Allocation Unit Size – 64K best; default (4K) can result in a 30% perf hit – To view: chkdsk <drive_letter> – To set: format E: /Q /FS:NTFS /A:64K /V:Data1 /Y
• Ensure correct Windows “Sector Alignment” – Incorrect setting can result in up to 50% perf hit – Use WMI script (ReportOffset.vbs) provided in notes section to check existing value – 64K most common. Windows 2008 aligns sectors by default
![Page 16: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/16.jpg)
Placement and priority of databases • If possible: Put tempdb to RAID 10 array • Data and log files to physically separate disks! • Dedicated disks for search • IO requirements (sorted descending)
– Tempdb data & logs – Transaction logs – Search DB – Content DBs – For publishing sites that are mostly read only: possibly data before logs
• SharePoint 2010: Logging DB can have high load! • Large environments: Multiple files for tempdb, Content and Search DBs
– Same-sized files on multiple disks – # files <= # processor cores – Multiple files are not supported for other SharePoint DBs
![Page 17: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/17.jpg)
SQL Best Practices: MAXDOP
• Set MAXDOP to 1
• How? – In Server Properties - Advanced
• Why? – Symptom: SQL timeouts in ULS logs
– Reason: CXPACKET waits cause deadlocks
– SharePoint barely benefits from parallelized queries
![Page 18: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/18.jpg)
SQL Best Practices: Memory • Set Max Server Memory to appropriate value • How?
– Recommended setting: Physical RAM – Memory for OS/apps – ((MaxWorkerThreads * StackSize) + DefaultReservationSize) • MaxWorkerThreads = 256 + (NumOfSchedulers-4) *8 • StackSize = 1MB on x86, 2 MB on x64, 4 MB on IA64
– In Server Properties – Memory
• Why? – In some circumstances, SQL Server does not leave enough memory for
Windows
![Page 19: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/19.jpg)
SQL Best Practices: Statistics • Make sure that SQL statistics are up to date • How?
– Make sure the Health Analyzer Rule (under Performance) is enabled
– If you still have very long query execution time problems, run UPDATE STATISTICS with FULLSCAN
– Make sure AUTO_CREATE_STATISTICS and AUTO_UPDATE_STATISTICS are off
• Why? – Up to date SQL statistics are required for the query optimizer
![Page 20: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/20.jpg)
SQL Best Practices: Index Fragmentation
• Make sure indexes are defragmented • How?
– Make sure the Index fragmentation Health Jobs are enabled • Databases used by SharePoint have fragmented indices (SharePoint
Foundation 2010) • One or more Search crawl databases have fragmented indices
(SharePoint Server 2010) • One or more Search property databases have fragmented indices
(SharePoint Server 2010)
• Why? – Index fragmentation costs lots of CPU and memory and slows IO
![Page 21: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/21.jpg)
SQL Best Practices: Database File Size
• Size SharePoint databases correctly • How?
– If possible, create database files in target size – Rely on “Auto Growth” option only for catastrophic
situations – Set next growth to fixed reasonable sizes but not %
growths (like 500MB increments for data, 50 for log) – For both data and log file
• Why? – Database autogrowth increases both internal and
external fragmentation of the data files
![Page 22: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/22.jpg)
Database sizing and options • Create Content DBs in projected target size!
– Create in Management Studio or T-SQL, then add to SharePoint – Options
• Projected size for data and log files • Data and log files to physically separate disks • Useful size for autogrow • Autogrow on • Collation: Latin1_General_CI_AS_KS_WS • Recovery model • Fulltext index off • Page Verify Checksum
– Script here: http://sharepointusergroup.corasworks.net/HOU/Lists/Speakers%20List/Attachments/17/SharePointScalabilityWhitepaper.pdf
![Page 23: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/23.jpg)
Creating Content DBs
Demo
![Page 24: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/24.jpg)
SQL Best Practices: Recovery models
• Recovery model – Full, if:
• The backup strategy includes frequent log backups (often: hourly)
• High availability like log shipping or mirroring will be used
– Otherwise: Use recovery model Simple to save IOs and simplify management
• Alter model DB to reflect selection – So new Content DBs are created with the correct option(s)
![Page 25: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/25.jpg)
SQL Best Practices: Instant File Initialization
• Enable Instant File Initialization for SQL Service account
• How? – Local Security settings
• Why? – Skips zeroing data pages
on file create/grow
![Page 26: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/26.jpg)
SQL Best Practices: Avoid TempDB contention
• Increase the number of TempDB data files – Configure tempdb with ¼ to ½ the number of data files as CPU cores (max of
8)
• How? – Properties of tempdb – Also size tempdb accordingly (ca. 25% of largest DB in total) – Only data files! Stick with 1 log file – Also place tempdb on fast disks
• Why? – If you see lots of PAGELATCH waits in sys.dm_os_waiting_tasks – Contention for writing to a single page in tempdb
![Page 27: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/27.jpg)
Resource Small Medium Large
Minimum DB server memory 4 GB 8 GB 16 GB
Processor L2 cache 2 MB > 2 MB > 2MB
Bus bandwidth Medium High High
Disks latencies (msec) < 20 < 10 < 10 (data) < 5 (T-log)
Network Gigabit Gigabit Gigabit
Network latency (msec) < 1 < 1 < 1
Configuration Recommended Capacities
![Page 28: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/28.jpg)
Configuration Processors
• Deploy on 64-bit if > 1000 users, or > 100 GB of data
• Use 64-bit SQL Server if using 64-bit OS • IA-64 only supported on DB tier, not on app tier • Plan for 2 WFE (dual core) / 40K users • Plan for 1db proc core / 20K users (min 8 cores) • Scale out beyond 8 processors
![Page 29: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/29.jpg)
Performance monitoring • Server (SQL)
– Processor: % Processor Time: _Total Below 50-75% except for peak times
– System: Processor Queue Length: Should be < 2 * # cores
– Memory: Pages/sec: (N/A) Should be below 100
• Disks – Transfer/sec for throughput trends – Disk sec/Read / Disk sec/Write for latency (IO bottleneck) – At least 25% free space on all disks
![Page 30: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/30.jpg)
Database maintenance • Good article: http://www.sqlskills.com/BLOGS/KIMBERLY/post/Database-
Maintenance-Best-Practices-Part-I-e28093-clarifying-ambiguous-recommendations-for-Sharepoint.aspx
• Recommended – Before implementing maintenance plans: Create and test
backups! – DBCC CHECKDB – Use sys.dm_db_index_physical_stats DMV to check for
fragmentation – On high fragmentation: reorganize or rebuild indexes
• dbo.proc_DefragIndexes from KB943345
![Page 31: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/31.jpg)
Database maintenance
• Not recommended – Drop & recreate indexes
– Database maintenance during business hours
– Update Statistics – done automatically • Except as described before
– Shrink databases if not absolutely necessary • If so: Do it right, do not use DBCC SHRINKDB
![Page 32: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/32.jpg)
The Logging Database aka WSS_UsageApplication
• "One Stop Shop for Health and Usage data
• “Data Providers” gather data across boxes
• Extensible framework
– Potential Custom Providers
– Potential Data Warehousing
– Potential Reporting and Analysis tools
The Farm’s “Black Box”
![Page 33: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/33.jpg)
Large List Throttling
Configurable List Throttling
And Thresholds
You control when and how
much!
![Page 34: SQL Server and SharePoint - Best Practices presented by Steffen Krause, Microsoft](https://reader034.vdocument.in/reader034/viewer/2022042713/54890e2db47959140d8b58b5/html5/thumbnails/34.jpg)
For more info • Webcasts
– http://www.microsoft.com/germany/technet/webcasts
• Blog Steffen Krause – http://blogs.technet.com/steffenk
• SQL Server/SharePoint Whitepaper – http://technet.microsoft.com/de-de/library/cc263261.aspx (no Shrink in
maintenance plan!!!)
– http://technet.microsoft.com/en-us/library/cc262731.aspx
• Defragmenting databases – http://support.microsoft.com/kb/943345/