SQL Storage for the Real World

Brett Rouxbrettroux.blogspot.com

Overview• Requirements• Storage concepts• Storage options and priorities• SAN / DAS • HA / DR implications• Decide on a storage strategy• General guidance• References and more information• Q/A

Who Am I ?• I am a DBA• I am not a sales person• What qualifies me?• I don’t have all the answers!• Personal opinion:

▫Almost any reliable storage solution could be used effectively for your SQL server

▫Almost any solution could be a bad purchase if it is implemented incorrectly or does not fit in with your overall storage strategy

Who Should Be Here?• Aimed at DBA’s responsible for small to medium

SQL deployments• Will NOT be focusing on extreme capacity or

performance solutions• Going to avoid tuning, as Kevin covered this in

the previous session• Focus on overall strategy and options available –

so you can narrow your own focus in the future

Why Change Your Storage?• Is your storage just old?

▫ Unable to extend warranty?• Is your storage the bottleneck?

▫ Have you got enough RAM?• Do you just need more capacity?• Are you standardising or looking to fit in with the

rest of the company’s storage plan?• Do you have a new application to support?

Know Your Data!• Understanding your data and storage usage is

the key to everything• Is your environment OLTP, DSS or OLAP?• Do you need bandwidth or IO’s?• What read/write ratio?• How much data is there?• How much of that data is active?• Daily trends and monitoring• Use perfmon and DMV’s to investigate• Are you really using your TempDB?

Storage Concepts• What are IO’s?

▫8KB – 1024KB• What is disk latency?

▫Measured in ms▫20ms limit not

always relevant• What is bandwidth?

▫Measured in Gb• Cache

• Random vs sequential▫No way to truly

measure▫Very little chance of

absolutes▫Virtualisation tends

to make all IO’s random

• Queue Depth• Read/Write Ratio

▫Important with SSD’s

IO Size vs Throughput

http://download.intel.com/support/network/sb/inteliscsiwp.pdf

RAID Types• Striping – fast ,no redundancy and lower

reliability• RAID Factor: 1

RAID 0: • Mirroring - no performance gain• RAID Factor: 2RAID 1: • Stripping with parity – read gain, but write

overhead• RAID Factor: 4

RAID 5: • Dual parity – higher reliability, big write

overhead*• RAID Factor: 6

RAID 6: • Striped Mirror–best write performance, lowest

capacity• RAID Factor: 2

RAID10: • Striped RAID 5:Good overall compromise• RAID Factor: 4

RAID 50:

Storage Options• Known quantity• Reasonably predictable• Fantastic capacity• Can suffice for most apps• 10K drive = 100 – 120 IOPS• 15K drive = 150 – 180 IOPS

HDD

• Future of storage• Cost can be prohibitive• Fantastic IO performance• Write falloff varies between models• Write endurance issue for write

intensive apps• All SSD’s NOT created equal• IO’s vary from 1K-1M depending on the

model and manufacturer

SSD

Storage Priorities

SQL Storag

e

Performance

CapacityCost Effective

Direct Attached Storage• Easy to implement• Known quantity and easier to troubleshoot• No longer limited in terms of performance• Small footprint• Potentially the cheapest solution• Cannot be used for clustering by default• Several SAN benefits can be achieved when

using storage virtualisation on DAS• Can’t be affected by other systems

PCIe SSD• New standard in terms of latency• Completely changed perceptions of DAS• Limited capacity• High cost/capacity, but low relative to

performance• Fantastic for Denali TempDB• Can affect CPU and RAM utilisation• Multiple vendors with varying performance and

quality

Storage Area Networks• Getting more affordable• Massive benefits in large deployments• Powerful functionality

▫Snapshots▫Online capacity expansion▫Automated tiering▫SAN replication▫Thin provisioning

• Boot from SAN can increase availability• Increased redundancy is possible • Shared environment, so others can affect SQL

Framed vs Frameless SANFramed SAN Frameless SAN• Has dedicated head unit• Most common• High initial cost• Lower expansion cost• Capacity expansion only • Head units can become

a bottleneck• Expensive head unit

upgrades

• Each shelf has controller

• Newer generation• Lower entry cost• Higher expansion cost• Increased

performance with expansion

• Limited capacity / scalability

Framed SAN

Frameless SAN

Rackmount SSD• Generally independent units

▫Although some can act as a SAN• Individual unit capacity is limited, and is usually

bought up front• Incredibly fast but often lacking additional

functionality• Alternatively very feature rich, but without track

record• Possibly higher latency than PCIe SSD, but

allows for clustering or expansion when slots are limited

SAN TechnologiesSAN Type ISCSI 10GbE ISCSI Fibre ChannelBandwidth 1Gb (usually x4) 10Gb (Soon 40Gb) Up to 8Gb (Soon

16Gb)Implementation Easy Easy Average

Ongoing Support Depends on networking

Depends on networking

Usually Set and forget

Pros You already have the infrastructure

Easy to implement Very stable

Known quantity Good roadmap De facto standard

Cons Low bandwidth Initial cost Initial cost

Easily affected by comms team Learning curve

Other SAN options:

FCoE Successor to FC in converged data centre's, similar con’s to 10GbE

Infiniband Incredibly fast, low latency but not common in MS environments

ATA over Ethernet (AoE)

Niche option, but extremely simple to use

Alternative SAN Technologies

• Integrate existing storage• Add functionality to older units

Storage Virtualisation

• Excels at integrating local storage • Ideal for remote offices• Test virtualisation overhead before

committing• Maintain your own hardware

Virtual Storage Appliances / SAN software

SAN vs DASFeature DAS SAN

Snapshots None* Yes

Thin Provisioning None Yes

Replication None* Yes

Cloning None* Yes

Installation Easy Longer and more intricate

Maintenance Low Depends on how you use the SAN – can be high

Online expansion None Yes

Cost Starts very low Significantly higher

Additional functionality

None ASAP’s, VSA’s etc.

* Possible through 3rd party software

High Availability / Disaster RecoveryTechnology DAS SANStandalone YES YES

Clustering NO YES

Mirroring YES YES

AlwaysOn YES YES

Backups YES Snapshots to backup server

Log shipping YES SAN Replication

NDMP NO SAN dump to tape

Storage Layers – DR OptionsSQL

Filter Driver

Operating system

Hypervisor

DAS

Disk

SANArray / Storage Virtualisation

Disk

Possible SAN Features• Block or file depending on vendor• Do you really want to de-duplicate everything?• Varying effectiveness with SQL• Inline or Asynchronous• Windows 8 may have this as standard

De-duplication

• Very effective with raw SQL data• Can head units handle additional load?• What performance impact in terms of latency?• Not useful on compressed partitions

Compression

SSD Acceleration• Add SSD to almost any SAN• Instant performance boost• Potentially more cost effective for large datasets

SAN Acceleration (ASAP’s)

• Announced by more than one vendor• Only accelerates reads in clustered environment• Accelerates reads and writes in standalone configuration

PCIe SSD acceleration of SAN LUNS

• Large unknowns as this is not available yet• Would give the best of both worlds

SAN controlled PCIe drives

SSD ASAP

SSD LUN Accelerati

on

SAN Controlle

d PCIe

Choosing A Vendor• Give all the vendors the same set of requirements

and let them propose solutions• For example: IO requirement, usable capacity,

connectivity, form factor, read/write ratio, lifespan and budget for the project

• Feedback to vendors and allow them to improve the solution iteratively – they won’t get it right first time

• Make sure you are getting the best pricing!▫ Some vendors first quote is 50% off retail...

• Build a relationship – storage is never once off!• Make sure the solution is both balanced and

redundant

General Guidance• Separate Data, Logs and TempDB• Separate DB’s if performance is justified• Put logs on fastest disk possible

▫ For write heavy DB’s or OLTP▫ Remember that log writes are sequential...

• OLAP and DSS generally like bandwidth and can handle more latency

• Make data files of equal size• Pre-size data and log files• Autogrow loves to work at the worst times

TempDB• TempDB can be very IO heavy, but is easy to

move• Check for contention on TempDB

▫Consider adding data files if required▫Start at 0.25 per core and move up▫Additional log files are not beneficial

• Storage engine blog on allocation bottlenecks:▫ http://bit.ly/q1JDJN

• Denali allows for TempDB on local storage in clustered instances

Storage Configuration• Write cache is generally more useful to SQL

▫ You may need to change the number of data files to get the best from your storage

▫ Check for bottlenecks in the data path• CHECK PARTITON ALIGNMENT !!!

▫ http://msdn.microsoft.com/en-us/library/dd758814(v=sql.100).aspx

• Check your MPIO is working• Update storage firmware and use latest drivers

Storage Testing• Benchmark to validate, baseline and test• Test the redundancy – pull some cables and

drives...• Ensure you test more than your cache• SQLIO

▫ http://bit.ly/p85dJj

• IOMeter▫ http://bit.ly/oewi6g

• SQLIOSim▫ http://bit.ly/rbXt6A

• AS SSD Benchmark▫ http://bit.ly/pTlk0k

Diagnostic Tools• Logical Disk

▫ Avg Disk Sec/Read (Write) - Latency▫ Disk Read Bytes/sec (Write) – Bandwidth▫ Disk Reads/sec (Writes) – IO’s▫ Avg Disk Queue Length

Deprecated but high numbers can indicate room for improvement

• Boot LUN doesn’t count unless page file is being hit or you are booting from SAN

• sys.dm_io_virtual_file_stats(DB_ID(), NULL)▫ Itzik Ben-Gan on IO stats: http://bit.ly/rcy2TX

• sys.dm_os_wait_stats▫ Look for PAGEIOLATCH_*

• sys.dm_db_index_usage_stats

References• Storage Search

▫ storagesearch.com• MS Storage Best

Practices:▫ http://bit.ly/qLfyli▫ http://bit.ly/oajks8▫ http://bit.ly/oZsZ71

• Glenn Berry’s blog:▫ sqlserverperformance.wordpress.c

om• Intel Whitepaper: Simple,

Reliable Performance for iSCSI Connectivity.▫ http://bit.ly/o95NTL

• My Blog:▫ brettroux.blogspot.com

• Brent Ozar:▫ brentozar.com

• MCM Training video’s:▫ http://bit.ly/p7IlSR

• Storage Mojo▫ storagemojo.com

• AnandTech▫ anandtech.com

• Paul Randall’s blog:▫ sqlskills.com/BLOGS/PAUL

• Joe Chang’s blog:▫ sqlblog.com/blogs/joe_chang/

Summary• Requirements• Storage concepts• Storage options and priorities• SAN / DAS • HA / DR implications• Decide on a storage strategy• General guidance

Questions? Thank You For Attending!