Trustworthy DatabasesTrustworthy Databases

Jim Gray (gray@microsoft.com)Jim Gray (gray@microsoft.com)Distinguished EngineerDistinguished EngineerMicrosoft CorporationMicrosoft Corporation

AgendaAgenda

Trustworthy Computing Trustworthy Computing and Databasesand Databases

Yukon SecurityYukon Security Yukon AvailabilityYukon Availability

Pre-InternetPre-InternetWalled Information SocietyWalled Information Society

DevicesDevices

InformatiInformationon

PeoplePeople

SystemsSystems

Digital Decade:Digital Decade:Everything is ConnectedEverything is Connected

DevicesDevicesPeoplePeople

SystemsSystemsInformatiInformationon

The Digital Decade Requires The Digital Decade Requires Trustworthy ComputingTrustworthy Computing Y2K, Melissa, Nimda, Code Red…Y2K, Melissa, Nimda, Code Red…

causedcaused Gate’s Trustworthy Computing memo.Gate’s Trustworthy Computing memo.

The biggest challenge the industry faces.The biggest challenge the industry faces. Moved Trustworthy ComputingMoved Trustworthy Computing

from top 10 from top 10 to top 1 priority.to top 1 priority.

““Trustworthy Computing is the highest priority Trustworthy Computing is the highest priority for all the work we are doing. for all the work we are doing.

We must lead the industry to a whole new level We must lead the industry to a whole new level of Trustworthiness in computing. “of Trustworthiness in computing. “

Trustworthy Computing’s Four PillarsTrustworthy Computing’s Four Pillars

SecuritySecurity Defend system against attacksDefend system against attacks

PrivacyPrivacy Control personal data storage and useControl personal data storage and use

ReliabilityReliability System System alwaysalways works correctly works correctly

Business integrityBusiness integrity Be clear, open, respectful, and Be clear, open, respectful, and responsive to customers and to publicresponsive to customers and to public

Trustworthy Database PillarsTrustworthy Database Pillars

AvailabilityAvailability Online operation / evolutionOnline operation / evolution Self-healing automatic-recoverySelf-healing automatic-recovery

SecuritySecurity Data fortresses Data fortresses within software and serviceswithin software and services Bugs hit codeBugs hit code and and datadata Simple security modelSimple security model

PrivacyPrivacy Clear and simple information use policies Clear and simple information use policies Enable fine-grain control of data usage Enable fine-grain control of data usage

Trustworthy Database Tenets Trustworthy Database Tenets Secure by DesignSecure by Design

Secure and robust codeSecure and robust code Threat analysis and testingThreat analysis and testing

Secure by DefaultSecure by Default Default configuration is a secure systemDefault configuration is a secure system Discourage insecure configurations Discourage insecure configurations Minimize attack surface Minimize attack surface

install only necessary components install only necessary components

Secure by DeploymentSecure by Deployment Principle of least privilegePrinciple of least privilege

Grant minimal permission required to functionGrant minimal permission required to function Low privileged service accountsLow privileged service accounts

Security assessment / admin wizardsSecurity assessment / admin wizards Automate / Assist software maintenanceAutomate / Assist software maintenance

AgendaAgenda

Trustworthy Computing Trustworthy Computing and Databasesand Databases

Yukon SecurityYukon Security Yukon AvailabilityYukon Availability

Security PushSecurity Push3 Months 3 Months DedicatedDedicated to Security to Security

PreparationPreparation Goal: Full team Goal: Full team

(800) productive (800) productive from start from start Identify Identify

ComponentsComponents Complete Complete

threat modelsthreat models Complete Complete

EducationEducation Security planSecurity plan Reps from all teamsReps from all teams Set up toolsSet up tools Infrastructure set upInfrastructure set up

PushPush Review 5M+ LOC Review 5M+ LOC Two releases in serviceTwo releases in service One release in devOne release in dev 100% team focus 100% team focus

Dev, Test, PM, & UEDev, Test, PM, & UE No non-security workNo non-security work

Three pronged:Three pronged: Targeted code reviewsTargeted code reviews Audit tools Audit tools Threat driven testingThreat driven testing

Exit Criteria:Exit Criteria:SQL2K: SQL2K:

all security bugs fixedall security bugs fixedYukon: Yukon:

all sec bugs registeredall sec bugs registered

ExitExit New mentalityNew mentality All security bugs fixedAll security bugs fixed Ship SP3Ship SP3 Much larger test suiteMuch larger test suite Swat team continuesSwat team continues Black hat teamBlack hat team Post-mortem on all Post-mortem on all

customer found customer found security bugssecurity bugs

Testing:Testing:Run at low privileged Run at low privileged Complete coverage Complete coverage

in min privilege in min privilege Audit tools Audit tools

Better test and Better test and deployment tools.deployment tools.

Lockdown: Lockdown: Results and lessons learnedResults and lessons learned SQL Server 2000 Service Pack 3SQL Server 2000 Service Pack 3

More secure code basesMore secure code bases Development process improvementsDevelopment process improvements

Security conscious designs and codeSecurity conscious designs and code Mandatory security focused code reviewsMandatory security focused code reviews Accountability of code/designAccountability of code/design

Development toolsDevelopment tools Baked in to build processBaked in to build process

Implementation lockdown won’t sufficeImplementation lockdown won’t suffice Need to help users secure systems (more later)Need to help users secure systems (more later) Need better documentation and tools (more later)Need better documentation and tools (more later)

Yukon design changesYukon design changes

Secure By Design: YukonSecure By Design: Yukon

Row Level SecurityRow Level Security Can secure sets of rowsCan secure sets of rows Generalizes view mechanismGeneralizes view mechanism Predicate Restricts table subset Predicate Restricts table subset Leverages SQL query optimizerLeverages SQL query optimizer Leverages column/table permissions Leverages column/table permissions

Basis for privacy policy Basis for privacy policy Fine gain access controlFine gain access control

Basis for catalog securityBasis for catalog security

Secure By Design: YukonSecure By Design: Yukon

Catalog and Metadata securityCatalog and Metadata security Minimal public permissions Minimal public permissions

Prevents information disclosurePrevents information disclosure System tables are catalog viewsSystem tables are catalog views

row level securedrow level secured Object metadata only visible to:Object metadata only visible to:

OwnerOwner Principals with permission on object Principals with permission on object

User / Schema separationUser / Schema separation Separation of principal and schemaSeparation of principal and schema App still works when owner changesApp still works when owner changes

Secure By Design: YukonSecure By Design: Yukon

Granular permissionsGranular permissions More permissions at multiple scopesMore permissions at multiple scopes Principle of least privilege Principle of least privilege

e.g., No need to be sysadmin to run profilere.g., No need to be sysadmin to run profiler

Password policyPassword policy Consistent policy across enterpriseConsistent policy across enterprise Enforcement ofEnforcement of

Password StrengthPassword Strength Password ExpirationPassword Expiration Account lockoutAccount lockout

Secure By Default Secure By Default

SQL Server 2000 SP3SQL Server 2000 SP3 Require strong SA password on upgradeRequire strong SA password on upgrade Tighter permissions on Stored ProceduresTighter permissions on Stored Procedures Cross DB Ownership Chaining lockdownCross DB Ownership Chaining lockdown MSX account not auto-generated SQL loginMSX account not auto-generated SQL login MSDB off network by defaultMSDB off network by default

YukonYukon Domain password policies enforced by defaultDomain password policies enforced by default Metadata secured by defaultMetadata secured by default SA password required in all Auth modesSA password required in all Auth modes Stronger authentication protocol for SQL logins Stronger authentication protocol for SQL logins

Secure Deployment: Helping Secure Deployment: Helping Administrators Secure SystemsAdministrators Secure Systems

Secure Code samples and guidanceSecure Code samples and guidance In all documentation and booksIn all documentation and books

Doc team was part of security pushDoc team was part of security push Best security practices in all Best security practices in all

documentationdocumentation Security Best Practices checklistSecurity Best Practices checklist

Security ToolsSecurity Tools For assessing security For assessing security For vulnerability detectionFor vulnerability detection For patching systemsFor patching systems

Secure Deployment: Secure Deployment: Security Tools: MBSASecurity Tools: MBSA

Microsoft Baseline Microsoft Baseline Security AnalyzerSecurity Analyzer Verify current Verify current

configuration securityconfiguration security Local and remote Local and remote

scans scans Windows, Windows, IIS, IIS, Exchange, Exchange, SQL serverSQL server More in future More in future

Graphical and scriptableGraphical and scriptable

Secure Deployment: Secure Deployment: The Slammer lesson: short-term responseThe Slammer lesson: short-term response Response Team Response Team working 24 X 7working 24 X 7 Tool improvements continueTool improvements continue

SQL Critical UpdateSQL Critical Update Geared towards easier application of patchesGeared towards easier application of patches

SQL ScanSQL Scan Scans vulnerable instances in domain (or IP range)Scans vulnerable instances in domain (or IP range) Can optionally disable instancesCan optionally disable instances

SQL CheckSQL Check Scans all vulnerable instances on local boxScans all vulnerable instances on local box Can optionally disable bad servicesCan optionally disable bad services

Software Update Service (SUS)Software Update Service (SUS)

Secure Deployment: Secure Deployment: Moving forwardMoving forward

Increased focus on Increased focus on deploymentdeployment andand vulnerability assessment toolsvulnerability assessment tools

Tighter integration with MBSATighter integration with MBSA Microsoft update modelMicrosoft update model

Allow publish / subscription modelAllow publish / subscription model Allow publish / distributor / subscribe modelAllow publish / distributor / subscribe model Cover all software (not just OS)Cover all software (not just OS)

Document knowledge and experience Document knowledge and experience

Trustworthy DatabaseTrustworthy DatabaseSQL 2000 And YukonSQL 2000 And Yukon Secure Secure by Designby Design

Secure and robust codeSecure and robust code Threat analysis and testingThreat analysis and testing

Secure Secure by Defaultby Default Default configuration is a secure systemDefault configuration is a secure system Minimize attack surface Minimize attack surface

Secure Secure by Deploymentby Deployment Principle of least privilegePrinciple of least privilege Automate / Assist software maintenanceAutomate / Assist software maintenance Good tools for security assessment / adminGood tools for security assessment / admin

AgendaAgenda

Trustworthy Computing Trustworthy Computing and Databasesand Databases

Yukon SecurityYukon Security Yukon Availability Features Yukon Availability Features

AvailabilityAvailability

Available: Correctly services requestsAvailable: Correctly services requests within specified time within specified time

AvailabilityAvailability

Un-availability: Un-availability:

MTTF and MTTR equally importantMTTF and MTTR equally important

MTTF

MTTR

MTTRMTTF

MTTF

RepairToTimeMeanFailureToTimeMean

FailureToTimeMean

1

______

___

MTTF

MTTR

MTTRMTTRMTTFMTTF

AvailabilityAvailabilityReducing outagesReducing outages

QualityQuality: Design, code-read, test, …: Design, code-read, test, … SecureSecure by design, default, deployment by design, default, deployment Online operationsOnline operations: Many outages are operations tasks: Many outages are operations tasks

Online password changeOnline password change Online (re)-orgOnline (re)-org Online index build/drop/reorgOnline index build/drop/reorg Partitions for bulk operationPartitions for bulk operation

Simpler operationsSimpler operations: Many outages are mistakes: Many outages are mistakes More wizards (pre-scripted)More wizards (pre-scripted) Better scriptingBetter scripting

MTTRMTTRMTTFMTTF

AvailabilityAvailabilityReducing outagesReducing outages

MTTRMTTRMTTFMTTF

Online Index buildOnline Index build Build index in background (table always up)Build index in background (table always up)

(also works for rebuild/drop)(also works for rebuild/drop) No table lock – good for foreign keys tooNo table lock – good for foreign keys too

Snapshot Isolation and ViewpointsSnapshot Isolation and Viewpoints Snapshot Isolation: Snapshot Isolation: Read-consistent scans Read-consistent scans

Readers don’t wait for writersReaders don’t wait for writers EphemeralEphemeral

Viewpoint:Viewpoint: Read-only copy of DB Read-only copy of DB PersistentPersistent

How ViewPoints WorkHow ViewPoints Work

Looks like a new Read-Only databaseLooks like a new Read-Only database Very cheap to createVery cheap to create ViewPoint uses Copy On Write ViewPoint uses Copy On Write

to make snapshot of the DBto make snapshot of the DB Can be used for reportingCan be used for reporting Can recover to ViewPoint – fat finger deleteCan recover to ViewPoint – fat finger deleteData BaseData Base

Viewpoint:Viewpoint:

changechange

Old valueOld valuePagePage

DirectoryDirectory

Online bulk Data LoadOnline bulk Data LoadPartitions Partitions Partition Tables + Indices by hash / rangePartition Tables + Indices by hash / range Transparent to applicationsTransparent to applications Can add, drop, split, merge partitionsCan add, drop, split, merge partitions

in seconds.in seconds. Create new filegroup FCreate new filegroup F Create empty version of the table F.T (no keys)Create empty version of the table F.T (no keys) Heap load F.THeap load F.T Index F.TIndex F.T Now add F.T to online table T (takes a second)Now add F.T to online table T (takes a second)

FF

Table TTable TTable TTable T

AA BB CC DD EE FF

AvailabilityAvailabilityImproving repairImproving repair Backup improvements Backup improvements

All datatypes All datatypes (full text indices, cubes, filestreams, ...)(full text indices, cubes, filestreams, ...)

Optional multiple copies and checkingOptional multiple copies and checking Filegroup granularity for simple recoveryFilegroup granularity for simple recovery

Restore/recovery improvementsRestore/recovery improvements Online: Page, file, filegroup granularityOnline: Page, file, filegroup granularity ““Instant” file format (must be admin) Instant” file format (must be admin) Much shorter outage at software upgradeMuch shorter outage at software upgrade FASTER: Just redo FASTER: Just redo

Redo committedRedo committed Undo incomplete xactionsUndo incomplete xactions

RecoveryRecovery

MTTRMTTRMTTFMTTF

Availability: ClustersAvailability: Clusters

Cluster FailoverCluster Failover Shipped with SQL 7Shipped with SQL 7 Much improved in SQL 2000Much improved in SQL 2000 Yukon adds 8-pack supportYukon adds 8-pack support

Robust Robust Widely deployedWidely deployed First-line-of-defenseFirst-line-of-defense

MTTRMTTRMTTFMTTF

Availability: Mirrored SystemsAvailability: Mirrored Systems

How do you deal with catastropheHow do you deal with catastrophe Fire, flood, storm, earthquake, power/net breakdown?Fire, flood, storm, earthquake, power/net breakdown? Data center moveData center move Sabotage, Gremlins?Sabotage, Gremlins?

Cluster failover can take minutesCluster failover can take minutes What if I want 5 -9s? What if I want 5 -9s? (5 minutes/year)(5 minutes/year)

Mirrored SystemsMirrored SystemsTwo independent systemsTwo independent systems

Replicas of one anotherReplicas of one another Continuous SQL log shippingContinuous SQL log shipping Mirror tracks primaryMirror tracks primary Witness breaks tiesWitness breaks ties

Easy to configure and admin Easy to configure and admin

MirrorMirror

MTTRMTTRMTTFMTTF

PrimaryPrimary

WitnessWitness

Database MirroringDatabase Mirroring

Database failover – Database failover – Instant standbyInstant standby Very fastVery fast … < 3 seconds … < 3 seconds Automatic or manual Automatic or manual Automatic re-sync after failoverAutomatic re-sync after failover

Automatic and transparent Automatic and transparent client redirectclient redirect

No single point of failureNo single point of failure No special hardware; standard computers No special hardware; standard computers

and storageand storage Minimal impact to transaction throughputMinimal impact to transaction throughput

PrimaryPrimary MirrorMirror

WitnessWitness

FailedFailedPrimaryPrimary

NewNewPrimary Primary

WitnessWitness

PrimaryPrimary MirrorMirror

WitnessWitness

RepairedRepairedPrimaryPrimary

NewNewPrimary Primary

WitnessWitness

How Database Mirroring WorksHow Database Mirroring Works

Transaction Log Shipping – 2-safeTransaction Log Shipping – 2-safe Backup system in continuous redoBackup system in continuous redo Database “up” when redo completesDatabase “up” when redo completes

MirrorMirrorPrimaryPrimary

WitnessWitness

SQL ServerSQL ServerSQL ServerSQL ServerSQL ServerSQL Server

LogLog LogLog

ApplicationApplication

Redo Redo RecoveryRecovery

Log recordsLog recordsLog recordsLog records

Ack!Ack!Ack!Ack!

Database MirroringDatabase Mirroring

Mark WistromMark WistromProgram ManagerProgram ManagerMicrosoft CorporationMicrosoft Corporation

Reporting On MirrorReporting On Mirror

Use Database ViewPoint on MirrorUse Database ViewPoint on Mirror

MirrorMirror

PrimaryPrimary

Reporting Reporting ClientClient

OLTP ClientsOLTP ClientsWitnessWitness

Viewpoint1Viewpoint1@ Noon@ Noon

Viewpoint2Viewpoint2@ 2PM@ 2PM

Spectrum Of TechnologiesSpectrum Of Technologies Maximize availability forMaximize availability for

Scale outScale out Offload primary serverOffload primary server

Heavy ReportingHeavy Reporting Mobile/disconnected usersMobile/disconnected users Autonomous data sharingAutonomous data sharing

Maximize availability Maximize availability System-of-record databasesSystem-of-record databases Zero data loss – current infoZero data loss – current info Mask downtimeMask downtime

PlannedPlanned UnplannedUnplanned

ReplicationReplication

MirrorMirror

Rock Solid SiteRock Solid Site

ReplicationReplication

ClusterCluster

Disaster TolerantDisaster Tolerant

Spectrum Used In CombinationSpectrum Used In Combination

Can mix and match Cluster, Mirror, ReplicaCan mix and match Cluster, Mirror, Replica

Failover Cluster

Failover Cluster

ReplicationReplicationM

irror

Mirror

Availability SummaryAvailability Summary

Deep analysis of availability, Deep analysis of availability, holistic approachholistic approach

Fewer outagesFewer outages Online operationsOnline operations Simpler operations Simpler operations Snapshots, partitions, …Snapshots, partitions, …

Faster repairFaster repair More complete backup/recoveryMore complete backup/recovery Finer grain (so faster) recoveryFiner grain (so faster) recovery Mirrored systemsMirrored systems

MTTRMTTRMTTFMTTF

Summary / Call To ActionSummary / Call To Action

Trustworthy Computing is more than Trustworthy Computing is more than securitysecurity

Yukon improvesYukon improvesTrustworthy Database supportTrustworthy Database support Secure by design / default / deploymentSecure by design / default / deployment

Deploy SQL Server 2000 today as the Deploy SQL Server 2000 today as the stepping stone to Yukonstepping stone to Yukon

Keep current and safe with the tools at Keep current and safe with the tools at www.microsoft.com/sqlwww.microsoft.com/sql and and www.microsoft.com/securitywww.microsoft.com/security

© 2002 Microsoft Corporation. All rights reserved.© 2002 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.