SQL Server 2005Memory Management

Lesson 1

• Description– Operating System and SQL Server

• Learning Goals:– Physical Memory (RAM) – Virtual Address Space (VAS)– Interaction between SQL Server and

Operating System– Resource monitor

Windows Operating System

• Windows Memory Manager– Abstracts the concept of memory to the applications– Switches data from RAM to Paging File or vice-versa

• Trimming Memory

• Different Platform Versions– 32/64-bit Implementations

Windows Applications

• Process– Virtual Address Space– Threads– Other resources

• Virtual Address Space– 32-bit Native: 2-3GB– 64-bit Compatibility: 4GB– 64-bit Native: 16TB

SQL Server Process• Virtual Address Space

– SQL Engine– Buffer Pool

• SQL Engine– Executable (SQLSERVR.EXE)– Libraries (DLL)– Thread Structures

• Buffer Pool– Sized accordingly to RAM– Limited by VAS

SQLSERVR.EXE

Buffer Pool

SQL Engine

Resource Monitor• Low Memory Conditions

– Buffer Pool– Virtual Address Space– Windows Memory Manager

• Resource Monitor– Background Tasks– Monitor memory pressure– Log the status– Broadcast to other components

SQLSERVR.EXE

Buffer Pool

SQL Engine

Other

Other

Paging FileRAM

Windows Memory Manager

Notification Example<Record id="1701" type="RING_BUFFER_RESOURCE_MONITOR" time="149740267"> <ResourceMonitor> <Notification>RESOURCE_MEMPHYSICAL_LOW</Notification> <Indicators>2</Indicators> <NodeId>0</NodeId> </ResourceMonitor> <MemoryNode id="0"> <ReservedMemory>1646380</ReservedMemory> <CommittedMemory>432388</CommittedMemory> <SharedMemory>0</SharedMemory> <AWEMemory>0</AWEMemory> <SinglePagesMemory>26592</SinglePagesMemory> <MultiplePagesMemory>17128</MultiplePagesMemory> <CachedMemory>17624</CachedMemory> </MemoryNode> <MemoryRecord> <MemoryUtilization>50</MemoryUtilization> <TotalPhysicalMemory>3833132</TotalPhysicalMemory> <AvailablePhysicalMemory>3240228</AvailablePhysicalMemory> <TotalPageFile>5732340</TotalPageFile> <AvailablePageFile>5057100</AvailablePageFile> <TotalVirtualAddressSpace>2097024</TotalVirtualAddressSpace> <AvailableVirtualAddressSpace>336760</AvailableVirtualAddressSpace> <AvailableExtendedVirtualAddressSpace>0</AvailableExtendedVirtualAddressSpace> </MemoryRecord> </Record>

Notification

Memory Node

Operating System

SQL Server Process

Lesson 2• Description

– Buffer Pool and Database Caching– Memory Clerks Concepts

• Objectives– Check the performance related to Buffer Pool – Identify memory dedicated for database cache– Extend the database cache with AWE– Identify the memory usage by components– Find who is stealing memory from Buffer Pool

Database Caching

• Data Unit– Pages = 8Kb

• Accessing Data– Logical Reads– Physical Reads

- SQL Server:Buffer Manager – Page Lookups/sec- SQL Server:Buffer Manager – Page Reads/sec- SQL Server:Buffer Manager – Readahead Pages/sec- SQL Server:Buffer Manager – Page Life Expectancy

Disks

Buffer PoolMemory

PhysicalRead

LogicalRead

Buffer Pool Size• Configuration

– Minimum and Maximum Limits

• Static Memory Management– Define a fixed amount of memory

• Dynamic Memory Management– SQL Server dynamically adjust the amount of memory used

- SQL Server:Buffer Manager – Total Pages- SQL Server:Buffer Manager – Target Pages

Memory Distribution• Data Pages

– Cache Database Pages

• Stolen Pages– Internal SQL components

(eg, Optimizer)

• Free Pages

- SQL Server:Buffer Manager – Database Pages- SQL Server:Buffer Manager – Stolen Pages- SQL Server:Buffer Manager – Free Pages

STOLEN

FREE

DATA

Buffer P

ool

Buffer Pool Extension• AWE

– Going beyond VAS– Only used for Data

Cache

• Comparison– 3GB x AWE– Can I use both

together?

STOLEN

FREE

DATA

SQL Server

Operating System

2GB

8GB

Aging Buffers

• Two purposes: – Tries to keep a minimum number of free

buffers (freeing dirty buffers requires I/O)– Keeps enough physical memory free on the

machine to avoid paging• Sweeps across BUFs to “age” them• Many things that it can’t remove

Demo

• Who’s using my memory?

Internal SQL Server Allocations

ProcedureCache CLR

SQL Memory Node

MC MCMC MCMemory Clerks

Allocation Statistics

SQL Memory Node

MC MCMC

Single Page AllocatorMulti Page AllocatorVirtual Memory InterfaceShared Memory Interface

CACHESTORE_OBJCP

CACHESTORE_SQLCP

CACHESTORE_PHDR

ProcedureCache

Memory Allocation

SQL Memory Node

Single Page Allocator

Multi Page AllocatorVirtual Memory InterfaceShared Memory Interface

SQLSERVR.EXE

Buffer Pool

SQL Engine

Demo

• Who’s stolen my memory?• CLR consuming memory• SQL Plans growing

ENDThe following slides have ideas for

the next workshop update

Lesson 3

• Description– Memory Clerks Concepts

• Objectives– Understand the concept of Memory Clerks– Identify the memory usage by components– Find who is stealing memory from Buffer Pool

Memory Clerk

• Allocates the memory as needed• React differently to Memory Pressure• Sys.dm_os_memory_clerks

MEMORYCLERKResource Monitor

Controlling Memory Usage

• Specifics Types – Custom Memory Clerk– Cache and User Store– Object Store

CUSTOMMEMORY

CLERK

CACHESTORE

USERSTORE

OBJECTSTORE

Custom Memory Clerk

• Each Memory Clerk implements the action to the notification

• Examples– General– CLR– Full Text

Object Store

• Free half of the entries• sys.dm_os_memory_pools

Cache and User Store

• Remove the cheapest items • Use Clock hands algorithm• sys.dm_os_memory_cache_counters

Clock Hand

• External– Resource Monitor

• Internal– Internal Limits / Boundaries (INTERNAL)

• sys.dm_os_memory_cache_clock_hand

Memory Broker