demand technology software, inc. memory leaks demand technology 1020 eighth avenue south, suite 6,...
TRANSCRIPT
Demand Technology Software, Inc.
Memory LeaksDemand Technology
1020 Eighth Avenue South, Suite 6, Naples, FL 34102phone: (941) 261-8945 fax: (941) 261-5456
e-mail: [email protected]://www.demandtech.com
2Demand Technology, Inc. Memory Leaks
Outline Topics
Virtual Memory concepts
Virtual Memory constraints in 32-bit Windows
Leaking processes
Page faults/sec
Vir t ualaddress
space
V ir t ualaddress
space
P hysical M em ory
Page Tables
3Demand Technology, Inc. Memory Leaks
Virtual Memory Manager
4 GB virtual per process address space Lower 2 GB - Private
Page 0 reserved Code pages Heap
Upper 2 GB - System System code Shared dlls System cache
User
System
0000 0000
7fff ffff16
ffff ffff16
4Demand Technology, Inc. Memory Leaks
UserUser
Virtual Memory Manager
All address spaces share the virtual addresses from the same upper 2 GB of the System area
All virtual addresses in the lower 2 GB User area are unique
Shared memory in the System range can be used for IPC
User
System
0000 0000
7fff ffff16
ffff ffff16Shared
5Demand Technology, Inc. Memory Leaks
Virtual Memory Manager
Boot.ini /3 GB switch Virtual storage
constraint relief for some applications
SQL Server Exchange 2000 Etc.
0000 0000
c000 0000 16
ffff ffff 16
User
System
Shared
6Demand Technology, Inc. Memory Leaks
AWE
Windows 2000 - Address Windowing Extensions Address RAM > 4 GB
limit -Intel Pentium III, IV
requires PSE36 Driver Real addresses only Not backed on paging
files– PTEs remain 32-bit 0000 0000
c000 0000 16
ffff ffff 16
User
System
Shared
PSE36
0f ffff ffff 16
AW
E
7Demand Technology, Inc. Memory Leaks
Win64 Virtual Memory
Architectural component 64 bit 32 bit
Virtual Memory 16TB 4GB
Paging File Size 512 TB 64 GB
Hyperspace 8 GB 4 MB
Paged Pool 128 GB 192 MB
Non-paged Pool 128 GB 128 MB
System cache 1 TB 1 GB
System PTE (page table entries) 128 GB 660 MB
8Demand Technology, Inc. Memory Leaks
Virtual Memory Manager
Real Memory allocation Counters Available Bytes Pool non-paged Bytes, Pool Paged Resident Bytes, System Cache Resident Bytes, System Code Resident Bytes, System Driver Resident Bytes
Instantaneous Counters (all reported as Bytes)
Cache Bytes is actually the pageable System Working Set = Pool Paged Resident Bytes + System Cache Resident Bytes
+ System Code Resident Bytes + System Driver Resident
Bytes
9Demand Technology, Inc. Memory Leaks
Virtual Memory Manager
Memory allocation Counters Available Bytes Available KBytes Available MBytes
Pool non-paged Bytes, Pool Paged Resident Bytes, System Cache Resident Bytes, System Code Resident Bytes, System Driver Resident Bytes
10Demand Technology, Inc. Memory Leaks
What is the difference between Pool Paged Bytes and Pool Nonpaged Bytes?
Which applications would use which VM pool?
11Demand Technology, Inc. Memory Leaks
Private
System
10000
7F F E F F F F
A 4000000
C1000000
E 0000000
F F F F F F F F
PTEs/ cache extension
System data cache
Paged system pool
80000000
C0000000
Virtual Memory map
12Demand Technology, Inc. Memory Leaks
Private
System
10000
7F F E F F F F
A 4000000
C1000000
E 0000000
F F F F F F F F
PTEs/ cache extension
System data cache
Paged system pool
80000000
C0000000
Virtual Memory map
Paged Memory mapped
files Shared DLLs
File Server IIS (html, jpg, gif)
Page tables
nonPaged: I/O buffers used by
device drivers TCP Session data
13Demand Technology, Inc. Memory Leaks
Private
System
10000
7F F E F F F F
A 4000000
C1000000
E 0000000
F F F F F F F F
PTEs/ cache extension
System data cache
Paged system pool
80000000
C0000000
Virtual Memory map
Virtual memory map is static, created once during OS initialization
When any one area is full, memory allocations to that pool will fail
One reserved area of virtual memory can fill up faster than the others;
14Demand Technology, Inc. Memory Leaks
Nonpaged pool size (ref. Q126402):
MinimumNonPagedPoolSize = 256KMinAdditionNonPagedPoolPerMb = 32KDefaultMaximumNonPagedPool = 1 MBMaxAdditionNonPagedPoolPerMb = 400KPTE_PER_PAGE = 1024PAGE_SIZE=4096
NonPagedPoolSize = MinimumNonPagedPoolSize + ((Physical MB - 4) * MinAdditionNonPagedPoolPerMB)
If NonPagedPoolSize > 128 MB THEN NonPagedPoolSize = 128 MB
15Demand Technology, Inc. Memory Leaks
Paged pool size:
Size = (2 * MaximumNonPagedPoolSize) / PAGESIZE
Size = (Size + (PTE_PER_PAGE - 1)) / PTE_PER_PAGE
PagedPoolSize = Size * PAGESIZE * PTE_PER_PAGE
If PagedPoolSize > 192 MB THEN PagePoolSize = 192 MB
Note: PagedPoolSize sizeof (RAM) for systems < 192 MB
16Demand Technology, Inc. Memory Leaks
Default Paged and Nonpaged pool size can be overridden:
17Demand Technology, Inc. Memory Leaks
Virtual Memory constraints
Virtual Memory constraints in 32-bit Windows tend to appear when sizeof (RAM) 4 GB 2 GB private area is not enough virtual memory for some
applications e.g., SQL Server, Exchange database (store.exe) Due to fragmentation, it is typically not possible to allocate all 2 GB
2 GB system area is not enough virtual memory for some applications
File cache for a conventional IIS-managed web site with many static .htm, jpg, gif, etc., files to retrieve
18Demand Technology, Inc. Memory Leaks
Virtual Memory constraints
Virtual Memory constraints in 32-bit Windows tend to appear when sizeof (RAM) 4 GB Ample RAM exists, but it is not possible for your
applications to access it due to virtual memory addressing limitations
Large number of Available Bytes
19Demand Technology, Inc. Memory Leaks
Virtual Memory constraints
So try the /3 GB switch 1 GB system area is not enough virtual memory for some
applications Possible shortage of Free System Page Table Entries Possible shortage of Nonpaged Pool
– Where Session data from TCP connections is stored
Due to fragmentation, it may not be possible to failover a 2 GB+ private address space (e.g., SQL Server, MS Exchange database – store.exe) using Microsoft Cluster Server (MCS)
During address space recovery on the standby node, the entire virtual memory allocation is acquired at one time
20Demand Technology, Inc. Memory Leaks
Exchange 2000 memory tuning Exchange default memory allocation
parameters are “self-tuning,” but may not be optimal on servers with > 1 GB RAM Adjust HKLM\SYSTEM\CurrentControlSet\Services\
SMTPSVC\Queuing MsgHandleThreshold & MsgHandleAsyncThreshold
HKLM\SYSTEM\CurrentControlSet\Services\Inetinfo\Parameters
FileCacheMaxHandles
21Demand Technology, Inc. Memory Leaks
Exchange 2000 memory tuning Store DB cache
Store cache normally expands to fill RAM But, maximum cache = 900 MB This value can be adjusted using ADSI Edit
tool: msExchESEParamCacheSizeMax msExchESEParamCacheSizeMin
Also, consider adjusting msExchESEParamLogBuffers attribute for active, back-end servers
22Demand Technology, Inc. Memory Leaks
Detecting memory leaks Processes that allocate virtual memory, but
later “forget” to free it. MS says leaks won’t happen in .Net
managed code due to automatic garbage collection
But, meanwhile…, Where to look depends on whether process
or system addresses are being allocated: Per Process: Virtual Bytes, Private Bytes, Pool Paged
Bytes, Handle Count System level: Memory Pool Paged Bytes, Pool Nonpaged
Bytes and the Objects Object
23Demand Technology, Inc. Memory Leaks
Detecting memory leaks
Look for a steady increase or a sharp spike in process Virtual Bytes, or the System’s Pool Paged Bytes. If RAM is not full, the leak may also be manifest
in the Memory allocation counters and result in increased paging, if RAM fills up.
For example:
24Demand Technology, Inc. Memory Leaks
Detecting memory leaks (1)Memory Leak example
0
50
100
150
200
250
300
350
400
Time of Day
MB
A vailable Bytes N on-paged Pool BytesPool Paged Resident Bytes System Cache Resident BytesSystem Code Resident Bytes System Dr iver Resident Bytes
25Demand Technology, Inc. Memory Leaks
Detecting memory leaks (2)
26Demand Technology, Inc. Memory Leaks
Detecting memory leaks (2)
27Demand Technology, Inc. Memory Leaks
Detecting memory leaks (2)
28Demand Technology, Inc. Memory Leaks
Detecting memory leaks (2)
29Demand Technology, Inc. Memory Leaks
Lab exercise. Open
SUPPORT.200205080300.EXCHG2K.sum.smf, SUPPORT.200205090300.EXCHG2K.sum.smf, and SUPPORT.200205100300.EXCHG2K.sum.smf and investigate virtual memory usage.
Can you find the application that is leaking?
30Demand Technology, Inc. Memory Leaks
Detecting Memory Leaks (3)
31Demand Technology, Inc. Memory Leaks
Detecting Memory Leaks (3)
32Demand Technology, Inc. Memory Leaks
Detecting Memory Leaks (3)
33Demand Technology, Inc. Memory Leaks
Detecting Memory Leaks (3)
34Demand Technology, Inc. Memory Leaks
Page replacement
Theoretically, Page Faults/sec
Cache faults/sec + Transition faults/sec + Demand Zero Faults/sec + Pages Read/sec
But empirically…
35Demand Technology, Inc. Memory Leaks
Page Faults/ sec
0
300
600
900
1200
1500
Time of Day
Pag
es/s
ec
0
300
600
900
1200
1500
cache faults/ secpage reads/ sect ransit ion faults/ secdemand zero faults/ secpage faults/ sec
36Demand Technology, Inc. Memory Leaks
Page replacement
Empirically, Page Faults/sec
Transition faults/sec + Demand Zero Faults/sec + Pages Read/sec
37Demand Technology, Inc. Memory Leaks
Questions
?