the present and future of large memory in db2• zparm realstorage_max defines the sandbox •...
TRANSCRIPT
![Page 1: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/1.jpg)
The Present and Future of Large Memory in DB2John B. ToblerSenior Technical Staff Member DB2 for z/OS, IBM
Michael SchultzAdvisory Software Engineer DB2 for z/OS, IBM
Session A11Wednesday October 16, 2013 11:00AM - 12:00PM
With the flattening of Moore's Law, processor speeds will not serve as a significant source of performance improvement in coming years. Database systems are looking to exploit large real memory to avoid I/O latency and CPU overhead. DB2 10 and future releases of DB2 will look to exploit large real memories and z/OS features such as large page sizes and flash memory. This session will cover the potential performance benefits of large real memories and how DB2 plans to exploit such opportunities.
![Page 2: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/2.jpg)
2Click to edit Master title style
Objectives
• Understand processor speed trends and the shift toward software to drive performance gains.
• Understand the present and future goals of memory availability on the z-platform.
• Understand the current (DB2 10 and DB2 Sequoia) exploitation of large memory pages (e.g., 1M, 2G page sizes)
• Understand other features like query processing that are looking to exploit large memories.
• Understand the future potential for DB2 exploitation of large memory and the potential ramifications for performance gains.
L2 Cache - 4 machine cycles• L3 Cache - variable 10‘s of machine cycles• L4 Cache - variable 100‘s of machine cycles• Real Memory - 1000 machine cycles2
![Page 3: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/3.jpg)
3Click to edit Master title style
Moore’s Law and Mainframe Evolution
0
1000
2000
3000
4000
5000
1997G4
1998G5
1999G6
2000z900
2003z990
2005z9 EC
2008z10 EC
2010z196
300MHz
420 MHz
550 MHz
770 MHz
1.2 GHz
1.7 GHz
4.4 GHz
5.2 GHz
MH
z
5.5 GHz
2012zEC12
• The observation made in 1965 by Gordon Moore, that density of transistors on integrated circuits would double every 2 years. This relates to processor speeds and computing capacity.
3
![Page 4: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/4.jpg)
4Click to edit Master title style
Moore’s Law and Mainframe Evolution
• IBM set world microprocessor records with the z196 and zEC12 with the zEC12 clocked at 5.5GHZ with 2.75 billion transistors in 597mm2. Unfortunately this won’t last forever
"It can't continue forever. The nature of exponentials is that you push them out and eventually disaster happens"
Gordon Moore 2005
• We seen the ‘speed’ trend flattening so we must turn our attention to other means to get performance.
• Emphasis will turn to software to more intelligently use hardware resources for significant performance gains.
4
![Page 5: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/5.jpg)
5Click to edit Master title style
Where to Gain Performance?
• Cache and memory latency on a hypothetical modern server• L1 Cache - 1 machine cycle• L2 Cache - 4 machine cycles• L3 Cache - variable 10‘s of machine cycles• L4 Cache - variable 100‘s of machine cycles• Real Memory - 1000 machine cycles• Disk - depends on device type 1,000,000 - 10,000,000+ cycles
A baby stepvs.
a walk around the
equator
5
![Page 6: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/6.jpg)
6Click to edit Master title styleEnterprise Machines with Large Main Memories
• zEC12 and z196’s support up to 3T of real storage with a limit of 1T per LPAR (z10 1.5T and 1T per LPAR)
• This is expected to increase significantly in future generation machines.• Memory is a one-time charge item that can provide cost savings across
the stack.
Terabytes of available storage
6
![Page 7: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/7.jpg)
7Click to edit Master title style
Can DB2 Exploit Memory?
◄ 64-bit End of Virtual“Above the Bar”
◄ 32-bit Bar“Below the Bar”
◄ 16 MB “Line”
Afterv7
v7 andprior
DB2 10 sets the stage for vast virtual space exploitation
7
![Page 8: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/8.jpg)
Click to edit Master title style
Are Vast Virtual Spaces Safe?
“Above the Bar” 16 exabytes
Real and Auxx Gigabytes
Virtual Object128G
Performance Degrades as Real Storage is ConsumedOnce all aux is consumed, the LPAR goes into a wait state
![Page 9: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/9.jpg)
9Click to edit Master title style
Nothing to Fear - Real Storage Management
• New system configuration parameters with apar PM24723/PM49816 for DB2 10 to define real storage boundaries and how aggressively DB2 should strive to remain in the defined boundary (Note: z/OS apar OA35885/OA37821 are required to enable the full functionality).
• ZPARM REALSTORAGE_MAX defines the sandbox• Amount of real and aux in GB a given DB2 subsystem is allowed to
consume
• The frequency of contraction mode can also be controlled by system parameter REALSTORAGE_MANAGEMENT. ZPARM REALSTORAGE_MANAGEMENT defines the behavior within the sandbox.
• REALSTORAGE_MANAGEMENT options include:• OFF - only regulate when critical condition is detected• ON - operate with the smallest real footprint possible
AUTO (the default) - contract real footprint when paging (Note: new apar PM88804 reduces discards in AUTO or OFF mode)
9
![Page 10: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/10.jpg)
10Click to edit Master title styleWhat Does Real Storage Contraction Mean?
“Above the Bar”
16 exabytes
Real and Auxx Gigabytes
Virtual Object128G Thread logically
frees pages sothey are just cached
Upon paging, DB2 discards
pages. They are still valid
but will be in first reference.
The real storage footprint is
gone!
10
![Page 11: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/11.jpg)
11Click to edit Master title style
DB2 Exploitation
• Buffer Pools• High Performance DBAT and RELEASE(DEALLOCATE)• EDM• Other Exploiters
11
![Page 12: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/12.jpg)
12Click to edit Master title style
Buffer Pools
• DB2 buffer pools serve as a data cache for DB2 objects• Each buffer represents a page from a given object. Buffers come in 4K,
8K, 16K, and 32K page sizes (not 1M! explanation is forthcoming)• The key to buffer pool performance is to retain buffers that are re-
referenced. If a workload does not re-reference data, buffering will be of limited or no benefit.
• Buffer pool tuning is beyond the scope of this talk, but a basic tuning strategy is to increase the buffer pool size to buffer more data.• Large buffer pools today are in the 10’s of G range• All in service releases of DB2 support a total of 1T of buffer pool
space (for smaller installations it is limited to 2x amount of real available to the LPAR)
12
![Page 13: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/13.jpg)
13Click to edit Master title style
What if We Had Huge Buffer Pools?
• DB2 control structures and algorithms need analysis for scalability• CF considerations -- how about the GBPs?• May reduce the need for micro-management tuning
13
![Page 14: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/14.jpg)
14Click to edit Master title styleOther Means to Exploit Memory with Buffer Pools
• In Memory Objects• DB2 10 supports PGSTEAL(NONE)• All objects on first reference are read completely into the buffer pool• Object needs to fit else performance problems may ensue• As buffer pools get bigger with large main memories, this can be more easily
exploited• Page Fixing
• Page fixing is not new, but is well suited for a large memory environment• When a buffer is read or written to disk, it must be fixed in real memory• Long term page fixing the buffer pool requires sufficient memory to fix all
frames. This is a good opportunity for performance gains by eliminating the need to fix and unfix frames.
• Future thoughts are that it will be standard to page fix buffer pools (currently required for large frame exploitation)
14
![Page 15: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/15.jpg)
15Click to edit Master title styleWhat About Large Page Sizes and Buffer Pools?• z10’s support 1M page sizes and zEC12’s introduce 2G page sizes• What is this all about? Answers 1 and 2 are:
“This has nothing to do with data page sizes!”
Answer 3 is:
“This relates solely to the computing environment on these machines. It is merely an optimization when translating a virtual address into a real
address”
15
![Page 16: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/16.jpg)
16Click to edit Master title style
What About Large Page Sizes and Buffer Pools?
Program refers to virtual address 0x5F001800
Look in processor cache TLBfor real frame address. In this case
we look for page entry 0x5F001000
Hit
Get 4K page in memory
Miss Resolve via page table structure
TLB entries will represent a full 1M range. We will be
looking for 0x5F000000
More efficient page table lookup with
entries representing a full 1M. We will be
looking for 0x5F000000
Will get 1M page in memory. DB2
exploits only fixed pages so it will be in
memory already!
16
![Page 17: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/17.jpg)
17Click to edit Master title styleWhat About Large Page Sizes and Buffer Pools?• Exploitation Summary
• DB2 10 support fixed 1M frames• DB2 11 plans to support fixed 2G frames• Enabled via LFAREA in IEASYSxx• Observed 1-4% with DB2 10 and 1M buffer pools
• Futures?• Possible exploitation of 1M pageable frames• Page fixed large frame buffer pools will be common?• Larger frame sizes?
17
![Page 18: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/18.jpg)
18Click to edit Master title style
DB2 Exploitation
• Buffer pools• High Performance DBAT and RELEASE(DEALLOCATE)• EDM• Other Exploiters
18
![Page 19: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/19.jpg)
19Click to edit Master title style
RELEASE(DEALLOCATE)
• RELEASE is a BIND option that tell DB2 how to handle the caching of package locks and structures. The default is COMMIT and DB2 will free package structure upon commit. The alternative is DEALLOCATE where package structures will persist until full thread deallocation.
• This can be an opportunity for performance for:• Long running batch jobs that COMMIT frequently• Thread reuse (e.g., CICS protected threads, JCC T2)
• Why isn’t RELEASE(DEALLOCATE) used pervasively?• Virtual storage constraints (mainly v9 and earlier)• DDF workload• The need to break in for BIND/DDL activity
19
![Page 20: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/20.jpg)
20Click to edit Master title style
High Performance DBAT
• Re-introducing RELEASE(DEALLOCATE) in distributed packages – Could not break in to do DDL, BIND– V6 PQ63185 to disable RELEASE(DEALLOACTE) on DRDA DBATs
• High Performance DBATs reduce CPU consumption by– RELEASE(DEALLOCATE) to avoid repeated package allocation/deallocation– Avoids processing to go inactive and then back to active– Bigger CPU reduction for short transactions
• Using High Performance DBATs – Stay active if there is at least one RELEASE(DEALLOCATE) package exists– Connections will turn inactive after 200 times (not changeable) to free up DBAT – Normal idle thread time-out detection will be applied to these DBATs– Good match with JCC packages – Not for KEEPDYNAMIC YES users
20
![Page 21: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/21.jpg)
21Click to edit Master title style
Can I Break In?
• v10 DDF Threads:- To enable
- Rebind packages with RELEASE(DEALLOCATE)- -MODIFY DDF PKGREL(BINDOPT)
- To disable- Wait 200 commits- -MODIFY DDF PKGREL(COMMIT) and this only effects newly
allocated DBATs!- v10 Other Threads:
- No solution
21
![Page 22: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/22.jpg)
22Click to edit Master title style
Can I Break In...Continued?
• v11 DDF Threads:- To enable same as v11- To disable
- Automatically done on next COMMIT if waiter on a package lock- v11 Other Threads:
- Automatically done on next COMMIT if waiter on a package lock- Idle threads? Coming soon!
22
![Page 23: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/23.jpg)
23Click to edit Master title style
DB2 Exploitation
• Buffer pools• High Performance DBAT and RELEASE(DEALLOCATE)• EDM• Other Exploiters
23
![Page 24: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/24.jpg)
24Click to edit Master title style
EDM
• The EDM is another cache repository for DB2 storage. It caches dynamic statements, DBDs, and skeleton packages/plans.
• Controlled by a number of ZPARMs EDMDBDC, EDMPOOL, EDMSTMTC, EDM_SKELETON_POOL
• Logically the bigger the pool, this great the chance for a cache hit• Less I/O to load packages• Fewer full prepares• Less chance of EDM full
• The max size for these parms in v10 is 2G. In v11 these will be increased to 4G.
• Is the future simply bigger is better?• We may need to restructure some of the internals to properly scale higher• Page fixing to keep critical cache in memory?• Large page exploitation for TLB access benefits?
24
![Page 25: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/25.jpg)
25Click to edit Master title style
DB2 Exploitation
• Buffer pools• High Performance DBAT and RELEASE(DEALLOCATE)• EDM• Other Exploiters
25
![Page 26: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/26.jpg)
26Click to edit Master title style
Other Memory Exploiters
• v11 Storage Awareness• Parallelism
- When spawning child tasks, runtime will evaluate the storage environment. If we are paging or cannot allocate many new agents due to ECSA constraints, we will cut the amount of parallelism.
• Sort- Will evaluate the storage environment to determine if an in-
memory sort can be conducted.• Sparse Index
- When attempting to allocate MXDTCACH storage, reduce the request if storage environment is not conducive to the request.
26
![Page 27: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/27.jpg)
27Click to edit Master title style
Other Memory Exploiters
• Storage Awareness Futures• Current strategies are more green/red in nature -- “Everything looks
OK, go for it!” Probably fall apart with high degree if parallel inquiries.
• Developing strategies to determine the size of available “slush”storage and give it to threads in controlled fashion
• Reduction in ZPARM settings to control storage usage (e.g., MXDTCACH)
vs.
27
![Page 28: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/28.jpg)
28Click to edit Master title style
Futures TBD?
• Large page support for all of DB2• RELEASE(DEALLOCATE) only limited by appropriateness of the option
and not by real or virtual storage constraints• Huge buffer pools with many objects completely in the pool• EDM micro-managing disappears• Other ZPARMs controlling storage sizes disappear (probably not
REALSTORAGE_MAX)• In memory data-structures? DB2 will likely never be an “in-memory”
database due to the size of the data we handle but hybrid solutions may be available for exploitation.
28
![Page 29: The Present and Future of Large Memory in DB2• ZPARM REALSTORAGE_MAX defines the sandbox • Amount of real and aux in GB a given DB2 subsystem is allowed to consume • The frequency](https://reader034.vdocument.in/reader034/viewer/2022042116/5e947debefa241707808b5dc/html5/thumbnails/29.jpg)
29Click to edit Master title style
• Thanks!• [email protected]
• Session A11• Wednesday October 16, 2013 11:00AM - 12:00PM
29