oracle on aix - configuration and tuning

IBM Oracle Solutions Advanced Technical Support - Americas

Oracle on AIX Workshop © 2006 IBM Corporation

Oracle on AIX – Configuration & Tuning

R. Ballough, IBM Advanced Technical Support

2

IBM Oracle Solutions - Advanced Technical Support - Americas

Oracle for AIX Workshop © 2006 IBM Corporation

Legal information

The information in this presentation is provided by IBM on an "AS IS" basis without any warranty, guarantee or assurance of any kind. IBM also does not provide any warranty, guarantee or assurance that the information in this paper is free from errors or omissions. Information is believed to be accurate as of the date of publication. You should check with the appropriate vendor to obtain current product information.

Any proposed use of claims in this presentation outside of the United States must be reviewed by local IBM country counsel prior to such use.

IBM, ̂ , and pSeries are trademarks or registered trademarks of the International Business Machines Corporation.

Oracle and Oracle9i are trademarks or registered trademarks of Oracle Corporation.

All other products or company names are used for identification purposes only, and may be trademarks of their respective owners.

3



Basic AIX Configuration/Tuning for Oracle– Memory

– CPU

– I/O

– Network

– Miscellaneous

RAC ConfigurationRAC Tuning

Agenda

4



The suggestions presented here are considered to be basic configuration “starting points” for general Oracle workloads

Customer workloads will vary

Ongoing performance monitoring and tuning is recommended to ensure that the configuration is optimal for the particular workload characteristics

AIX Configuration for Oracle “starting points”

5



Oracle Server Architecture – Memory Structures

System Global Area (SGA)Program Global Area (PGA)

CHKP

PMON SMON

DBWR

LGWR

ARC0

D000

Redo Log Buffer

Online Redo Logs

ArchiveLogs

DatabaseFiles

Control FilesUser

Private SQL Area

DatabaseBuffer Cache

SharedPool

IBM Advanced Technical Support - Americas

© 2003 IBM Corporation6 © 2005 IBM Corporation

The AIX “vmo” command provides for the display and/or update of several parameters which influence the way AIX manages physical memory

– The “-a” option displays current parameter settingsvmo –a

– The “-o” option is used to change parameter values vmo –o minfree=1440

– The “-p” option is used to make changes persist across a reboot

vmo –p –o minfree=1440

Memory Tools - Virtual Memory Manager (VMM)

A number of the default “vmo” settings are not optimized for database workloads and should be modified for Oracle environments

7



General Memory Tuning

Memory Use testsys 8/2/2005

0102030405060708090

100

8:03

8:04

8:05

8:06

8:07

8:08

8:09

8:10

8:11

8:12

8:13

8:14

8:15

8:16

8:17

8:18

8:19

8:20

8:21

8:22

8:23

8:24

8:25

8:26

8:27

8:28

8:29

8:30

8:31

8:32

8:33

8:34

8:35

%comp %file

8



VMM Tuning Pre AIX 5.2 ML4

MINPERM% – minimum % real memory for fs buffer cache

15-20%: JFS or JFS2 filesystems without DIO or CIO5%: RAW logical volumes

JFS or JFS2 with DIO or CIOGPFS

MAXPERM%, MAXCLIENT% - max % real memory for fs buffer cache

40-60%: JFS or JFS2 filesystems without DIO or CIO<= 20%: Raw logical volumes

JFS or JFS2 with DIO or CIOGPFS

Never more than 20 GB prior to AIX 5.3To start, set to vmtune "numperm" valueReduce until vmstat freed (fr) to scanned (sr) ratio is 4:1

9



VMM Tuning – AIX 5.2ML4+

MINPERM% =5%

MAXPERM%, MAXCLIENT%=80% or highermake this a threshold which is > (1-computational memory)

LRU_FILE_REPAGE=0LRU_POLL_INTERVAL=10ms

LRU_FILE_REPAGE=0 is a “hint” to lrud to ignore repage rates when determining what to page out – effectively favoring paging out file pages (filesystem buffer cache) rather than computational pages

LRU_POLL_INTERVAL indicates the time period after which LRUD pauses and interrupts can be serviced. Default value of “0” means no preemption.

STRICT_MAXPERM=0 (default) STRICT_MAXCLIENT=1 (default)

10



Understanding Memory Pools

Memory pools are configured at boot time based on physical hardware configurationNumber can be seen with ‘vmstat –v |grep pools’Size can only be seen using KDB

LRUD operates per memory pool

SGA is allocated equally from memory pools

If free list has been depleted in a memory pool, LRUD will scan/reclaim memory in that pool

Consider implementing one of the following: Setting memory_affinity=0 ignores physical hardware configuration and

allocates evenly sized memory poolsAIX 5.3 ML3 with APAR IY69237 modifies VMM to consider all free frames in

system regardless of memory pool before reclaiming memory

11



VMM Page Stealing ThresholdsThe following define thresholds for the VMM page stealing process (lrud):

minfreeSet minfree = 120 x # logical CPUs /#mem poolsConsider increasing if vmstat “fre” column frequently approaches zero or if

“vmstat –s” shows significant “free frame waits”

maxfreeSet maxfree = minfree + (MAX(maxpgahead, j2_maxPageReadAhead) * #

logical CPUs)/ # mem pools

Example:For a 6-way LPAR with SMT enabled, maxpgahead=8 and j2_maxPageReadAhead=8:minfree = 1440 = 120 x 6 x 2maxfree = 1536 = 1440 + (max(8,8) x 6 x 2)

vmo –o minfree=1440 –o maxfree=1536 -p

12



General AIX memory tuningNo requirements for allocating SGA

SGA may be pinned on AIX, but IS NOT RECOMMENDED – we have better ways of keeping sga resident in memory

Do not over commit real memory!!!!Server should be configured with enough physical memory to satisfy memory

requirements

Paging spaceWith AIX demand paging, paging space does not have to be large½ memory + 4GB

Monitor paging activity:vmstat -ssar -rnmon

Resolve paging issues:Reduce Filesystem cache size (MAXPERM, MAXCLIENT)Reduce Oracle SGA or PGA (9i or later) sizeAdd physical memory



Changing SGA size

MemoryOracle 9i or 10g– SGA can be dynamically resized, but has an upper bound by the parameter

SGA_MAX_SIZE. • SQL> alter system set db_cache_size=2048m scope=both;• (

SGA_TARGET (10g)• DB_CACHE_SIZE, SHARED_POOL_SIZE., etc.

– PGA_AGGREGATE_TARGET can be dynamically resized

– SGA_TARGET and PGA_AGGREGATE_TARGET are not hard limits

Something to keep in mind for use of DLPAR with memory…..



Determining SGA size

SGA Memory Summary for DB: test01 Instance: test01 Snaps: 1046 -1047

SGA regions Size in Bytes

------------------------------ ----------------

Database Buffers 16,928,210,944

Fixed Size 768,448

Redo Buffers 2,371,584

Variable Size 1,241,513,984

----------------

sum 18,172,864,960

SQLPLUS:SQL> show sga

SQL> show parameters

Statspack:

15




– CPU

– I/O

– Network

– Miscellaneous

Agenda



Oracle parameters based on # CPUs

– DB_WRITER_PROCESSES

– Degree of Parallelism

– - table level

– - query level

– - PARALLEL_MAX_SERVERS or AUTOMATIC_PARALLEL_TUNING (CPU_COUNT * PARALLEL_THREADS_PER_CPU)

– CPU_COUNT

– FAST_START_PARALLEL_ROLLBACK – should be using UNDO instead

– CBO – execution plan may be affected; check explain plan



CPU ConsiderationsUse SMT with AIX 5.3/Power5 environments

Micropartitioning considerations- Virtual cpus)<= physical processors in shared pool- CAPPED

- Virtual CPUs should be the nearest integer >= capping limit- UNCAPPED- Virtual CPUS should be set to the max peak demand requirement

DLPAR considerationsOracle 9i– Oracle CPU count does not recognize change in # cpus– AIX scheduler can still use the added CPUsOracle 10g– Oracle CPU count recognizes change in # cpus

18




– CPU

– I/O

– Network

– Miscellaneous

Agenda

19



Oracle Server Architecture - Files

System Global Area (SGA)Program Global Area (PGA)

CHKP

PMON SMON

DBWR

LGWR

ARC0

D000

Redo Log Buffer

Online Redo Logs

ArchiveLogs

DatabaseFiles

Control FilesUser

Private SQL Area

DatabaseBuffer Cache

SharedPool

20



Flashback database design



Options for storing Oracle data files

Filesystems– Single-instance:

– JFS, JFS2, Veritas VxFS

– Clustered:– GPFS, Veritas CFS

Raw

Automatic Storage Management (ASM) – new in 10g



Data Layout for Optimal I/O Performance

Stripe and mirror everything (SAME) approach:

Goal is to balance I/O activity across all disks, loops, adapters, etc...

Avoid/Eliminate I/O hotspots

Manual file-by-file data placement is time consuming, resource intensive and iterative

Use RAID-5 or RAID-10 to create striped LUNs (hdisks)

Create AIX Volume Group(s) (VG) w/ LUNs from multiple arrays, striping on the front end as well for maximum distribution

Physical Partition Spreading (mklv –e x) –or-

Large Grained LVM striping (>= 1MB stripe size)

http://www-1.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100319



Data Layout cont’d…Stripe using Logical Volume (LV) or Physical Partition (PP) striping

LV Striping– Oracle recommends stripe width of a multiple of

Db_block_size * db_file_multiblock_read_countUsually around 1 MB

– Valid LV Strip sizes:AIX 5.2: 4k, 8k, 16k, 32k, 64k, 128k, 256k, 512k, 1 MBAIX 5.3: AIX 5.2 Strip sizes + 2M, 4M, 16 MB, 32M, 64M, 128M

– Use AIX Logical Volume 0 offset (9i Release 2 or later) • Use Scalable Volume Groups (VGs), or use “mklv –T O” with Big VGs• Requires AIX APAR IY36656 and Oracle patch (bug 2620053)

PP Striping– Use minimum Physical Partition (PP) size (mklv -t, -s parms)

Spread AIX Logical Volume (LV) PPs across multiple hdisksin VG

• (mklv –e x)

24



Single Instance Environments - Filesystems

Filesystems

JFS – no longer being enhancedBetter for lots of small file creates & deletes

JFS2 – generally the preferred single-instance filesystemBetter for large files/filesystems

Mount options:Buffer Caching (default)– stage data in fs buffer cacheDirect I/O (DIO)– no caching on readsConcurrent I/O (CIO) – DIO + no write lock (JFS2 only)Release Behind Read (RBR)– During sequential reads, memory pages released after

pages copied to internal buffersRelease Behind Write (RBW) – During sequential writes, memory pages released after

pages written to disk

In 9i, DIO and CIO must be specified at the filesystem level

In 10g, Oracle issues o_cio and o_dio calls as appropriate

25



Single-instance environments Cached vs. non-Cached (Direct) I/O

File System caching tends to benefit heavily sequential workloads with low write content. To enable caching for JFS/JFS2:

Use default filesystem mount options

Set Oracle filesystemio_options=ASYNCH

DIO tends to benefit heavily random access workloads and CIO tends to benefit heavy update workloads. To disable JFS, JFS2 caching:

In 9i, set filesystemio_options=SETALL and use dio or ciomount option

In 10g, set filesystemio_options=SETALL

When using DIO/CIO, fs buffer cache isn’t used. Consider the following db changes:

Increase db_cache_size

Increase db_file_multiblock_read_count

Read Metalink Note #272520.1



Single-instance environments - Oracle Database File AccessData Base Files (DBF)

I/O size is db_block_size or db_block_size * db_file_multiblock_read_countUse CIO or no mount options for extremely sequential I/OIf block size is >=4096, use a filesystem block size of 4096, else use 2048

Redo Log/Control FilesI/O size is always a multiple of 512 bytesUse CIO or DIO and set filesystem block size to 512

Archive Log FilesDo not use CIO or DIO‘rbrw’ mount option can be advantageous

Oracle BinariesDo not use CIO or DIO

27



Single-instance environments - I/O Tuning (ioo)READ-AHEAD (Only applicable to JFS/JFS2 with caching enabled)

MINPGAHEAD (JFS) or j2_minPageReadAhead (JFS2)Default: 2Starting value: MAX(2,DB_BLOCK_SIZE / 4096)

MAXPGAHEAD (JFS) or j2_maxPageReadAhead (JFS2)Default: 8 (JFS), 128 (JFS2)Set equal to (or multiple of) size of largest Oracle I/O request

DB_BLOCK_SIZE * DB_FILE_MULTI_BLOCK_READ_COUNT

Number of buffer structures per filesystem:

NUMFSBUFS (JFS2): Default: 196, Starting Value: 568Monitor with “vmstat –v”, increase if value of “filesystem I/Os blocked with no

fsbuf” is increasingj2_dynamicBufferPreallocationDefault: 16Monitor with “vmstat –v”, increase if value of “external pager filesystem I/Os

blocked with no fsbuf “is increasing

28



Asynchronous I/O for filesystem environmentsAIX parameters

minservers = 10 * # cpusmaxservers = (10 * # disks) / # cpusmaxreqs = a multiple of 4096 > 4 * #disks * queue_depth“enable” at system restartTypical settings: minservers=100, maxservers=200, maxreqs=16384

Oracle parametersdisk_asynch_io = TRUEfilesystemio_options = {ASYNCH | SETALL}db_writer_processes = default

Monitor usage:Watch alert.log for errors:

Warning “lio_listo returned EAGAIN”Monitor from AIX

“pstat –a | grep aios”Use “-A” option for NMONIostat –Aq (new in AIX 5.3)

29



ASM configurations

AIX parameters– Async I/O needs to be enabled, but default values may be used

ASM instance parameters– ASM_POWER_LIMIT=1

Makes ASM rebalancing a low-priority operation. May be changed dynamically. It is common to set this value to 0, then increase to a higher value during maintenance windows

– PROCESSES=25+ 15n, where n=# of instances using ASM

DB instance parameters– disk_asynch_io=TRUE– filesystemio_options=ASYNCH– Increase Processes by 16– Increase Large_Pool by 600k– Increase Shared_Pool by [(1M per 100GB of usable space) + 2M]

30




– CPU

– I/O

– Network

– MiscellaneousRAC ConfigurationRAC Tuning

Agenda

31



Network Parameters – all environments

Set sb_max >= 1 MB (1048576) (generally ok by default)

Set tcp_sendspace = 262144

Set tcp_recvspace = 262144

Set rfc1323=1

Also confirm these are set properly at network interface level

32



Miscellaneous parameters

/etc/security/limitsSet to “-1” for everything except core for Oracle user

Sys0 attribute maxuproc >= 4096

Environment variables:AIXTHREAD_SCOPE=SNUM_SPAREVP=1 (AIX 5.1 only)

Use a 64-bit kernel

33




– CPU

– I/O

– Network

– Miscellaneous

RAC Configuration/Tuning

Agenda

34



Full Cache FusionCache-to-cache data shipping Shared cache eliminates

slow I/OEnhanced IPC

Allows flexibleand transparentdeployment

Users

Shared Shared CacheCache

Drive and Exploit Industry Advances in Clustering

ClusteredDatabase Servers

Storage Area NetworkShared Disk Subsystem

Low Latency InterconnectVIA or Proprietary

High Speed Switch or Interconnect

Hub or Switch Fabric

Centralized Management Console

Cache FusionCache Fusion

RAC Architecture Summary

35



Logical Shared Disk Architecture

Each RAC node requires access to all shared disk:Physical Shared disk

typically Fiber Channel or SAN attached

Best Practices: 1) Use 2 or more HBAs using multipathing software for load balancing and path failover2) plan for fabric redundancy

N o d e 1

N e t w o r k I n t e r c o n n e c t

S h a r e dD is k

N o d e 2 N o d e 3

36



GPFS (http://publib.boulder.ibm.com/clresctr/library/gpfs_aix_faq.html)See metalink note 302806.1 for details

10gR2: GPFS 2.3.0.3+10gR1: GPFS 2.1, 2.2, 2.3.01+9i: GPFS 2.1, 2.2, 2.3.0.1+

Link above includes references to supported storage platforms

HACMP Raw Logical Volumes10g: 5.1, 5.2 – NOTE: HACMP 5.3 not certified with 10g9i: 4.4.x, 4.5, 5.1, 5.2, 5.3

Automatic Storage Management (10g only)Check with Storage Vendor for support/recommended configurationCheck with Internal operations groups for management implications

eg, OS level backups, scripts to manage disk space, etc.Required for Standard Edition RAC

VERITAS SFRAC (http://support.veritas.com)

Other Notes:ORACLE_HOME directories must be local unless using GPFS or SFRAC.10g REQUIRES separate LUNs for Voting Disk and OCR partitions unless a clustered filesystem or HACMP is usedUse separate directories for CRS, ASM, ORACLE_HOME

Information subject to change, and additional requirements may apply.

Storage Software Options for Oracle RAC:

37



Oracle Options for Data Storage

Voting Disk

OCR

Oracle Binaries

Archive Log Files

Control Files

Redo Log Files

Data Base Files

ASMGPFSRAW

38



Clustering Software Requirements for RAC on pSeries:

Oracle 9i RAC:HACMP - 5.3 recommendedVeritas SFRAC 4.0

Other:PSSP 3.5 is required for use of SP switch and can replace HACMP as

clusterware

Oracle 10g RAC:Oracle CRS provides basic clusterware capabilityHACMP required for raw logical volume environments only - 5.2 currently

most recent certified versionVeritas SFRAC may be used if desired

Other:PSSP 3.5 is required for use of SP switch, but cannot function as

clusterware

Latest Requirements can be found at http://metalink.oracle.comCheck note # 282036.1 for detail

39



Oracle 10g R2: Clusterware

Provides same base functions as 10g R1 CRS

Extended to provide protection for 3rd Party Application components

– Employs Application Virtual IPs for transparent network connectivity

– May require Clustered File System support for Application Configuration Files

– Applications do not necessarily have to reside on DB node

40



Public Interface Requirements for RAC on pSeries:

Public interfaceAt least one network interface for client trafficMay use virtual ethernet if sufficient bandwidth – watch for 10.2.0.1 bug10g requires one virtual IP address for client traffic for each node

Validate that the application uses OCI calls and can take advantage of TAFJDBC Thick does not support use of OCI

TAF Supports failure/reconnect of failed connectionsOptional select statement failover

ORHardware load balancer/sprayerSoftware load balancerDNS CNAMES

41



RAC Interconnect Configuration

Gigabit Ethernet satisfactory for most applicationsIBM HPS offers lower latency/higher bandwidthIP over InfiniBand 10.2.0.1+UDP network tuning required for optimum performanceSwitched Networks REQUIRED – crossover cables not supportedVirtual Ethernet not yet supported

Oracle9i: Oracle “Fault Tolerant IP” (FTP-IP) feature provides Interconnect Network Fault Tolerance

Oracle10g: Etherchannel or 802.3ad Link Aggregation AIX options replace Oracle FTP-IP codeMay also be used with 9i CLUSTER_INTERCONNECTS parameter

N ode 1 N ode 2 N ode 3

P ublic N etw ork

P rivate N etw ork (B ackup)P rivate N etw ork (P rim ary) RAC Interconnect

42



Private Interconnect Requirements for RAC on pSeries:

Private interconnect

Oracle 9i RAC:HACMP/RAW solutions:

cluster interconnect: minimum of 1, recommend 2 gigabit EthernetHACMP/GPFS solutions:

cluster interconnect: same as aboveadditional: private subnet required for GPFS traffic; can be 10/100 ethernet

VERITAS SFRAC:cluster interconnect: recommended 3 Ethernet interfaces, minimum 2

Oracle 10g RAC:HACMP/RAW solutions

cluster interconnect: minimum of 1, recommend 2 gigabit Ethernet

GPFS solutions:cluster interconnect: same as aboveadditional: GPFS traffic can share with the cluster interconnect!ASM:

cluster interconnect: same as above

43



RAC Interconnect: 9i R2 (Default Configuration)

Private Networks (normally 2) defined to HACMP

At instance startup, Oracle searches HACMP Network Topology to identify up to 3 eligible networks for Interconnect use

– Normally, the 2 private networks, plus 1 add’l public one

Networks are operated in High Availability “failover”(primary/backup/backup) mode via Oracle provided “Fault Tolerant IP” code

Oracle “Fault Tolerant IP” code registers DB connections with HACMP Event Manager (haemd) to assist with network failover



RAC Interconnect: 9i R2 (Default Configuration)

en0en1en2en3

en0en1en2en3

Switch1Switch2

Public

Private

Node 1 Node 2

Oracle

AIX/HACMP

FT-IP

• Fault Tolerant

• Primary/Backup

• No Bandwidth Aggregation

Oracle

AIX/HACMP

FT-IP

10.0.0.1

10.0.0.2 10.0.0.4

10.0.0.3

45



10g RAC Interconnect Configuration

Best practice: Use 2 Gigabit Ethernet interfaces for interconnect

Oracle9i “Fault Tolerant IP” feature not available in 10gEtherchannel and 802.3ad Link Aggregation recommended

Network interface names must be the same on all nodes – eg, if en0 is a cluster interconnect, it must be a cluster interconnect on all nodes.

N ode 1 N ode 2 N ode 3

P ublic N etw ork

P rivate N etw ork (B ackup)P rivate N etw ork (P rim ary) RAC Interconnect

46



RAC Interconnect Configuration: 10g

HACMP Network Topology no longer used

Oracle “Fault Tolerant IP” code has been removed

Public and Private (RAC Interconnect) network names are specified at cluster configuration time

– Public interface names (e.g. en0) MUST be the same on all nodes

– Private interface names (e.g. en6) SHOULD be the same on all nodes

– Private networks should be non-routable

Fault Tolerance provided by either:– Logical Network with Primary/Backup adapters (9iR2 Alternative 2)

– EtherChannel or 802.3ad Link Aggregation

Requires protocol capable switches

Supports bandwidth aggregation and load balancing



RAC Interconnect: 10g (Bandwidth Aggregation/Backup)

Oracle

en0en1en2en3

en0en1en2en3

Switch1

Public

Private

Node 1 Node 2

AIX/TCP-IP Fault Tolerant

Load Balanced or Active/Standby

en6

Oracle

AIX/TCP-IPen6

• Fault Tolerant

• Load Balanced

• Bandwidth Aggregation

Switch2

en4

en410.0.0.5 10.0.0.6



Additional Network Parameters for RAC:Set udp_sendspace = db_block_size * db_file_multiblock_read_count +4k

• (not less than 65536)

Set udp_recvspace = 10 * udp_sendspace– Must be < sb_max

– Increase if buffer overflows occur

Ipqmaxlen=512 for GPFS environments

Use Jumbo Frames if supported at the switch layer

Time synchronization – use the “-x” flag with xntpd

Examples:no -a |grep udp_sendspaceno –o -p udp_sendspace=65536netstat -s |grep "socket buffer overflows"

49



GPFS tunablesSee metalink note 302806.1

Async I/O:Oracle parameter filesystemio_options is ignoredSet Oracle parameter disk_asynch_io=TRUEWorker1threads = GPFS asynch I/OPrefetchthreads= exactly what the name saysWorker1threads +prefetchThreads<=550Usually set prefetchthreads=default (64) and worker1threads=550-prefetchthreadsSet aio maxrequests=(worker1threads/#cpus) + 10

Other settings:GPFS block size is configurable; most will use 512KB-1MBPagepool – GPFS fs buffer cache, not used for RAC but may be for binaries. Default=64M

mmchconfig pagepool=100MMaxMBpS = maximum I/O that GPFS can submit per second. Default=150MBpS, should be set to approximate capacity of I/O subsystem

Ipqmaxlen=512No –r –o ipqmaxlen=512



GPFS & Oracle – best practices

Have < 10 filesystems

Because of different usage patterns, use separate filesystems for binaries than Oracle data files

Use local filesystems for CRS HOME

When I/O to GPFS filesystems outside of Oracle instance I/O is expected, it can be desirable to increase the pagerpool parameter

OCR, vote placement is recommended on raw luns, rather than as GPFS files



Scalability: OLTP Environments Scale-out tends to be good for OLTP environments when:

– There is low to moderate update activity– The workload is relatively uniform and predictable– The application is well designed and there are minimal lock/latch or serialization

related contention issues– A functional partitioning strategy is used to direct users to a limited number of

nodes–However, functional partitioning may reduce load balancing effectiveness

Industry benchmark proof points are limited (as of 11/1/05):– 1 out of 178 TPC-C benchmarks used RAC (16-node HP Integrity rx5670)

Relatively poor scale-out vs. non-RAC result on same hardwareNo demonstrable $/tpmC advantage vs. IBM p595 non-RAC results

– Limited number of SAP, Oracle E-Business Suite, PeopleSoft, other results

Most customer RAC environments are 2 or 3 nodes

A deployment involving a small number (2 or 3) of large nodes carries significantly less business risk than one involving many (4+) small nodes



Scalability: DSS Environments Scale-out tends to be good for Decision Support environments when:

– There is minimal data update or DDL operations during peak shift– Data is predominantly in large, partitioned tables which have even key/data

distribution– Data has a low “locality of reference” (low buffer cache hit%)– Are used predominantly for large query with Parallel Query Option (PQO) and

PARALLEL_AUTOMATIC_TUNING=TRUE– Query response time is not critical

Oracle focus on RAC based TPC-H benchmarks (as of 11/1/05):– 8 of 17 clustered results (all DB) were with Oracle RAC– 8 of 23 Oracle DB results were with Oracle RAC

Small number of known Oracle accounts > 1TB and/or involving more than 6 nodes

If you want to scale-out beyond 4-nodes, plan to do comprehensive stress testing before production deployment

53



Oracle DocumentationOracle Reference Manuals:

http://otn.oracle.com/documentation/index.html

Oracle Whitepapers:http://otn.oracle.com/products/database/clustering/RACWhitepapers.html

Oracle (Metalink) Certification Info:http://otn.oracle.com/support/metalink/index.html

Oracle Database 10g Release 2 Automatic Storage Management Overview and Technical Best Practices

http://www.oracle.com/technology/products/database/asm/pdf/asm_10gr2_bptwp_sept05.pdf

Oracle Metalink:http://metalink.oracle.com– 282036.1

IBM Redbooks: (www.ibm.com/redbooks)Techdocs – Technical Sales Library:

http://www.ibm.com/support/techdocs

54



GPFS Documentation

GPFS for AIX home page:www.ibm.com/servers/eserver/pseries/software/sp/gpfs.html

GPFS for AIX Library:www.ibm.com/servers/eserver/pseries/library/gpfs.html– Concepts, Planning and Installation Guide" (GA22-7453)– Administration and Programming Reference (SA22-7452)– Problem Determination Guide (GA22-7434)

Clusters Library:http://www.ibm.com/servers/eserver/clusters/library/– GPFS for AIX Frequently Asked Questions

Oracle Metalink:http://metalink.oracle.com– 302806.1: IBM General Parallel File System (GPFS) and Oracle RAC on AIX

5L and IBM eServer pSeries

55



Q U E S T I O N SQ U E S T I O N SA N S W E R SA N S W E R S

oracle on aix - configuration and tuning

Documents