04 exadata storage configuration

SAGE Configuration - Beta TrainingAgenda
- * -
- * -
Exadata Server Initialization
Initial Network Preparation
First Boot Sequence
Performance Sanity Test
- * -
Initial Network Preparation
1 for Embedded Ethernet Port
1 for LO100c Remote Management Port
© 2008 Oracle Corporation – Proprietary and Confidential
Oracle Database 11g: SAGE 0 - *
- * -
Exadata Server First Boot
Power on Exadata Server
Configure Remote Management (LO100c).
Answer First Boot Questions:
Timezone
Change default passwords
Root, celladmin, cellmonitor
- * -
Lights Out (LO100c) Setup
Configure Lights Out Remote Management
The slide shows you how to configure static IP addresses for lights out (LO) remote management on an Exadata Cell:
1. Press F10 on your console during the power-on self test (POST) to enter the BIOS Setup Utility.
2. Press the right arrow to go to the Advanced menu.
3. Press the down arrow to scroll to IPMI section.
4. Press Enter to get to the IPMI page.
5. Perform the following to set the network BIOS settings on the cell:
Press the down arrow to scroll to the LAN Configuration part of the page near the bottom.
Select Static or Disable on DHCP IP Source entry depending on your DHCP configuration. To select an option on a field, press enter. A popup window appears from where you can scroll to the right value. Press enter to select your choice.
Press the down arrow to scroll down, and enter a valid IP address, subnet mask, and gateway address on corresponding entries. Press Tab or period (.) to move between address fields.
- * -
First Boot Sequence
Nameserver: mynameserv.company.com
Select country by number, [n]ext, [l]ast:228
Select zone by number, [n]ext: 20
Selected timezone: America/Los_Angeles
The current ntp server(s): 0.us.pool.ntp.org …
Do you want to change it (y/n) [n]: y
Primary (mandatory) ntp server: ntp1.company.com
Secondary (optional, but recommended) ntp server: ntp2.company.com
Tertiary ntp server:
Is it correct (y/n) [y]: y
<list of all available network interfaces> …
Select interface name to configure or press Enter to continue: eth0
Selected interface. Eth0
Continue configuring or re-configuring interfaces? (y/n) [y]: y
<list of all available network interfaces> …
Select interface name to configure or press enter to continue: bond0
Selected interface. bond0
IP address: 192.168.50.76
<list of all available network interfaces with modified information> …
Is this correct? (y/n): y
Answering questions during boot sequence
Once you configured the BIOS, you return to the boot sequence up to when the system asks you to provide information for the following:
Fully qualified hostname
One or more Domain Name Service (DNS) server IP addresses
Time preference: local time region number from the displayed list of available time regions, and location within the time region number from the displayed list of locations
Network Time Protocol (NTP) servers. These servers are required to maintain the time on the system correctly, and are synchronized to your local time source.
Ethernet and the InfiniBand IP addresses: A list of all Ethernet and InfiniBand interfaces that have a live network cable connection is displayed with the name of the corresponding interface. InfiniBand interface is named bond0 and uses bonding between physical InfiniBand interfaces ib0 and ib1. Bonding provides the ability to transparently fail over from ib0 to ib1 or from ib1 to ib0 if connectivity is lost to ib0 or ib1. If you choose not to configure each interface in the list, then that interface will not be configured, and it will not be started at system startup.
Note: After the configuration of the IP addresses, the system completes the startup process. At the end of the process, additional packages are installed and then the installation of Exadata Storage Server is complete.
- * -
Performance Sanity Test
Individual Disks
Disk Controller
- * -
Calibrate – CellCLI command
[root@cell01 ~]# cellcli
CellCLI: Release 11.1.3.0.0 - Production on Tue Oct 04 22:13:21 PDT 2008
Copyright (c) 2007, 2008, Oracle. All rights reserved.
Cell Efficiency ratio: 1.0
Aggregate random read throughput across all luns: 991.86 MBPS
Aggregate random read IOs per second (IOPS) across all luns: 3926
Controller read throughput: 1057.41 MBPS
Lun 6_1 on drive [1I:6_1:1 ] random read throughput: 98.81 MBPS, and 334 IOPS
Lun 6_10 on drive [1I:6_1:10] random read throughput: 97.38 MBPS, and 339 IOPS
CALIBRATE results are within an acceptable range.
Testing Performance Using CALIBRATE
The CALIBRATE command enables you to verify the disk performance before the cell is put online. CALIBRATE is a CellCLI command. You must execute this command while being logged as the root user at the operating system level.
On the above example, the results were acceptable compared to standard properties. A message alerts you if the performance measurements are substandard.
- * -
Creating a New Exadata Cell
[celladmin@cell01 ~]$ cellcli
CellCLI: Release 11.1.3.0.0 - Production on Tue Oct 04 22:13:21 PDT 2008
Copyright (c) 2007, 2008, Oracle. All rights reserved.
Cell Efficiency ratio: 1.0
CellCLI> CREATE CELL cell01 -
smtpToAddr='[email protected]', -
notificationPolicy='critical,warning,clear', -
notificationMethod='mail'
CellCLI>
- * -
- * -
Exadata Storage Allocation
- * -
Exadata System Area
First Two Disks
Software RAID 1
700MB iSCSI Metadata Disk for OCR, Voting Disks
Celldisk / Griddisk Metadata
Remaining Ten Disks
- * -
Create Celldisks
Default name is CD<id>_<cellname>
Idempotent operation – skips any existing celldisks
Can also create single celldisk at a time
- * -
Creating Cell Disks
- * -
Create Griddisks
Griddisk = ASM disk
Creates one griddisk per celldisk
Default Name is <prefix>_<celldiskname>
Size of each griddisk
Size is optional – will use all remaining space
Order of griddisk creation (hottest coldest)
- * -
Creating Grid Disks
GridDisk data_CD_1_cell01 successfully created
GridDisk recov_CD_1_cell01 successfully created
- * -
Exadata CellCLI Basic Commands
CREATE CELLDISK ALL
LIST [ LUN | CELLDISK | GRIDDISK ] [<name>] [DETAIL]
DROP [CELLDISK | GRIDDISK] ALL
- * -
- * -
ASM and Database Configuration
ASM instance initialization parameter:
- * -
ASM and Database Configuration
Create two new files at each ASM & Database node in
/etc/oracle/cell/network-config
cellinit.ora
cellip.ora
IP addresses of remote Exadata Servers InfiniBand interfaces
Needs to be identical on all nodes of the ASM cluster
- * -
ASM & Database Node Config Files
# mkdir -p /etc/oracle/cell/network-config
# chown oracle:dba /etc/oracle/cell/network-config
- * -
RAC – OCR and Voting Disks
Each Exadata Server exports an iSCSI Metadata Disk
700MB RAID1
Pick the same 3 Exadata Servers on all hosts
HP Oracle Database Machine
Specify local host name
Other Database Servers
- * -
iSCSI Initiator Script
[root@dscbac01 ~]# ./db.iscsi.sh
Are you setting this node as member of Oracle RAC (y/n) [n]: y
Configuring iSCSI drives for OCR and voting disks
Client name of iSCSI initiator (hostname of this Oracle Database server): dscbac01
Owner of ocrvot devices (Oracle Database owner usually user oracle) [oracle]:
Group of ocrvot devices (Oracle Database owner group, usually dba) [dba]:
Permissions of ocrvot devices (default 640) [0640]:
IP address of the Infiniband interface bond0 of the cell providing ocrvota: 192.168.50.209
IP address of the Infiniband interface bond0 of the cell providing ocrvotb: 192.168.50.210
IP address of the Infiniband interface bond0 of the cell providing ocrvotc: 192.168.50.211
Client name of iSCSI initiator (hostname of this Oracle Database server): dscbac01
IP address of the Infiniband interface bond0 of the cell providing ocrvota: 192.168.50.209
IP address of the Infiniband interface bond0 of the cell providing ocrvotb: 192.168.50.210
IP address of the Infiniband interface bond0 of the cell providing ocrvotc: 192.168.50.211
Is this correct (y/n) [y]:
…
…
Add ip-192.168.50.209:3260-iscsi-iqn.2001-04.com.dscbac01s.us.oracle.com-disk0 to the persistent storage
- * -
ASM Discovery and Diskgroups
Can specify NULL discovery string (default)
format is “o/<exadata_connect_info>/<griddisk_name>”
<exadata_connect_info> format in cellip.ora
One failure group per Exadata Server
New parameters for create diskgroup
‘cell.smart_scan_capable’=‘TRUE’
‘compatible.rdbms’=‘11.1.0.7.0’
‘compatible.asm’=‘11.1.0.7.0’
- * -
Exadata Only ASM Diskgroup
'o/*/data*'
…
…
- * -
- * -
Database / Application Changes
Transparent to SQL, users, applications
SQL Offload processing only
Tablespace in Exadata only diskgroup
- * -
Exadata Server Configuration Summary
Exadata Server First Boot
New configuration files (cellinit.ora, cellip.ora)
New diskgroup parameters
Database & Application Changes
- * -
A
Q

04 exadata storage configuration

Documents