ocfs users guide

OCFS

Oracle Cluster FileSystem for Linux

Users Guide

Table of Contents1 - Introduction......................................................................................42 - Obtaining OCFS................................................................................43 - Installing OCFS.................................................................................5

3.1 - Automatically mount OCFS during boot....................................53.2 - Configuring the OCFS................................................................5

3.2.1 - /etc/ocfs.conf file.................................................................53.2.2 - Automatic Configuration using ocfstool..............................63.2.3 - Manual configuration..........................................................6

3.3 - Loading ocfs...............................................................................73.4 - Formatting OCFS Partition........................................................7

3.4.1 - Format using ocfstool..........................................................73.4.2 - Format using mkfs.ocfs.......................................................7

3.5 - Mount OCFS partitions..............................................................93.5.1 - Mounting manually.............................................................93.5.2 - Mounting automatically......................................................9

4 - Tools..................................................................................................94.1 - ocfstool.......................................................................................9

4.1.1 - Format...............................................................................134.1.2 - Generate Config................................................................14

4.2 - Extfinder..................................................................................154.3 - Debugocfs................................................................................15

5 - Support Utilities.............................................................................185.1 - /etc/init.d/ocfs..........................................................................185.2 - /sbin/fsck.ocfs...........................................................................185.3 - /sbin/load_ocfs..........................................................................185.4 - /sbin/mkfs.ocfs.........................................................................185.5 - /sbin/mounted.ocfs...................................................................195.6 - /sbin/ocfs_uid_gen....................................................................19

6 - Best Practices.................................................................................197 - Frequently asked Questions & Answers.........................................20

Q. Can I install the Oracle Distribution in an OCFS partition?.......20Q. My partitions don't mount automatically during boot. What's wrong?..............................................................................................20Q. When running fsck.ocfs, it returns the error “WARNING: nonzero bytes after the disk header structure”. What does it mean?...........20Q. What are the most appropriate (recommended) tool IO and ca-pacity analysis of OCFS?..................................................................20

Q. Can I use ocfs on a NAS (Network Attached Storage) device like NetApp?............................................................................................20Q. Can I use LVM or MD to create my OCFS filesystem on top of it?.........................................................................................................20Q.I want to perform some tests using RAC/OCFS, but I don't want to spend too much money on hardware. Do I have any other solution?.........................................................................................................21Q.Why do I have to include the option _netdev in the fstab on Red-Hat?..................................................................................................21Q.What is the best way to archive the logs to an OCFS directory?.21Q.Is OCFS supported on 64 bit platform like Itanium?...................21Q. Can I run ocfs on non enterprise Linux distributions like RedHat 9?......................................................................................................21Q.Do I need any specific version of United Linux to run OCFS?.....22Q. Can I run the latest OCFS on a plain RedHat AS 2.1, without any errata applied?.................................................................................22Q.What is the advantage of running ocfs against raw devices?......22Q.What happen if I have to change the IP Address of my systems? 22Q.My Network Interface Card had to be replaced. Do I need to do something?.......................................................................................22Q.I have a partition that is not mounted. How do I know if it is an ocfs partition or not?........................................................................22Q.Can I use my OCFS partition to store regular files?....................23Q.How much do I lose in terms of performance compared to raw de-vices?................................................................................................23Q.How do I enable async I/O on Oracle using OCFS?.....................23Q.How do I backup my OCFS files? Can I use tar or other OS com-mand?...............................................................................................23Q.Is it possible to resize an existing OCFS partition?.....................23Q.I'm having problems with OCFS. How can I debug OCFS?.........23Q.Can I run OCFS in a stand-alone system? What are the advan-tages of running it?..........................................................................24Q.I have a database running on OCFS in a stand-alone node. Why it is so slow compared to other stand-alone systems running on ext3?.........................................................................................................24Q.How can I obtain more information about OCFS?.......................24Q.I have a customized RedHat AS kernel on my system. Does Oracle support OCFS and the RDBMS on it?..............................................24Q.How do I know if my OCFS version is officially supported by Oracle?.............................................................................................24

8 - Appendix A......................................................................................25

Introduction.

OCFS is a shared disk cluster filesystem. The current version (version 1) released for Linux is specifically de-signed to alleviate the need for managing raw devices. It can contain all the oracle datafiles, archive log files and controlfiles. It is however not designed as a gen-eral purpose filesystem.

This document describes the steps required to install OCFS on Linux and will also give guidelines for opti-mizations and some more in depth understanding of how the filesystem works.

Downloading OCFS.

OCFS can be downloaded from “http://oss.oracle.com/projects/ocfs” for the following distributions : RedHat Advanced Server 2.1 and United Linux 1.0 (Conectiva, SuSe, TuboLinux and SCO). Oracle officially supports the Oracle database on OCFS if it is installed from the binary packages that are avail-able for download.

If the user decides to download the source code and compile it, then there will be no formal support pro-vided by Oracle.

In addition to the OCFS binaries, we also provide a col-lection of utilities (cp, dd, tar and textutils) that en-able O_DIRECT. The updated tools are recommended to be used as they make more efficient use of the operat-ing system in conjunction with OCFS.

Binary distributions for ia32 and ia64 can be found un-der each one of the supported platforms.

There are basically three rpm packages to download in order to install OCFS. Those packages are:

•OCFS-Support

•OCFS-Tools

•OCFS Module

Before downloading the OCFS Module, make sure it is compatible with the kernel version in use (uname -a).

See “Appendix A” for a complete list of files and description of each one of the

http://oss.oracle.com/projects/ocfs

Installing OCFS

Installing OCFS is an easy process. After downloading the packages, issue the following command on the di-rectory where the packages were downloaded:

# rpm -Uhv ocfs*.rpm

This will install the support tools, the actual kernel module for the filesystem and a graphical configuration tool.

Automatically mount OCFS during boot.

After installing the OCFS packages, verify that the module will be properly initialized on startup using the command:

# chkconfig –list |grep -i ocfs

If the output looks like :ocfs 0:off 1:off 2:off 3:on 4:on 5:on 6:off

Then, no action is required, but if the output doesn't show “on” on 3,4 and 5 (rc levels), issue the following command to enable automatic startup of the ocfs dur-ing boot:

# chkconfig ocfs on

Configuring OCFS.

OCFS depends on a node specific configuration file. This file is named ocfs.conf and it is located in the /etc di-rectory; it can be generated automatically or manually using the ocfstool. During the next sections, the /etc/ocfs.conf file and both methods of configuring will be described in detail. This file is needed on every node in the cluster and it is highly recommended to use ocfs-tool to configure each node.

1.1.1 - /etc/ocfs.conf file.

The /etc/ocfs.conf file can have the following parame-ters:

ip_address – Specify the IP Address to be used by the OCFS DLM. The server must be able to reach all nodes

The rpm command in the right will install the OCFS packages if they are not installed yet and will upgrade them if they were

participating on the cluster through the interface re-lated to the IP Address specified in this field.

ip_port – Specify the port to be used by the OCFS DLM to communicate with the other nodes in the cluster. The port must be the same on all nodes in the cluster.

Node_name – Specify the server hostname associated to the IP Address specified in the ip_address parameter.

comm_voting – Specify which method for voting is go-ing to be used by OCFS. If set to 0 (default), it means that OCFS will be voting using the disk, if set to 1, it will be using the network to vote. If the OCFS is set to use network to vote and it becomes unavailable for some reason, it will automatically (and transparently) fall back to disk. Enabling comm_voting will drastically increase performance for regular filesystem operations such as rm, mv, mkdir etc.

guid – This parameter is automatically filled by the ocfs_uid_gen utility and should never be manually changed.

1.1.2 - Automatic Configuration using ocfstool.

In order to perform the automatic configuration with ocfstool, it is necessary that the GUI environment is properly set and enable.

Open a GUI session as root and execute the command “ocfstool”. Make sure the DISPLAY variable is set be-fore starting the tool.

When the ocfstool window(See Illustration 1) open, in-voke the Generate Config(See Illustration 7) task by ei-ther pressing the key sequence <CTRL-G> or using the menu, clicking on Tasks>Generate Config . For more information on the parameters, check the item Gener-ate Config in the ocfstool section.

1.1.3 - Manual configuration

Although the manual configuration is provided, Oracle strongly recommends the usage of the ocfstool since it does provide a reliable, consistent and easy way to

properly configure OCFS.

To manually configure the OCFS, create the file /etc/ocf-s.conf based on the sample below on each one of the nodes participant of the cluster. Make sure the parame-ters are consistent among nodes.

# ocfs config

# Ensure this file exists in /etc directory #

node_name = ca-test2.us.oracle.com

ip_address = 10.0.0.1

ip_port = 7000

comm_voting = 1

After the file is created, execute the utility ocfs_uid_gen with the -c argument as root in order to generate the unique identification key necessary for the OCFS to identify itself in the cluster. After the generation of the uid key, the /etc/ocfs.conf file should looks like:

#

# ocfs config

# Ensure this file exists in /etc#

node_name = ca-testt2.us.oracle.com

ip_address = 10.0.0.1

ip_port = 7000

comm_voting = 1

guid = 9B2996991BCB25DF4CBB0003470CFE75

Loading ocfs.

The /etc/init.d/ocfs startup script is provided in the package and automatically loads the OCFS module if there is an entry for it in /etc/fstab. Using this startup script is the preferred method to load OCFS as it does all the verification before loading the module and mounting the partitions.

If for some reason there still is a need of manually load the OCFS module, just issue the command load_ocfs as root. If the process is successfully executed, it should show a message like:

# load_ocfs

/sbin/insmod ocfs node_name=ca-test2.us.oracle.com ip_address=10.0.0.1 ip_port=7000 cs=1859 guid=9B2996991BCB25DF4CBB0003470CFE75

Using /lib/modules/2.4.9-e-enterprise-ABI/ocfs/ocfs.o

Formatting an OCFS Partition.

Similar to configuring the OCFS, there are two ways to

if using a specific network, make sure all nodes will have hostname and IP addresses

format an OCFS partition. One is using the “ocfstool” command and the GUI environment, and the other is using the “mkfs.ocfs” command from the shell prompt. Both commands needs to be executed as root.

None of the ocfs utilities will partition the disk at any time. So, before formatting, choose the utility of your preference and partition the disk according to the needs of your implementation, and make sure the disk/partition is not being used by anything else to avoid data loss.

With OCFS you have to format a partition only once on one node, after that every node will be able to mount this filesystem. (of course the device needs to be visible on every node in the cluster)

1.1.4 - Format using ocfstool.

Start by invoking the ocfstool command like described in the section “3.2.2 Automatic configuration using ocf-stool”. After getting the ocfstool window(See illustra-tion 6), press either the sequence key <CTRL-F> or from the menu, choose Tasks>Format. Fill in all fields and click the “OK” button. For more information on the pa-rameters, check the format item under ocfstool section.

1.1.5 - Format using mkfs.ocfs.

For people who do not have a GUI available, use mkf-s.ocfs. If a GUI is available, the preferred method to for-mat an OCFS partition is to use the ocfstool. The mkfs.ocfs command has the following syntax:

# mkfs.ocfs

usage: mkfs.ocfs -b block-size [-C] [-F] [-g gid] [-h] -L volume-label

-m mount-path [-n] [-p permissions] [-q] [-u uid] [-V] device

-b Block size in kilo bytes

-C Clear all data blocks

-F Force format existing OCFS volume

-g GID for the root directory

-h Help

-L Volume label

-m Path where this device will be mounted

-n Query only

-p Permissions for the root directory

-q Quiet execution

-u UID for the root directory

-V Print version and exit

When using the mkfs.ocfs command, the user has to pro-vide all the information that is prompt by the ocfstool utility.

The usage of the “-C” argument will force the mkfs.ocfs to clear all blocks. Depending on the size of the parti-tion, it may be a long process.

The “-F” argument should be used only if the partition was previously formatted as an OCFS.

The “-b” argument specify the blocksize that the parti-tion will be formatted. The blocksize specifies the maxi-mum size of the partition that can be mounted. It goes from 4k to 1M and allows volumes from 32Gb up to 8Tb. Format with 128kb blocksize is optimal size. Sizes between 4kb and 1mb are supported. The smaller blocksizes will have a performance penalty, but will be useful for the future when we will support regular files. 128kb blocksize means that every file created with con-tent uses up a minimum of 128kb space on disk. Even if there is only 1 byte of data in the file. The filesystem will allocate chunks of space in <blocksize> chunks.

The example below show an ordinary partition being formatted and its output.

# mkfs.ocfs -F -b 128 -g dba -u oracle -L /u01 -m /u01 - p 775 /dev/sdb1

Checking heart beat on volume ..........

Clearing volume header sectors...Cleared volume header sectors

Clearing node config sectors...Cleared node config sectors

Clearing publish sectors...Cleared publish sectors

Clearing vote sectors...Cleared vote sectors

Clearing bitmap sectors...Cleared bitmap sectors

Clearing data block...Cleared data block

Writing volume header...Wrote volume header

#

Mount OCFS partitions.

1.1.6 - Mounting manually.

At least for the first time, it is good to have the parti-tions mounted manually instead of automatically. That's because the user has control over all the process and can check if everything works fine.

To mount the new OCFS partition, use the mount com-mand with the “-t” argument specifying “ocfs” in front of it. The example below shows how an OCFS partition is mounted:

# mount -t ocfs /dev/sdb1 /u01

1.1.7 - Mounting automatically

To mount the OCFS partitions automatically, just add the partition information to the /etc/fstab file. The exam-ple below show an entry for RedHat AS 2.1.

/dev/sdb1 /u01 ocfs _netdev

/dev/sdd1 /u02 ocfs _netdev

Tools

ocfstool

ocfstool is a GUI frontend for managing and debugging OCFS volumes on the system; and also the preferred method for managing OCFS. One can mount and un-mount volumes, format partitions, view information and individual files, see the current node map, and block bitmap.

The _netdev option is supported on RedHat, but may not be for UL distributions. Check the mount man pages for

After starting the tool you are presented with a window con-sisting of 2 sections or segments.

The top portion maps all known partitions that are OCFS for-matted and allow users to mount and unmount these parti-tions. The mount operation will try to mount the filesystem to the mountpoint specified during the format operation. Like any other filesystem the umount operation will only succeed if there is no process using it.

The bottom portion has a series of folders that are divided by areas of management and browsing. It will show the informa-tion related to the device selected in the top portion of the screen.

The information that can be obtained is divided in :

General – Contain general information about the filesystem (See Illustration 1). Like ocfs version that formatted the de-vice, mountpoint, size of the filesystem, number of extents, userid and groupid with privileges on the filesystem and the appropriate permission.

File Listing – Will show file information about the filesystem (See Illustration 2). Selecting a file or directory, one can see its information, like size, allocation unit, ownership and protection. One can also see which nodes have the partition mounted at that specific point in time.

Configured Nodes – On this folder, one can see which nodes have the selected partition (See Illustration 3).

Bitmap View – On this folder, one can see the bitmap alloca-tion for the selected partition (See Illustration 4).

Free Space – On this folder, one can see a list of free space for the selected partition (See Illustration 5). This list will show the size and the bit# of the free space.

The preferences menu has effect only to the general folder, allowing the user, group and protection fields to be updated when in

Illustration 1

Illustration 2

Illustration 3

Illustration 4

Illustration 5In addition to the two portions showed, there are the tasks that can be reached by the menu. The available tasks on the menu are:

1.1.8 - Format.

There are two ways to invoke the format window, one is pressing the <CTRL+F> key sequence and the other is in-voking by selecting the menu Tasks>Format.

When the format window is invoked, the following options will be available:

Device – From the pull-down menu, select the device that is going to be OCFS formatted. Make sure the device is not in use by any other application or filesystem before proceeding.

Blocksize – Select the blocksize on which the partition is go-ing to be formatted. Valid values are 4k, 8k, 16k 32k, 64k, 128k, 256k, 512k and 1024k. The optimal size suggested by Oracle is 128k. Smaller blocksize can be selected but they will carry some performance penalty. This is also going to limit the size of the partition that can be used when format-ting with OCFS (32Gb to 8Tb).

Volume Label – This will specify the volume label. This is useful if the user wants to use the volume label to mount the filesystem.

Mountpoint – Specify the location where the partition is go-ing to be mounted. The mount point must exist on all nodes in the cluster that will share the device.

User – Specify the user that will own the filesystem. When the filesystem is mounted, it is automatically owned by the user specified in this field.

Group – Specify the group that the filesystem will belong to. Similar to the User field.

Protection – Sets the default permission of the filesystem when mounted. Usually set to 0755.

Clear all Data Blocks – When checked, this option will make ocfs format block by block, zeroing all the filesystem. This option will increase the time necessary to format the partition considerably. (SLOW)

Force – This options needs to be checked if the partition to be formatted was previously formatted by OCFS.

Illustration 6

1.1.9 - Generate Config.

Invoking this task will promptly generate the /etc/ocfs.-conf file according to the server configuration. Manda-tory fields will be automatically filled with the server in-formation. There is no need to do any further steps as the process will automatically generate the guid in the configuration file..

When the Generate Config window is invoked, the following options will be available:

Interface – Select the interface that is going to be used by the OCFS to generate unique identification. This interface will also be used by the OCFS Distributed Lock Manager (DLM) in future implementation. Make sure that all nodes that will be sharing the OCFS can be reached through the se-lected interface.

Port – Select the port number that is going to be used to communicate to the other nodes. Any non-used port can be assigned to OCFS. Make sure all nodes have the same port configured.

Node-Name – Select the node name associated to the inter-face selected.

Preferred Node Number – This is an optional field. If there is a need to have an specific node assigned to a number, fill this field, otherwise, leave it to the OCFS.

Illustration 7

Extfinder

Extfinder is an utility to report a list of contiguous free extents in the ocfs filesystem. It will show how frag-mented the filesystem is after usage.

Debugocfs

The debugocfs utility is used to extract metadata infor-mation from the OCFS partition. This information is useful to help diagnose eventual problems that may arise. The debugocfs syntax is :

debugocfs: Usage: debugocfs [-?] [-h] [-g] [-l] [-v range] [-p range]

[-d /dir/name] [-f /file/name [-s /path/to/file]] [-a range] [-A range]

[-b range] [-B range] [-r range] [-c range] [-L range] [-M range]

[-n nodenum] /dev/name

-h: volume header

-g: global bitmap

-l: full listing of all file entries

-v: vote sector

-2: print 8-byte number as 2 4-byte numbers

-p: publish sector

-d: ocfs_dir_node structure for a given path

-f: ocfs_file_entry structure for a given file

-F: ocfs_file_entry and ocfs_extent_group structures for a given file

-s: suck file out to a given location

-a: file allocation system file

-A: dir allocation system file

-b: file allocation bitmap system file

-B: dir allocation bitmap system file

-r: recover log file system file

-c: cleanup log system file

-L: vol metadata log system file

-M: vol metadata system file

-n: perform action as node number given

/dev/name: readable device

range: node numbers to inspect (0-31), commas and dashes ok

ex. 0-3,5,14-17

Below, some examples of the debugocfs outputVolume Header.

[root@ca-test2 root]# debugocfs -h /dev/sdb12

diskheader:

version = 1.2

signature = OracleCFS

mount_point = /u01

serial_num = 0

device_size = 4194860544

start_off = 0

bitmap_off = 56320

publ_off = 23552

vote_off = 39936

root_bitmap_off = 0

data_start_off = 1368064

root_bitmap_size = 0

root_off = 2416640

root_size = 0

cluster_size = 131072

num_nodes = 32

num_clusters = 31989

dir_node_size = 0

file_node_size = 0

internal_off = 1368064

prot_bits = 493

uid = 1011

gid = 1011

excl_mount = -1

volumelabel:

curr_master = 0

file_lock = OCFS_DLM_NO_LOCK

oin_node_map = 00000000000000000000000000000000

seq_num = 0

label = /u01

label_len = 4

[root@ca-test2 root]#

Vote Sector.[root@ca-test2 root]# debugocfs -v 0-2 /dev/sdb12

vote0:

seq_num = 0

dir_ent = 0

open_handle = No

Vote0 = (0x00000000)

Vote1 = (0x00000000)

Vote2 = (0x00000000)

vote1:

seq_num = 0

dir_ent = 0

open_handle = No

Vote0 = (0x00000000)

Vote1 = (0x00000000)

Vote2 = (0x00000000)

vote2:

seq_num = 0

dir_ent = 0

open_handle = No

Vote0 = (0x00000000)

Vote1 = (0x00000000)

Vote2 = (0x00000000)


ocfs_file_entry structure[root@ca-test2 root]# ls /u01/oradata/smdb/dbfiles/control01.ctl

/u01/oradata/smdb/dbfiles/control01.ctl

[root@ca-test2 root]# debugocfs -f /oradata/smdb/dbfiles/control01.ctl /dev/sdb12

fileinfo:

Name = /oradata/smdb/dbfiles/control01.ctl

curr_master = 0

file_lock = OCFS_DLM_ENABLE_CACHE_LOCK

oin_node_map = 10000000000000000000000000000000

seq_num = 0

local_ext = true

granularity = -1

filename = control01.ctl

filename_len = 13

file_size = 12509184

alloc_size = 12582912

attribs = OCFS_ATTRIB_REG

prot_bits = S_IRUSR S_IWUSR S_IXUSR S_IRGRP S_IWGRP S_IXGRP S_IROTH S_IXOTH

uid = 1011

gid = 1011

create_time = Fri Aug 29 16:49:10 2003

modify_time = Fri Aug 29 16:49:10 2003

dir_node_ptr = 2940928

this_sector = 2941440

last_ext_ptr = 0

sync_flags = OCFS_SYNC_FLAG_VALID

link_cnt = 0

next_del = 0

next_free_ext = 1

extent[0].file_off = 0

extent[0].num_bytes = 12582912

extent[0].disk_off = 4644864








NOTE: The path specified for the file and directory structure are relative to the mountpoint, not to the root filesystem. In the example in the right, to obtain file entry structure for /u01/oradata/smdb/control01.

Support Utilities

In this section, a description of each one of the utilities belonging to the ocfs-support package. These packages are the minimum necessary to make the OCFS work properly.

/etc/init.d/ocfs

This script is responsible to automatically load the ocfs module, perform a sanity check on the environment and mount all the ocfs partitions listed in /etc/fstab.

/sbin/fsck.ocfs

Perform a filesystem check on the partition. All nodes of a cluster that share a specific device need to have them unmounted before running fsck.ocfs, if one of the nodes has the partition mounted, the fsck.ocfs will fail. The command syntax is:

usage: fsck.ocfs [-N] [-v] device

-N No write

-V Version

-v Verbose

-q Quiet

/sbin/load_ocfs

The load_ocfs script will call the insmod command and load the ocfs module with all the correct parameters.

/sbin/mkfs.ocfs

This command is used to build a the filesystem struc-ture in a partition. The syntax of the mkfs.ocfs command is :

usage: mkfs.ocfs -b block-size [-C] [-F] [-g gid] [-h] -L volume-label

-m mount-path [-n] [-p permissions] [-q] [-u uid] [-V] device

-b Block size in kilo bytes

-C Clear all data blocks

-F Force format existing OCFS volume

-g GID for the root directory

-h Help

-L Volume label

-m Path where this device will be mounted

-n Query only

-p Permissions for the root directory

-q Quiet execution

-u UID for the root directory

-V Print version and exit

/sbin/mounted.ocfs

This command mounted.ocfs is used basically to check which nodes have a specific device mounted. The com-mand syntax is :

usage: mounted.ocfs <device>

/sbin/ocfs_uid_gen

The ocfs_uid_gen command is usually used once during the configuration of the OCFS after it is installed. Its function is to generate the unique key, in the /etc/ocfs.-conf file, that identify the node in the OCFS node man-ager. The command syntax is :

Usage: ocfs_uid_gen -c

ocfs_uid_gen -r

-c Create a new GUID for a new node before it enters the

cluster.

-r Recover the GUID, inserting the new MAC address. This

operation is only for existing nodes whose MAC address

has changed.

Best Practices

For RedHat AS 2.1 go to http://oss.oracle.com/projects/ocfs/dist/documentation/RHAS_best_practices.html.

http://oss.oracle.com/projects/ocfs/dist/documentation/RHAS_best_practices.html

http://oss.oracle.com/projects/ocfs/dist/documentation/RHAS_best_practices.html

ForUnited Linux, go to http://oss.oracle.com/projects/ocfs/dist/documentation/UL_best_practices.txt.

Frequently asked Questions & Answers

Q. Can I install the Oracle Distribution in an OCFS partition?

A. No. OCFS version 1 supports only database files (datafiles, logfiles, controlfiles, archivelog files, srvm configuration file and Oracle Cluster Manager quorum file).

Q. My partitions don't mount automatically during boot. What's wrong?

A. The ocfs script on /etc/init.d directory is not enable. Use the chkconfig command to verify that and enable it. Instructions on how to enable it are listed under item 3.1 - Automatically mount OCFS during boot.

Q. When running fsck.ocfs, it returns the error “WARNING: nonzero bytes after the disk header structure”. What does it mean?

A. Earlier releases of mkfs.ocfs did not clear up 1st block entirely. We've since fixed the issue in the format utility. As far as functioning goes, one can safely ignore this message.

Q. What are the most appropriate (recommended) tool IO and capacity analysis of OCFS?

A. Your usual tools (sar, vmstat, iostat) will work, it handles io just like any other filesystem.

Q. Can I use ocfs on a NAS (Network Attached Storage) device like NetApp?

A. No. NAS works through network and the physical volume and filesystem is under the fileserver control. OCFS needs to use a direct attached storage device or SAN (Storage Area Network) where the local server has total control over the physical volumes.

http://oss.oracle.com/projects/ocfs/dist/documentation/UL_best_practices.txt

http://oss.oracle.com/projects/ocfs/dist/documentation/UL_best_practices.txt

Q. Can I use LVM or MD to create my OCFS filesystem on top of it?

A. LVM and MD are not cluster aware and it makes it very difficult to manage in a cluster environment. No tests have been performed using LVM or MD. It may work, but it also may not. Corruptions are likely to happen since they are not cluster aware software.

Q.I want to perform some tests using RAC/OCFS, but I don't want to spend too much money on hardware. Do I have any other solution?

A.Yes. Users can run on firewire. Oracle does provide a special kernel and ocfs modules to run on firewire. More information can be found at http://oss.oracle.com/projects/firewire/.

Q.Why do I have to include the option _netdev in the fstab on RedHat?

A.The _netdev parameter is used when the filesystem resides on a device that requires network access, it is used to prevent the system from attempting to mount these filesystems until the network has been enabled on the system.

Q.What is the best way to archive the logs to an OCFS directory?

A.The ideal is to create a directory for each one of the nodes that will be archiving on OCFS. This way, the logs will be kept organized and there will be no concurrency when two or more nodes archive at the same time. This doesn't mean there will be only archivelogs of the node on its directory, because Oracle does allow a node to perform an archive operation if a specific node is down for long time to reduce recovery time.

Q.Is OCFS supported on 64 bit platform like Itanium?

A.Yes, you can find specific packages for ia32 and ia64.

Q. Can I run ocfs on non enterprise Linux distributions like RedHat 9?

A.OCFS is primarily developed for Enterprise

http://oss.oracle.com/projects/firewire/

systems and therefore not officially supported on platforms other than RedHat Advanced Server 2.1 or UL. The binaries can be compiled from source at your own risk.

Q.Do I need any specific version of United Linux to run OCFS?

A. Yes. The latest package of OCFS will work with UL SP1, but Oracle strongly recommends to run on SP2 or SP2a.

Q. Can I run the latest OCFS on a plain RedHat AS 2.1, without any errata applied?

A.No. Before the errata e.12, there was a specific module for each one of the errata releases. After e.12, the module is generic, working with any of the erratas released. Oracle strongly recommends the usage of the latest errata available.

Q.What is the advantage of running ocfs against raw devices?

A.There are many advantages on running OCFS. The biggest and more important is that it looks and feels like a regular filesystem, so users don't have to worry with all the raw devices administration difficulties. Also, users don't have to create a partition for each datafile, a few partitions or even a single partition, depending on the needs and storage configuration, may be able to handle big databases. It also eliminate the restriction of 255 raw partitions limit on Linux, giving almost endless number of datafiles that can be created.

Q.What happen if I have to change the IP Address of my systems?

A.Changing IP address on the same interface used by the ocfs will require you to change the IP Address specified in the /etc/ocfs.conf file only.

Q.My Network Interface Card had to be replaced. Do I need to do something?

A.Yes. Run the utility “ocfs_uid_gen -r” to update the /etc/ocfs.conf file with the new uid.

Q.I have a partition that is not mounted. How do I know if it is an ocfs partition or not?

A. Run the command “debugocfs -h /dev/xxxx” as root. If it is an ocfs partition, the signature will show “OracleCFS”. For all other type of filesystem, it will just return something else, not necessarily the fstype.

Q.Can I use my OCFS partition to store regular files?

A.Like Oracle Distribution, regular files are not supported on OCFS at the moment. This is planned for OCFS 2.

Q.How much do I lose in terms of performance compared to raw devices?

A.Not too much. Without async I/O, performance have been fluctuating from 0 to 5% of raw devices. Considering all the benefits of a filesystem against raw, the loss is not that big. Using async I/O, ocfs has shown to be about 5% faster than raw devices.

Q.How do I enable async I/O on Oracle using OCFS?

A.First of all, it is necessary to relink the Oracle RDBMS to enable asynchronous I/O (cd $ORACLE_HOME/rdbms/lib; make -f ins_rdbms.mk async_on). Then add to the init.ora file, the parameters :

filesystemio_options=setall

_dbwr_async_io=TRUE

tape_asynch_io=true

disk_asynch_io=true

Q.How do I backup my OCFS files? Can I use tar or other OS command?

A.The best way to perform an Oracle backup is to use RMAN. Since a lot of users have their own scripts that usually uses tar or other OS commands, we have made available a set of commands that support direct_io, so that users can take hot backup using tar, cp, dd. Go to http://oss.oracle.com/projects/coreutils/files/ to get the fileutils/coreutils appropriate to your environment. Note that for RHAS 2.1 and SLES8, the package is called fileutils and for RHEL3, it is called coreutils.

Q.Is it possible to resize an existing OCFS partition?

A.Yes, tuneocfs command will do that. Please refer to the tuneocfs manpages for additional information on syntax.

Q.I'm having problems with OCFS. How can I debug OCFS?

A.One needs to be careful enabling debug on ocfs because it

http://oss.oracle.com/projects/coreutils/files/

usually fills the /var/log/messages file very quickly. To enable OCFS tracing:

echo -1 > /proc/sys/kernel/ocfs/debug_level

echo -1 > /proc/sys/kernel/ocfs/debug_context

To disable OCFS tracing:

echo 0 > /proc/sys/kernel/ocfs/debug_level

echo 0 > /proc/sys/kernel/ocfs/debug_context

Q.Can I run OCFS in a stand-alone system? What are the advantages of running it?

A.Yes. There is no problems on running OCFS in a stand-alone system since you use it for database files only (Datafile, logfiles, controlfiles, srvm configuration file and Oracle Cluster Manager quorum file). The advantage of using it on stand-alone is that if one needs to upgrade to a RAC environment, it will be a smooth transition, basically adding a new node, recompiling the Oracle RDBMS kernel to enable RAC mode and add a new online log thread.

Q.I have a database running on OCFS in a stand-alone node. Why it is so slow compared to other stand-alone systems running on ext3?

A.OCFS does direct_io, which bypass the Linux cache. Ext3 in other hands makes full usage of the Linux cache. Depending on the memory available, it is possible that one will mostly find the data in cache, not having to fetch from the disk. But this condition happens only if there are enough free memory for cache. As you use more and more memory, performance drops significantly.

Q.How can I obtain more information about OCFS?

A.Go to http://oss.oracle.com/ocfs. You will be able to find documents, sources, binaries and all sort of information.

Q.I have a customized RedHat AS kernel on my system. Does Oracle support OCFS and the RDBMS on it?

A.No. Oracle only supports standard kernels for OCFS and Oracle RDBMS. It would be impossible to track and reproduce any problem without knowing all the customizations that were performed in a specific installation.

Q.How do I know if my OCFS version is officially supported by Oracle?

http://oss.oracle.com/ocfs

A.Check the file /var/log/messages for a message that looks like “Oracle Cluster FileSystem 1.0.9-PROD Wed Jul 30 16:48:00 PDT 2003 (build 686e542792ebee44eaa1f)”

Q.Is OCFS is mountable on Linux and Windows 2000 simultaneously?

A.No. The on-disk format is not the same. Is similar... but not the same.

Appendix A

Ocfs-support package Information:Name : ocfs-support Relocations: (not relocatable)

Version : 1.0.9 Vendor: Oracle Corporation

Release : 5 Build Date: Wed 30 Jul 2003 07:48:40 PM EDT

Install date: (not installed) Build Host: ca-build1.us.oracle.com

Group : System Environment/Kernel Source RPM: ocfs-2.4.9-e-1.0.9-5.src.rpm

Size : 698099 License: GPL

Packager : nobody <[email protected]>

URL : http://ocfs.otncast.otnxchange.oracle.com/servlets/ProjectHome

Summary : Support programs for the Oracle Cluster Filesystem

Description :

Support programs for using the Oracle Cluster Filesystem.

/etc/init.d/ocfs

/sbin/fsck.ocfs

/sbin/load_ocfs

/sbin/mkfs.ocfs

/sbin/mounted.ocfs

/sbin/ocfs_uid_gen

Ocfs-tools package information:Name : ocfs-tools Relocations: (not relocatable)








Summary : Tools for managing the Oracle Cluster Filesystem

Description :

Tools to manage the Oracle Cluster Filesystem

/usr/bin

/usr/bin/cdslctl

/usr/bin/debugocfs

/usr/bin/extfinder

/usr/bin/ocfstool

/usr/share

/usr/share/man

/usr/share/man/man1

/usr/share/man/man1/cdslctl.1.gz

/usr/share/man/man1/ocfstool.1.gz

Ocfs Module Package Information:Name : ocfs-2.4.9-e-enterprise Relocations: (not relocateable)








Summary : The Oracle Cluster Filesystem for enterprise systems.

Description :

OCFS is the Oracle Cluster Filesystem. This package is compiled for

symmetric processor kernels on machines with more than 4GB of RAM.

/lib/modules/2.4.9-e-enterprise-ABI/ocfs

/lib/modules/2.4.9-e-enterprise-ABI/ocfs-noaio

/lib/modules/2.4.9-e-enterprise-ABI/ocfs-noaio/ocfs.o

/lib/modules/2.4.9-e-enterprise-ABI/ocfs/ocfs.o

This example show the information about the enterprise module. The same files will be found in the other supported versions

OCFS - Oracle Clustered Filesystem for Linux

Users Guide.

August 2003

Author: Marcos E. Matsunaga

Contributing Authors: Wim Coekaerts, Kurt Hackel, Sunil Mushran, Manish Singh and Mark Fasheh.

Oracle Corporation

World Headquarters

500 Oracle Parkway

Redwood Shores, CA 94065

U.S.A.

Worldwide Inquiries:

Phone: +1.650.506.7000

Fax: +1.650.506.7200

www.oracle.com

For information about the Oracle Open Source, visit us at http://oss.oracle.com

Oracle is a registered trademark of Oracle Corporation. Various

product and service names referenced herein may be trademarks

of Oracle Corporation. All other product and service names

mentioned may be trademarks of their respective owners.

Copyright © 2001 Oracle Corporation

ocfs users guide

Documents