multipath on linux

8/9/2019 MultiPath on Linux

1/8

C H A P T E R

11-1

Cisco VFrame Third Party Integration Guide

OL-12603-01

11

Configuring PowerPath Multipathing for Linux

This chapter contains the following sections:

Configuring Boot over Multipathing, page 11-1

• Preparing the System and Performing PowerPath Installation, page 11-1

• Setting Up Files and Directories, page 11-2

• Copying Required Objects and Configuration Files, page 11-2

• Modifying Files, page 11-4

• Archiving the bootdisk Directory, page 11-6

Configuring Boot over MultipathingThe following steps configure Boot over IB using PowerPath, using Linux RHEL3 Update7 on a X86_64

(AMD) system as an example.

Preparing the System and Performing PowerPath InstallationTo prepare the system and install PowerPath, follow these steps:

Step 1 Set up the system for boot over SRP. When the system can boot over SRP, proceed with the steps below

to get the system boot over PowerPath.

Step 2 Install PowerPath on the system. Follow the usual EMC PowerPath installation procedure to install

PowerPath. Refer to the PowerPath installation document for instructions.

pm -ivh EMCpower.LINUX-4.5.1-022.rhel.x86_64.rpm

/etc/init.d/PowerPath start

emcpreg -install (register powerpath)

Step 3 After PowerPath is installed properly and started, enter the fdisk -l command. You should see output

similar to the example below. In this case, only one LUN is assigned to this host, and the LUN is

partitioned accordingly.

[root@sw-n4300-1 ~]# fdisk -l

Disk /dev/sda: 21.4 GB, 21474836480 bytes255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System


2/8

11-2


OL-12603-01

Chapter 11 Configuring PowerPath Multipathing for Linux

Setting Up Files and Directories

/dev/sda1 * 1 38 305203+ 83 Linux/dev/sda2 39 293 2048287+ 82 Linux swap

/dev/sda3 294 2610 18611302+ 83 Linux

Disk /dev/emcpowera: 21.4 GB, 21474836480 bytes255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/emcpowera1 * 1 38 305203+ 83 Linux/dev/emcpowera2 39 293 2048287+ 82 Linux swap

/dev/emcpowera3 294 2610 18611302+ 83 Linux

PowerPath is now up and running. The emcpowera* shows the PowerPath pseudo devices.

Setting Up Files and DirectoriesTo set up files and directories, follow these steps:

Step 1 Create a new directory called bootdisk.

Step 2 Copy the initrd file from the /boot to the new bootdisk directory.

-cp -p /boot/initrd-2.6.9-34.ELsmp.img bootdisk/

Step 3 Unzip the initrd-2.6.9-34.ELsmp.img file in the same bootdisk directory.

zcat initrd-2.6.9-34.ELsmp.img > initrd-2.6.9-34.ELsmp

Step 4 Copy the initrd directory structure and all the associated files into the same bootdisk directory, as follows

-cpio -i -I initrd-2.6.9-34.ELsmp

Step 5 Verify that the bootdisk directory now contains the following files and directories, in addition to

initrd-2.6.9-34.ELsmp.img and initrd-2.6.9-34.ELsmp files.

bin dev etc init lib loopfs proc sbin sys sysroot

Step 6 Delete the initrd-2.6.9-34.ELsmp.img and initrd-2.6.9-34.ELsmp files from the bootdisk directory.

Step 7 Create two more directories in the bootdisk directory and call them lib64 and usr.

Step 8 Verify that the following files and directories are in the bootdisk directory:

bin dev etc init lib lib64 loopfs proc sbin sys sysroot usr

Copying Required Objects and Configuration FilesTo copy the requireed PowerPath objects and configuration files, follow these steps:

Step 1 Copy the PowerPath objects and configuration files into the new bootdisk directory. Some Linux objects

also need to be copied.


3/8

11-3


OL-12603-01


Copying Required Objects and Configuration Files

The entire list is displayed in the table below.

• Note that it is not required to copy the IB module since it should already be in the /lib directory in

the original initrd file.

• The /lib64 directory is required since it is a x86_64 system. You will copy all the required library

files to either the /lib or /lib64 directory.

• Use the “ldd” command to obtain the required library, as shown in this example:

[root@host-1 bin]# ldd /bin/mkdir

libselinux.so.1 => /lib64/libselinux.so.1 (0x0000003977b00000)

libc.so.6 => /lib64/tls/libc.so.6 (0x0000003976800000)

/lib64/ld-linux-x86-64.so.2 (0x0000003976600000)[root@host-1 bin]# ldd /sbin/powermt

linux-gate.so.1 => (0xffffe000)

libc.so.6 => /lib/tls/libc.so.6 (0x00adf000)/lib/ld-linux.so.2 (0x00ac6000)

Table 11-1 Copy the PowerPath Objects and Configuration Files to the Directory

Notes Copy To Bootdisk Directory

/bin/ash ----> /sbin/ash

/bin/gawk ----> /sbin/gawk

/bin/mkdir ----> /sbin/mkdir

/sbin/powermt ----> /sbin/powermt

/sbin/rmmod ----> /sbin/rmmod

/bin/mount ----> /sbin/mount

/bin/umount ----> /sbin/umount

/bin/cat ----> /sbin/cat

/bin/mknod ----> /sbin/mknod

/sbin/pivot_root ----> /sbin/pivot_root

/bin/sleep ----> /sbin/sleep

Copy the whole directory /etc/emc ----> /etc/emc

/etc/emcp_devicesDB.dat ----> /etc/emcp_devicesDB.dat

/etc/emcp_devicesDB.idx ----> /etc/emcp_devicesDB.idx

/etc/emcp_registration ----> /etc/emcp_registration

This directory contains

/etc/opt/emcpower/powercf and

/etc/opt/emcpower/emcpmgr object.

These objects are used in the init file.

/etc/opt/emcpower ----> /etc/opt/emcpower

/etc/fstab ----> /etc/fstab

Copy the whole PowerPath module. /lib/modules/2.6.9-34.ELsmp/extr

a/emc*.ko

----> /etc/lib

/usr/lib/libemcp_core.so ----> /lib/libemcp_core.so

/usr/lib/libemcp_lam.so ----> /lib/libemcp_lam.so

/usr/lib/libemcp_lic_rtl.so ----> /lib/libemcp_lic_rtl.so

/usr/lib/libemcp_mp_rtl.so ----> /lib/libemcp_mp_rtl.so


4/8

11-4


OL-12603-01


Modifying Files

Modifying FilesTo modify the /etc/fstab file, which you previously copied, follow these steps:

Step 1 Modify the /etc/fstab file to reflect the PowerPath devices:

In this example, /dev/emcpowera3 is the root partition and /dev/emcpowera1 is the boot partition.

# This file is edited by fstab-sync - see ‘man fstab-sync’ for details

/dev/emcpowera3 / ext3 defaults 1 1/dev/emcpowera1 /boot ext3 defaults 1 2

none /dev/pts devpts gid=5,mode=620 0 0

none /dev/shm tmpfs defaults 0 0

none /proc proc defaults 0 0none /sys sysfs defaults 0 0

/dev/emcpowera2 swap swap defaults 0 0

/dev/hdc /media/cdrom autopamconsole,exec,noauto,managed 0 0

/dev/fd0 /media/floppy auto

pamconsole,exec,noauto,managed 0 0

Step 2 Modify the default init file to the file below:

#!/bin/ash

mount -t proc /proc /proc

echo Mounted /proc filesystem

echo Mounting sysfs

mount -t sysfs none /sys

/usr/lib/libemcpmp.so ----> /lib/libemcpmp.so

/usr/lib/libemcp.so ----> /lib/libemcp.so

/lib/ld-linux.so.2 ----> /lib/ld-linux.so.2

/lib/tls/libm.so.6 ----> /lib/libm.so.6

/lib/tls/libc.so.6 ----> /lib/libc.so.6

/lib64/libdl.so.2 ----> /lib64/libdl.so.2

/lib64/tls/libm.so.6 ----> /lib64/tls/libm.so.6

/lib64/tls/libc.so.6 ----> /lib64/tls/libc.so.6

/lib64/ld-linux-x86-64.so.2 ----> /lib64/ld-linux-x86-64.so.2

/lib64/libselinux.so.1 ----> /lib64/libselinux.so.1

/lib64/tls/librt.so.1 ----> /lib64/tls/librt.so.1

/lib64/tls/libpthread.so.0 ----> /lib64/tls/libpthread.so.0

/lib/libpn.so ----> /lib/libpn.so

/usr/sbin/chroot ----> /usr/sbin/chroot

Table 11-1 Copy the PowerPath Objects and Configuration Files to the Directory

Notes Copy To Bootdisk Directory


5/8

11-5


OL-12603-01


Modifying Files

echo Creating /dev mount -o mode=0755 -t tmpfs none /dev

mknod /dev/console c 5 1

mknod /dev/null c 1 3 mknod /dev/zero c 1 5

mkdir /dev/pts

mkdir /dev/shm

echo Starting udevudevstart

echo -n “/sbin/hotplug” > /proc/sys/kernel/hotplugecho “Loading mlxsys.ko module”

insmod /lib/mlxsys.ko

echo “Loading mod_vip.ko module”insmod /lib/mod_vip.ko

echo “Loading mod_thh.ko module”

insmod /lib/mod_thh.koecho “Loading mod_rhh.ko module”

insmod /lib/mod_rhh.ko

echo “Loading ts_kernel_services.ko module”insmod /lib/ts_kernel_services.ko

echo “Loading ts_ib_packet_lib.ko module”

insmod /lib/ts_ib_packet_lib.ko

echo “Loading ts_ib_core.ko module”insmod /lib/ts_ib_core.ko

echo “Loading ts_ib_mad.ko module”

insmod /lib/ts_ib_mad.koecho “Loading ts_ib_tavor.ko module”

insmod /lib/ts_ib_tavor.ko

echo “Loading scsi_mod.ko module”

insmod /lib/scsi_mod.koecho “Loading sd_mod.ko module”

insmod /lib/sd_mod.ko

echo “Loading ts_kernel_poll.ko module”insmod /lib/ts_kernel_poll.ko

echo “Loading ts_ib_client_query.ko module”

insmod /lib/ts_ib_client_query.ko

echo “Loading ts_ib_sa_client.ko module”insmod /lib/ts_ib_sa_client.ko

echo “Loading ts_ib_dm_client.ko module”

insmod /lib/ts_ib_dm_client.koecho “Loading ts_ib_cm.ko module”

insmod /lib/ts_ib_cm.ko

echo “Loading ts_srp_host.ko module”

insmod /lib/ts_srp_host.koecho “Loading emcplib.ko module”

insmod /lib/emcplib.ko

echo “Loading emcp.ko module”insmod /lib/emcp.ko

echo “Loading emcpmp.ko module”

insmod /lib/emcpmp.koecho “Loading emcpmpc.ko module”

insmod /lib/emcpmpc.ko

echo “Loading emcpmpaa.ko module”

insmod /lib/emcpmpaa.koecho “Loading emcpmpap.ko module”

insmod /lib/emcpmpap.koecho “Loading emcphr.ko module”

insmod /lib/emcphr.ko

echo “Loading emcpioc.ko module”insmod /lib/emcpioc.ko

echo “Removing module emcpioc.ko”

rmmod emcpiocecho “Loading mptbase.ko module”

insmod /lib/mptbase.ko


6/8

11-6


OL-12603-01


Archiving the bootdisk Directory

echo “Loading mptscsih.ko module”insmod /lib/mptscsih.ko

echo “Loading jbd.ko module”

insmod /lib/jbd.koecho “Loading ext3.ko module”

insmod /lib/ext3.ko

udevstart

rm -f /dev/emcpower

mmaj=`cat /sys/class/misc/emcpower/dev | gawk -F : '{print $1}'` mmin=`cat /sys/class/misc/emcpower/dev | gawk -F : '{print $2}'`

mknod /dev/emcpower c $mmaj $mmin

sleep 2

echo “Start the configuration of powerpath”powermt config

powermt load

/etc/opt/emcpower/emcpmgr map

sleep 7

/etc/opt/emcpower/powercf -C powermt savepowermt register

sleep 2

echo “Creating root device”

echo “mkrootdev /dev/root” | /sbin/nash

umount /sys

echo “Mounting root filesystem”

mount -o defaults --ro -t ext3 /dev/root /sysroot

mount -t tmpfs --bind /dev /sysroot/dev

umount /proc

cd /sysrootecho “Switching to the new root”

pivot_root . initrd

exec chroot . /sbin/init dev/console 2>&1

echo “Unmount the old dev now”

umount /initrd/dev

Archiving the bootdisk DirectoryTo archive the bootdisk directory, follow these steps:

Step 1 Archive the bootdisk directory now that all the required files are copied into it.

In this example the archive file is put at /root/

a. cd to the bootdisk directory.

b. Locate and copy the file.

find . | cpio -v -c -o > /root/initrd-2.6.9-34.ELsmp

c. Compress the file.


7/8

11-7


OL-12603-01



gzip -9 initrd-2.6.9-34.ELsmp

d. Move the directory.

mv initrd-2.6.9-34.ELsmp.gz initrd-2.6.9-34.ELsmp.img

Completing the PowerPath Configuration

To complete the PowerPath configuration, follow these steps:

Step 1 Move the initrd-2.6.9-34.ELsmp.img file to the /boot directory.

Step 2 Change the bootloader append for the root parameter of the kernel to the appropriate PowerPath pseudo

device.

In this example the PowerPath pseaudo device was root=/dev/emcpowera3

Step 3Reboot the server. It will boot the root filesystem on a PowerPath device instead of a sd device.


8/8

11-8


OL-12603-01



multipath on linux

Documents