Tag Archives: configration

Linux: Native Multipath Configration on RHEL 4

1. The device-mapper-multipath package is not installed as part of a default installation of the operating system. Select the package as part of the OS install or install the rpm later either from the install CD or by downloading the rpm from the Red Hat website.
2. Verify that the required version of uDev has been installed for your environment. To query the uDev version, execute the command:

# rpm -q udev
If required, upgrade the uDev package and then execute the
command to create a new device under the ‘/dev’ directory.

# udevstart
3. Load the dm_multipath kernel module if it is not already loaded.

# modprobe dm_multipath
4. Replace the default /etc/multipath.conf with the following multipath.conf file recommended by EMC for attach to EMC storage.

Follow the instructions in the annotated multipath.conf file for masking internal scsi disks or disks that need to be excluded from multipath control.

## This is the /etc/multipath.conf file recommended for
## EMC storage devices.
## OS : RHEL 4 U3
## Arrays : CLARiiON and Symmetrix
## The blacklist is the enumeration of all devices that are to be
## excluded from multipath control
## Replace the wwid with the output of the command
## ‘scsi_id -g -u -s /block/[internal scsi disk name]’
## Enumerate the wwid for all internal scsi disks.
## Optionally, the wwid of VCM database may also be listed here.

wwid 35005076718 d4224d
devnode “^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*”
devnode “^hd[a-z][0-9]*”
devnode “^cciss!c[0-9]d[0-9]*[p[0-9]*]”
## Use user friendly names, instead of using WWIDs as names.
defaults {
## Use user friendly names, instead of using WWIDs as names.
user_friendly_names yes
devices {
## Device attributes requirements for EMC Symmetrix
## are part of the default definitions and do not require separate
## definition.
## Device attributes for EMC CLARiiON
device {
vendor “DGC “
product “*”
path_grouping_policy group_by_prio
getuid_callout “/sbin/scsi_id -g -u -s /block/%n”
prio_callout “/sbin/mpath_prio_emc /dev/%n”
path_checker emc_clariion
path_selector “round-robin 0”
features “1 queue_if_no_path”
no_path_retry 300
hardware_handler “1 emc”
failback immediate
5. Perform a dry run and evaluate the setup by running the
following command.
# multipath -v2 -d
• With Symmetrix Logical Units, the output will look similar to:
create: SEMC_____SYMMETRIX______490073232000
[size=5 GB][features=”0″][hwhandler=”0″]
_ round-robin 0
_ 11:0:1:52 sdao 66:128
_ 11:0:2:52 sdaz 67:48
_ 10:0:1:52 sdm 8:192
_ 10:0:2:52 sdx 65:112
create: SEMC_____SYMMETRIX______490073233000
[size=5 GB][features=”0″][hwhandler=”0″]
_ round-robin 0
_ 11:0:1:53 sdap 66:144
_ 11:0:2:53 sdba 67:64
_ 10:0:1:53 sdn 8:208
_ 10:0:2:53 sdy 65:128
create: SEMC_____SYMMETRIX______490073234000
[size=5 GB][features=”0″][hwhandler=”0″]
_ round-robin 0
_ 11:0:1:54 sdaq 66:160
_ 11:0:2:54 sdbb 67:80
_ 10:0:1:54 sdo 8:224
_ 10:0:2:54 sdz 65:144
• With CLARiiON Logical Units, the output will look similar to:
create: 360060160b540160171f77f705558da11
[size=10 GB][features=”1
queue_if_no_path”][hwhandler=”1 emc”]
_ round-robin 0
_ 11:0:0:0 sdad 65:208
_ round-robin 0
_ 10:0:0:0 sdb 8:16
create: 360060160b540160170f77f705558da11
[size=10 GB][features=”1
queue_if_no_path”][hwhandler=”1 emc”]
_ round-robin 0
_ 11:0:0:1 sdae 65:224
_ round-robin 0
_ 10:0:0:1 sdc 8:32

6. If the listing is appropriate, commit the configuration as follows:
a. Start the required multipath processes.

# /etc/init.d/multipathd start
b. Execute the multipath command.

# multipath -v2
c. Perform an ‘lsmod’ and verify that the processes are running.

• With Symmetrix attach the following modules should be listed:
# lsmod |grep dm
dm_round_robin 4929 1
dm_multipath 22097 2 dm_round_robin
dm_mod 66433 1 dm_multipath

• With CLARiiON attach the following modules will be listed:

dm_emc 7489 1
dm_round_robin 4929 1
dm_multipath 22097 3 dm_emc,dm_round_robin
dm_mod 66433 3 dm_multipath

7. To get a listing of the current setup do:

# multipath -ll
8. Integrate the startup of the appropriate daemons in the boot sequence as follows:

# chkconfig --add multipathd
# chkconfig multipathd on

9. Device-mapper (dm) names and sd device names may not be persistent across reboots. There are two mechanisms to achieve persistence:

• Use of friendly names.
The friendly names are softlinks created in the /dev/mpath/ directory.

• Use of LVM-2 on top of device-mapper Configure logical volumes as necessary. LVM based entries in ‘/etc/fstab’ are supported. In the normal boot process, the device mapper and the multipath daemons start before LVM starts the discovery process.

You may optionally reboot the host to verify that the required processes automatically startup.

Linux: iSCSI Initiator installation & configration

Installation Instructions:

Red Hat Supplied iSCSI Initiator:

Find the RPM on the Red Hat Media, then install it using the rpm –ivh command as follows:

# rpm –ivh iscsi-initiator-utils-

NOTE: This is the version for Enterprise Linux AS 5. Your version may be different.

An alternative to installing this package manually in Red Hat Enterprise Linux (ES or AS) 5 or greater is to use the “Add/Remove Applications” menu item in the “System Settings” menu. In the details for the “Network Servers” package list, the iscsi-initiator-utils is one of the packages listed. This same choice is available in the same location during the initial install of Red Hat, so this can also be done at that time.

Once installed, there will be a file in the /etc directory named iscsi.conf. If this file does not exist this may indicate a problem with the installation. This file can be created with the following minimal entries:



This needs to be set to the Group IP Address of your UIT Array.


For the initiator to receive Vendor Specific async events from the target.


To globally specify that all discovery sessions be kept open.

Within the iscsi.conf file itself there are many more options available that can be set. You can look through the iscsi.conf file for information on what these variables are and what they are used for.

Once these values are either placed in a newly created /etc/iscsi.conf file, or the respective lines are uncommented and edited where necessary, the iscsi service can be started:

# service iscsi start

To verify that the iscsi service will be started at boot time, the chkconfig command can be used as follows:

# chkconfig –list iscsi

iscsi 0:off 1:off 2:off 3:off 4:off 5:off 6:off

By default, the newly added iscsi initiator is not enabled at boot which is the reason for each of the run levels listed to have the service set to off. To enable this at boot, again use the chkconfig command as follows:

# chkconfig –add iscsi

# chkconfig iscsi on

The above two commands first checks to be sure there are the necessary scripts to start and stop the service, then it sets this service to be on for the appropriate runlevels.

Then check to be sure the changes took effect:

# chkconfig –list iscsi

iscsi 0:off 1:off 2:on 3:on 4:on 5:on 6:off

To verify that you can see your iscsi devices, you can run the following command:

# iscsi-ls


SFNet iSCSI Driver Version … 6.2 (27-July-2009 )


TARGET NAME : iqn.2001-05.com.UIT:6-8a0900-148270001-987006efd6c43836-pat-rhel5-vol2

TARGET ALIAS : pat-rhel5-vol2


BUS NO : 0






SESSION ID : ISID 00023d000001 TSIH 06


To see greater details of the devices, you can run the above command with the –l option:

# iscsi-ls –s


SFNet iSCSI Driver Version … 6.2 (27-Jun-2009 )


TARGET NAME : iqn.2001-05.com.UIT:6-8a0900-148270001-987006efd6c43836-pat-rhel5-vol2

TARGET ALIAS : pat-rhel5-vol2


BUS NO : 0






SESSION ID : ISID 00023d000001 TSIH 06



LUN ID : 0

Vendor: EQLOGIC Model: 100E-00 Rev: 2.1

Type: Direct-Access ANSI SCSI revision: 05

page83 type3: 0690a018007082143638c4d6ef067098

page80: 3036393041303138303037303832313433363338433444364546303637303938

Device: /dev/sdc


As can be seen in the example iscsi-ls –l output above, the device in question is mapped to the /dev/sdc device.

Linux-iscsi Sourceforge Initiator:

If you are not running the required update of Red Hat Linux to have their precompiled iSCSI Initiator, you can try to compile the iSCSI Initiator supplied by the Sourceforge linux-iscsi project.

Beyond the required kernel revision as noted above, all development packages need to be installed for the compiling of the initiator as well as the kernel sources. The easiest way to install these items is to us the “Add/Remove Applications” in the “System Settings Menu” from within the Desktop GUI. Depending on the version of Red Hat you are running will determine what you select to be installed:

Red Hat AS 3:

Development Tools (Default packages have all required packages)

Kernel Development (Again, default is fine)

Red Hat AS 4:

Development Tools (Default packages have all required packages)

NOTE: If there is no Kernel Development choice, the Kernel Source files need to be found and installed prior to compilation.

Once these OS packages are installed, it should be as easy as getting the source package from the Sourceforge linux-iscsi project, then making the initiator. Refer to the README file that comes with the source for detailed instructions on how to make the initiator. If there are problems compiling the initiator, check the linux-iscsi Sourceforge project for assistance. You are able to search and post to their mailing lists to get information and assistance with this product.

Persistent Device naming:

Devices using the Red Hat software initiators do not have a persistent naming scheme, but a few ways to setup Persistent Naming for the different versions of Red Hat are as follows:

Red Hat Enterprise Linux (ES or AS) 3:

Devlabel (see the devlabel man page):

This will only work on Red Hat kernel’s 2.4.x.

Use devlabel to setup symlinks from known names to the current device name.

A basic add command to setup a devlabel link is as follows:

# devlabel add –d -s

An example:

sdc –s /dev/iscsi/vollink

# ls –l /dev/iscsi/vollink

lrwxrwxrwx 1 root root 8 Dec 1 16:31 newvol -> /dev/sdc

Red Hat Enterprise Linux (ES or AS) 4:

Use the udev facility (man udev, man scsi_id):

This is only available on Red Hat EL 4/Kernel 2.6.*

This creates device links to the device files when the device nodes are created. Udev uses a rules file (see man udev) to determine what the link names or device names it should create for different devices.

This is the least elegant of the solutions to configure and there is no straightforward example to provide on how this needs to be setup.

Red Hat may be able to provide additional information on persistent device naming for iSCSI devices using their iSCSI initiator with udev.

Both Red Hat Enterprise Linux 3 and 4:

Use filesystem LABELs (see the e2label man page):

This will work on all ext2/3 filesystem partitions.

Place an ext2/3 filesystem label on your filesystem partition. Once the Label has been added, use the LABEL identifier to identify the filesystem you want to mount in the fstab (man fstab and/or man mount). Following is an example of using the e2label command and what a resulting line in the fstab file would look like:

# e2label /dev/sdc1 EMC

# mkdir /EMC

# echo “LABEL=EMC /EMC ext3 _netdev,defaults 0 0” >> /etc/fstab

NOTE: _netdev delays the mounting of this filesystem until after the Network has been started and ensures that the filesystem is unmounted before stopping the Network.

# mount –a

# df –k | grep EMC

/dev/sdc1 5166332 43072 4860816 1% /EMC

Red Hat Linux iSCSI Configuration

Supported iSCSI Initiators:

Enterprise Linux (ES or AS) 3 Update 6:

Disc 2 of 4:


linux-iscsi 3.4.x: Minimum kernel release: 2.4.21

linux-iscsi 3.6.x: Minimum kernel release: 2.4.21

NOTE: Versions of the linux-iscsi Initiator above 3.x are not compatible with the 2.4.x and below kernel release.

Enterprise Linux (ES or AS) 4 Update 2:

Disc 4 of 4:



4.0.2 – Mnimum kernel release: 2.6.10

4.0.1 – minimum kernel Release: 2.6.0

NOTE: Versions of the linux-iscsi Initiator below 4.x are not compatible with the 2.6.x and higher kernel release.