Tag Archives: check

Check and list luns attached to HBA in RHEL6

This article will show you the mapping from physical HBA card to luns, I use SAN as example below, in general, it’s also applys to any other devices whichever use sysfs, for example direct sas connect.

[root@RHEL6 scsi_host]# lspci | grep Fibre
05:00.0 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre Channel Host Adapter (rev 03)
05:00.1 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre Channel Host Adapter (rev 03)
[root@RHEL6 scsi_host]# lspci -v -s 05:00.0
05:00.0 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre Channel Host Adapter (rev 03)
        Subsystem: Hewlett-Packard Company Device 338f
        Physical Slot: 1
        Flags: bus master, fast devsel, latency 0, IRQ 40
        Memory at f7ff0000 (64-bit, non-prefetchable) [size=4K]
        Memory at f7fe0000 (64-bit, non-prefetchable) [size=16K]
        I/O ports at 5000 [size=256]
        [virtual] Expansion ROM at f1700000 [disabled] [size=256K]
        Capabilities: [58] Power Management version 3
        Capabilities: [60] MSI: Enable- Count=1/16 Maskable+ 64bit+
        Capabilities: [78] MSI-X: Enable- Count=32 Masked-
        Capabilities: [84] Vital Product Data
        Capabilities: [94] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [12c] Power Budgeting <?>
        Kernel driver in use: lpfc
        Kernel modules: lpfc

[root@RHEL6 scsi_host]# ls -l /sys/class/scsi_host
total 0
lrwxrwxrwx. 1 root root 0 Jun 20 18:17 host0 -> ../../devices/pci0000:00/0000:00:02.2/0000:03:00.0/host0/scsi_host/host0
lrwxrwxrwx. 1 root root 0 Jun 20 18:17 host1 -> ../../devices/pci0000:00/0000:00:03.0/0000:05:00.0/host1/scsi_host/host1
lrwxrwxrwx. 1 root root 0 Jun 20 18:17 host2 -> ../../devices/pci0000:00/0000:00:03.0/0000:05:00.1/host2/scsi_host/host2

[root@RHEL6 scsi_host]# find /sys/class/pci_bus/0000\:05/device/0000\:05\:00.0/host*/rport-*/target*/*/state | awk -F'/' '{print $11}' | sort
1:0:0:0
1:0:0:10
1:0:0:11
1:0:0:12
1:0:0:13
1:0:0:14
1:0:0:15
1:0:0:16
1:0:0:33
1:0:1:0
1:0:1:10
1:0:1:11
1:0:1:12
1:0:1:13
1:0:1:14
1:0:1:15
1:0:1:16
1:0:1:33
1:0:2:0
1:0:2:31
1:0:2:32
1:0:2:33
1:0:2:34
1:0:3:0
1:0:3:1
1:0:3:2
1:0:3:33
1:0:4:0
1:0:4:31
1:0:4:32
1:0:4:33
1:0:4:34
1:0:5:0
1:0:5:1
1:0:5:2
1:0:5:33
1:0:5:34
[root@RHEL6 scsi_host]# find /sys/class/pci_bus/0000\:05/device/0000\:05\:00.1/host*/rport-*/target*/*/state | awk -F'/' '{print $11}' | sort
2:0:0:0
2:0:0:1
2:0:0:2
2:0:1:0
2:0:1:10
2:0:1:11
2:0:1:12
2:0:1:13
2:0:1:14
2:0:1:15
2:0:1:16
2:0:1:33
2:0:2:0
2:0:2:31
2:0:2:32
2:0:2:33
2:0:2:34
2:0:3:0
2:0:3:10
2:0:3:11
2:0:3:12
2:0:3:13
2:0:3:14
2:0:3:15
2:0:3:16
2:0:3:33
2:0:4:0
2:0:4:31
2:0:4:32
2:0:4:33
2:0:4:34
2:0:5:0
2:0:5:1
2:0:5:2
2:0:5:33
[root@RHEL6 scsi_host]# cat /proc/scsi/scsi | grep scsi2
Host: scsi2 Channel: 00 Id: 02 Lun: 00
Host: scsi2 Channel: 00 Id: 02 Lun: 31
Host: scsi2 Channel: 00 Id: 02 Lun: 32
Host: scsi2 Channel: 00 Id: 04 Lun: 00
Host: scsi2 Channel: 00 Id: 04 Lun: 31
Host: scsi2 Channel: 00 Id: 04 Lun: 32
Host: scsi2 Channel: 00 Id: 05 Lun: 00
Host: scsi2 Channel: 00 Id: 05 Lun: 01
Host: scsi2 Channel: 00 Id: 05 Lun: 02
Host: scsi2 Channel: 00 Id: 00 Lun: 00
Host: scsi2 Channel: 00 Id: 00 Lun: 01
Host: scsi2 Channel: 00 Id: 00 Lun: 02
Host: scsi2 Channel: 00 Id: 01 Lun: 00
Host: scsi2 Channel: 00 Id: 01 Lun: 10
Host: scsi2 Channel: 00 Id: 01 Lun: 11
Host: scsi2 Channel: 00 Id: 01 Lun: 12
Host: scsi2 Channel: 00 Id: 01 Lun: 13
Host: scsi2 Channel: 00 Id: 01 Lun: 14
Host: scsi2 Channel: 00 Id: 01 Lun: 15
Host: scsi2 Channel: 00 Id: 01 Lun: 16
Host: scsi2 Channel: 00 Id: 03 Lun: 00
Host: scsi2 Channel: 00 Id: 03 Lun: 10
Host: scsi2 Channel: 00 Id: 03 Lun: 11
Host: scsi2 Channel: 00 Id: 03 Lun: 12
Host: scsi2 Channel: 00 Id: 03 Lun: 13
Host: scsi2 Channel: 00 Id: 03 Lun: 14
Host: scsi2 Channel: 00 Id: 03 Lun: 15
Host: scsi2 Channel: 00 Id: 03 Lun: 16
Host: scsi2 Channel: 00 Id: 05 Lun: 33
Host: scsi2 Channel: 00 Id: 04 Lun: 33
Host: scsi2 Channel: 00 Id: 03 Lun: 33
Host: scsi2 Channel: 00 Id: 01 Lun: 33
Host: scsi2 Channel: 00 Id: 02 Lun: 33
Host: scsi2 Channel: 00 Id: 02 Lun: 34
Host: scsi2 Channel: 00 Id: 04 Lun: 34
[root@RHEL6 scsi_host]# cat /proc/scsi/scsi | grep scsi1
Host: scsi1 Channel: 00 Id: 02 Lun: 00
Host: scsi1 Channel: 00 Id: 02 Lun: 31
Host: scsi1 Channel: 00 Id: 02 Lun: 32
Host: scsi1 Channel: 00 Id: 04 Lun: 00
Host: scsi1 Channel: 00 Id: 04 Lun: 31
Host: scsi1 Channel: 00 Id: 04 Lun: 32
Host: scsi1 Channel: 00 Id: 05 Lun: 00
Host: scsi1 Channel: 00 Id: 05 Lun: 01
Host: scsi1 Channel: 00 Id: 05 Lun: 02
Host: scsi1 Channel: 00 Id: 03 Lun: 00
Host: scsi1 Channel: 00 Id: 03 Lun: 01
Host: scsi1 Channel: 00 Id: 03 Lun: 02
Host: scsi1 Channel: 00 Id: 00 Lun: 00
Host: scsi1 Channel: 00 Id: 00 Lun: 10
Host: scsi1 Channel: 00 Id: 00 Lun: 11
Host: scsi1 Channel: 00 Id: 00 Lun: 12
Host: scsi1 Channel: 00 Id: 00 Lun: 13
Host: scsi1 Channel: 00 Id: 00 Lun: 14
Host: scsi1 Channel: 00 Id: 00 Lun: 15
Host: scsi1 Channel: 00 Id: 00 Lun: 16
Host: scsi1 Channel: 00 Id: 01 Lun: 00
Host: scsi1 Channel: 00 Id: 01 Lun: 10
Host: scsi1 Channel: 00 Id: 01 Lun: 11
Host: scsi1 Channel: 00 Id: 01 Lun: 12
Host: scsi1 Channel: 00 Id: 01 Lun: 13
Host: scsi1 Channel: 00 Id: 01 Lun: 14
Host: scsi1 Channel: 00 Id: 01 Lun: 15
Host: scsi1 Channel: 00 Id: 01 Lun: 16
Host: scsi1 Channel: 00 Id: 05 Lun: 33
Host: scsi1 Channel: 00 Id: 00 Lun: 33
Host: scsi1 Channel: 00 Id: 01 Lun: 33
Host: scsi1 Channel: 00 Id: 03 Lun: 33
Host: scsi1 Channel: 00 Id: 04 Lun: 33
Host: scsi1 Channel: 00 Id: 02 Lun: 33
Host: scsi1 Channel: 00 Id: 05 Lun: 34
Host: scsi1 Channel: 00 Id: 02 Lun: 34
Host: scsi1 Channel: 00 Id: 04 Lun: 34
[root@RHEL6 scsi_host]# find   /sys/class/pci_bus/0000\:05/device/0000\:05\:00.0/host*/rport-*/target*/*/block/*/stat | awk -F'/' '{print $11,$13}'
1:0:0:0 sdb
1:0:0:10 sdc
1:0:0:11 sdd
1:0:0:12 sde
1:0:0:13 sdf
1:0:0:14 sdg
1:0:0:15 sdh
1:0:0:16 sdi
1:0:1:0 sdj
1:0:1:10 sdk
1:0:1:11 sdl
1:0:1:12 sdm
1:0:1:13 sdn
1:0:1:14 sdo
1:0:1:15 sdp
1:0:1:16 sdq
1:0:2:0 sdr
1:0:2:31 sds
1:0:2:32 sdt
1:0:2:33 sdbi
1:0:2:34 sdbk
1:0:3:0 sdu
1:0:3:1 sdv
1:0:3:2 sdw
1:0:4:0 sdx
1:0:4:31 sdy
1:0:4:32 sdz
1:0:4:33 sdbh
1:0:4:34 sdbl
1:0:5:0 sdaa
1:0:5:1 sdab
1:0:5:2 sdac
[root@RHEL6 scsi_host]# find   /sys/class/pci_bus/0000\:05/device/0000\:05\:00.1/host*/rport-*/target*/*/block/*/stat | awk -F'/' '{print $11,$13}'
2:0:0:0 sdad
2:0:0:1 sdae
2:0:0:2 sdaf
2:0:1:0 sdag
2:0:1:10 sdah
2:0:1:11 sdai
2:0:1:12 sdaj
2:0:1:13 sdak
2:0:1:14 sdal
2:0:1:15 sdam
2:0:1:16 sdan
2:0:2:0 sdao
2:0:2:31 sdap
2:0:2:32 sdaq
2:0:2:33 sdbg
2:0:2:34 sdbj
2:0:3:0 sdar
2:0:3:10 sdas
2:0:3:11 sdat
2:0:3:12 sdau
2:0:3:13 sdav
2:0:3:14 sdaw
2:0:3:15 sdax
2:0:3:16 sday
2:0:4:0 sdaz
2:0:4:31 sdba
2:0:4:32 sdbb
2:0:4:33 sdbf
2:0:4:34 sdbm
2:0:5:0 sdbc
2:0:5:1 sdbd
2:0:5:2 sdbe
[root@RHEL6 scsi_host]# udevadm info --query=path --name /dev/sdad
/devices/pci0000:00/0000:00:03.0/0000:05:00.1/host2/rport-2:0-2/target2:0:0/2:0:0:0/block/sdad
[root@RHEL6 scsi_host]# df -h
Filesystem          Size  Used Avail Use% Mounted on
/dev/sda3            50G   20G   27G  43% /
tmpfs                64G 1012K   64G   1% /dev/shm
/dev/sda1           485M   40M  420M   9% /boot
/dev/sda6            92G  9.9G   77G  12% /home
/dev/sda5           7.9G  147M  7.4G   2% /tmp
/dev/mapper/mpathb   99G   17G   82G  18% /oracle
/dev/mapper/mpathc  985G  907G   78G  93% /data01
/dev/mapper/mpathd  985G  911G   74G  93% /data02
/dev/mapper/mpathe  985G  942G   43G  96% /data03
/dev/mapper/mpathf  985G  933G   52G  95% /data04
/dev/mapper/mpathn  985G  920G   65G  94% /data05
/dev/mapper/mpathh  985G  927G   58G  95% /data06
/dev/mapper/mpathi  985G  937G   48G  96% /data07
/dev/mapper/mpathj  985G  895G   90G  91% /data08
/dev/mapper/mpatho  985G  966G   19G  99% /data09
/dev/mapper/mpathg  985G  828G  157G  85% /data10
/dev/mapper/mpathp  985G  545G  441G  56% /data11
/dev/mapper/mpathq  985G   87M  985G   1% /data12

[root@RHEL6 scsi_host]# for port in /sys/class/fc_host/host[0-9]/port_name; { echo -n "$port : "; cat $port; }
/sys/class/fc_host/host1/port_name : 0x1000a0481ce4f1da
/sys/class/fc_host/host2/port_name : 0x1000a0481ce4f1db
[root@RHEL6 scsi_host]#

First, use lspci get HBA card  info installed on the host

# lspci | grep Fibre
15:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
15:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)

HBA detail info

# lspci -v -s 15:00.0
15:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
    Subsystem: QLogic Corp. Device 015d
    Physical Slot: 2
    Flags: bus master, fast devsel, latency 0, IRQ 24
    I/O ports at 2200 [size=256]
    Memory at 97b00000 (64-bit, non-prefetchable) [size=16K]
    Expansion ROM at 90000000 [disabled] [size=256K]
    Capabilities: [44] Power Management version 3
    Capabilities: [4c] Express Endpoint, MSI 00
    Capabilities: [88] MSI: Enable- Count=1/32 Maskable- 64bit+
    Capabilities: [98] Vital Product Data
    Capabilities: [a0] MSI-X: Enable+ Count=2 Masked-
    Capabilities: [100] Advanced Error Reporting
    Capabilities: [138] Power Budgeting <?>
    Kernel driver in use: qla2xxx
    Kernel modules: qla2xxx

It tells you that one HBA card in PCI slot2, two FC ports

Find pci slot and scsi_host mapping

# ls -l /sys/class/scsi_host
total 0
...
lrwxrwxrwx 1 root root 0 Oct  9 12:58 host4 -> ../../devices/pci0000:00/0000:00:1f.5/host4/scsi_host/host4
lrwxrwxrwx 1 root root 0 Oct  9 12:58 host5 -> ../../devices/pci0000:00/0000:00:03.0/0000:15:00.0/host5/scsi_host/host5
lrwxrwxrwx 1 root root 0 Oct  9 12:58 host6 -> ../../devices/pci0000:00/0000:00:03.0/0000:15:00.1/host6/scsi_host/host6

You can easily tell that the first port of pcs slot 2(15:00.0) mapped to host5, the other mapped to host6

Find target luns by HBA port

Once you know the pci info of a HBA card, then you can find its port0 target luns or SAN devices.

Note 15:00.0 is used in this case

#find /sys/class/pci_bus/0000\:15/device/0000\:15\:00.0/host*/rport-*/target*/*/state | awk -F'/' '{print $11}' | sort
...
5:0:0:0
5:0:0:1
5:0:0:10
5:0:0:11
5:0:0:2
5:0:0:3
5:0:0:31
5:0:0:4
5:0:0:5
...

It should be consistent with the devices in /proc/scsi/scsi

#cat /proc/scsi/scsi | grep scsi5
...
Host: scsi5 Channel: 00 Id: 04 Lun: 04
Host: scsi5 Channel: 00 Id: 04 Lun: 05
Host: scsi5 Channel: 00 Id: 04 Lun: 06
Host: scsi5 Channel: 00 Id: 04 Lun: 07
Host: scsi5 Channel: 00 Id: 04 Lun: 08
Host: scsi5 Channel: 00 Id: 04 Lun: 09
Host: scsi5 Channel: 00 Id: 04 Lun: 10
Host: scsi5 Channel: 00 Id: 04 Lun: 11
Host: scsi5 Channel: 00 Id: 04 Lun: 31
...

Note: if use the command for sas direct attached devices, change ‘rport’ to ‘port’, same applies to the example below.

Find block devices

If you are only interested in block devices, like tape drive, disk lun or cd rom, here is a way similar.

# find   /sys/class/pci_bus/0000\:15/device/0000\:15\:00.0/host*/rport-*/target*/*/block/*/stat | awk -F'/' '{print $11,$13}'
5:0:0:0 sdb
5:0:0:1 sdc
5:0:0:10 sdl
5:0:0:11 sdm
5:0:0:2 sdd
5:0:0:3 sde
5:0:0:4 sdf
5:0:0:5 sdg
5:0:0:6 sdh

Reverse search, find the physical port that a lun connected to

/proc/scsi/scsi doesn’t tell you which physical port target luns are connected to In the reverse look, for a given device name, for example /dev/sdd, how do I know which hba port it connected to?

# udevadm info --query=path --name /dev/sdd
/devices/pci0000:00/0000:00:03.0/0000:15:00.0/host5/rport-5:0-0/target5:0:0/5:0:0:2/block/sdd

Is is clear?

Or
Multipath also can tell you some hint

multipath -ll | grep sdd
  `- 5:0:0:2  sdd  8:48    active ready running

Or, look into /dev/disk/by-path/ tree
...
lrwxrwxrwx 1 root root 10 Aug 15 16:49 /dev/disk/by-path/pci-0000:15:00.1-fc-0x22430080e524ebac-lun-4 -> ../../sdcx
lrwxrwxrwx 1 root root 10 Aug 15 16:49 /dev/disk/by-path/pci-0000:15:00.1-fc-0x22430080e524ebac-lun-5 -> ../../sdcy

Get HBA WWNA info:

# for port in /sys/class/fc_host/host[0-9]/port_name; { echo -n "$port : "; cat $port; }
/sys/class/fc_host/host5/port_name : 0x21000024ff3434e4
/sys/class/fc_host/host6/port_name : 0x21000024ff3434e5

Dynamically insert and remove SCSI devices

If a newer kernel and the /proc file system is running, a non-busy device can be removed and installed ‘on the fly’.

To hot remove a SCSI device:

    echo 1 > /sys/class/scsi_device/h:c:t:l/device/delete
    or
    echo 1 > /sys/block/<dev>/device/delete
    where <dev> is like sda or sdb etc..
    old way
    echo "scsi remove-single-device a b c d" > /proc/scsi/scsi

and similar, to hot add a SCSI device, do

    echo "c t l" >  /sys/class/scsi_host/host<h>/scan
    or use wildcard like below
    echo "- - -" > /sys/class/scsi_host/host<h>/scan

    old way
    echo "scsi add-single-device h c t l" > /proc/scsi/scsi

where

          h == hostadapter id (first one being 0)
          c == SCSI channel on hostadapter (first one being 0)
          t == ID
          l == LUN (first one being 0)

How to check and install missing perl modules

  • Check if module is installed. Errors mean missing module.

    # perl -MModule::Name -e 1
    
  • See documentation of the module if installed.
    # perldoc Module::Name
    
  • Open CPAN shell
    # perl -MCPAN -e shell
    
  • To reconfigure the shell if needed.
    cpan>o conf init
    
  • Install an available module.
    cpan> install HTML::Template
  • You can run the Perl CPAN module via command line perl and get it installed in a single line:

# perl -MCPAN -e ‘install HTML::Template

  • Force install if test fails.
    cpan> force install Module::Name
    
  • To manual install perl modules. Unzip and go to module directory.

# tar -zxvf HTML-Template-2.8.tar.gz

  • # perl Makefile.PL
    # make
    # make test
    # make install