Logical volumes not being recognized and mounted even varyonvg not work

After two lun removed from host by mistake, we were experiencing problems and so
I rebooted the system and now I'm having big problems with two LVs on my user-defined volume group.  The user-defined volume group consists
of three disks.  When I try to vary them on I get "PVREMOVED" for three
of the disks.  Incidentally, these two disks are external and went down due to accidentally removed and  went out.  Here is the output we get when we try to varyon the uservg volume group:

#  varyonvg datavg
PV Status:      hdisk3  00f63e0e510b4815        PVREMOVED
                hdisk4  00f63e0e510b48b4        PVREMOVED
                hdisk5  00f63e0e510b4954        PVACTIVE

varyonvg: Volume group uservg is varied on.

Response:

This is happening because the three external disks were marked
as removed when you took the power hit.  You had been forcing
the import and varyon of the volume group and the external disks
were forced to a removed state.

You should try making the three drives active with the following
commands:

  chpv -v a hdisk3
  chpv -v a hdisk4

After you run those commands, varyoff and varyon uservg and see
if that corrects the problem.

Raw Work ::::::

*******************************************************************************
*                                                                             *
*                                                                             *
*  Welcome to AIX Version 7.1!                                                *
*                                                                             *
*                                                                             *
*  Please see the README file in /usr/lpp/bos for information pertinent to    *
*  this release of the AIX Operating System.                                  *
*                                                                             *
*                                                                             *
*******************************************************************************
# lsfs
Name            Nodename   Mount Pt               VFS   Size    Options    Auto Accounting
/dev/hd4        —         /                      jfs2  1048576 —         yes  no
/dev/hd1        —         /home                  jfs2  524288  —         yes  no
/dev/hd2        —         /usr                   jfs2  15204352 —         yes  no
/dev/hd9var     —         /var                   jfs2  11534336 —         yes  no
/dev/hd3        —         /tmp                   jfs2  4194304 —         yes  no
/dev/hd11admin  —         /admin                 jfs2  524288  —         yes  no
/proc           —         /proc                  procfs —      —         yes  no
/dev/hd10opt    —         /opt                   jfs2  1048576 —         yes  no
/dev/livedump   —         /var/adm/ras/livedump  jfs2  524288  —         yes  no
/dev/cd0        —         /mnt                   cdrfs —      ro         no   no
/dev/fslv03     —         /data01                jfs2  —      rw         yes  no
/dev/fslv04     —         /data02                jfs2  —      rw         yes  no
# bash
ksh: bash:  not found.
# vi /etc/filesystems
# mount /data01
mount: 0506-324 Cannot mount /dev/fslv03 on /data01: There is a request to a device or address that does not exist.
# lspv
hdisk1          00f63e023551f39d                    None
hdisk0          00f63e0e32fddc85                    rootvg          active
hdisk2          00f63e0e510b49f3                    None
hdisk5          00f63e0e510b4954                    datavg
hdisk6          00f63e0e7061217b                    None
# lspv -l hdisk6
0516-320 : Physical volume 00f63e0e7061217b0000000000000000 is not assigned to
a volume group.

# lspv -l hdisk5
0516-010 : Volume group must be varied on; use varyonvg command.
# varyonvg datavg
0516-052 varyonvg: Volume group cannot be varied on without a
quorum. More physical volumes in the group must be active.
Run diagnostics on inactive PVs.

# mpio_get_config -Av
Frame id 0:
Storage Subsystem worldwide name: 600a0b80006e0fc4000000004d59db49
Controller count: 2
Partition count: 1
Partition 0:
Storage Subsystem Name = ‘DS5300_PRI’
hdisk#           LUN #   Ownership          User Label
hdisk2               4   B (preferred)      6
hdisk5               3   A (preferred)      5
hdisk6               5   A (preferred)      7
# hostname
NUSW2WDB1
# redefinevg -d hdisk5 datavg
# lspv
hdisk1          00f63e023551f39d                    None
hdisk0          00f63e0e32fddc85                    rootvg          active
hdisk2          00f63e0e510b49f3                    None
hdisk5          00f63e0e510b4954                    datavg
hdisk6          00f63e0e7061217b                    None
# lspv -l hdisk5
0516-010 : Volume group must be varied on; use varyonvg command.
# varyonvg datavg
0516-052 varyonvg: Volume group cannot be varied on without a
quorum. More physical volumes in the group must be active.
Run diagnostics on inactive PVs.
# cfgmgr
# lspv
hdisk1          00f63e023551f39d                    None
hdisk0          00f63e0e32fddc85                    rootvg          active
hdisk2          00f63e0e510b49f3                    None
hdisk3          00f63e0e510b4815                    None
hdisk4          00f63e0e510b48b4                    None
hdisk5          00f63e0e510b4954                    datavg
hdisk6          00f63e0e7061217b                    None
# bootinfo -s hdisk5
256000
# varyonvg -f datavg
PV Status:      hdisk5  00f63e0e510b4954        PVACTIVE
00f63e0e510b48b4        NONAME
00f63e0e510b4815        NONAME
varyonvg: Volume group datavg is varied on.
# lspv
hdisk1          00f63e023551f39d                    None
hdisk0          00f63e0e32fddc85                    rootvg          active
hdisk2          00f63e0e510b49f3                    None
hdisk3          00f63e0e510b4815                    None
hdisk4          00f63e0e510b48b4                    None
hdisk5          00f63e0e510b4954                    datavg          active
hdisk6          00f63e0e7061217b                    None
# lspv -l hdisk5
hdisk5:
LV NAME               LPs     PPs     DISTRIBUTION          MOUNT POINT
fslv04                499     499     100..100..99..100..100 /data02
# mount /data02
Replaying log for /dev/fslv04.
mount: 0506-324 Cannot mount /dev/fslv04 on /data02: The media is not formatted or the format is not correct.
0506-342 The superblock on /dev/fslv04 is dirty.  Run a full fsck to fix.
# varryoffvg datavg
ksh: varryoffvg:  not found.
# varyoffvg datavg
#  lspv -l hdisk5
0516-010 : Volume group must be varied on; use varyonvg command.
# redefinevg -d hdisk5 datavg
# lspv
hdisk1          00f63e023551f39d                    None
hdisk0          00f63e0e32fddc85                    rootvg          active
hdisk2          00f63e0e510b49f3                    None
hdisk3          00f63e0e510b4815                    datavg
hdisk4          00f63e0e510b48b4                    datavg
hdisk5          00f63e0e510b4954                    datavg
hdisk6          00f63e0e7061217b                    None
#  mpio_get_config -Av
Frame id 0:
Storage Subsystem worldwide name: 600a0b80006e0fc4000000004d59db49
Controller count: 2
Partition count: 1
Partition 0:
Storage Subsystem Name = ‘DS5300_PRI’
hdisk#           LUN #   Ownership          User Label
hdisk2               4   B (preferred)      6
hdisk3               0   A (preferred)      3
hdisk4               1   B (preferred)      4
hdisk5               3   A (preferred)      5
hdisk6               5   A (preferred)      7
# rmdev -Rdl hdisk2
hdisk2 deleted
# rmdev -Rdl hdisk6
hdisk6 deleted
# cfgmgr
Method error (/usr/lib/methods/cfgscsidisk -l hdisk3 ):
0514-082 The requested function could only be performed for some
of the specified paths.
Method error (/usr/lib/methods/cfgscsidisk -l hdisk4 ):
0514-082 The requested function could only be performed for some
of the specified paths.
# lspv
hdisk1          00f63e023551f39d                    None
hdisk0          00f63e0e32fddc85                    rootvg          active
hdisk3          00f63e0e510b4815                    datavg
hdisk4          00f63e0e510b48b4                    datavg
hdisk5          00f63e0e510b4954                    datavg
# varyonvg datavg
PV Status:      hdisk3  00f63e0e510b4815        PVREMOVED
hdisk4  00f63e0e510b48b4        PVREMOVED
hdisk5  00f63e0e510b4954        PVACTIVE
varyonvg: Volume group datavg is varied on.
# varyoffvg datavg
You have mail in /usr/spool/mail/root
# importvg
0516-604 importvg: Physical volume name not entered.
Usage: importvg [ [ [-V MajorNumber] [-y VGname] [-f] [-c] [-x] ] | [-L VGname] ]
[-n] [-F] [-R] [-O] PVname
Imports the definition of a volume group.
#
ksh: ^[[A^[[B:  not found.
# importvg -y datavg hdisk3
0516-360 getvgname: The device name is already used; choose a
different name.
0516-776 importvg: Cannot import hdisk3 as datavg.
#  varyonvg datavg
PV Status:      hdisk3  00f63e0e510b4815        PVREMOVED
hdisk4  00f63e0e510b48b4        PVREMOVED
hdisk5  00f63e0e510b4954        PVACTIVE
varyonvg: Volume group datavg is varied on.
# lspv -l hdisk3
hdisk3:
LV NAME               LPs     PPs     DISTRIBUTION          MOUNT POINT
fslv03                499     499     100..100..99..100..100 /data01
# lspv -l hdisk4
hdisk4:
LV NAME               LPs     PPs     DISTRIBUTION          MOUNT POINT
loglv00               1       1       00..01..00..00..00    N/A
fslv03                1       1       00..01..00..00..00    /data01
fslv04                1       1       00..01..00..00..00    /data02
# lspv -l hdisk5
hdisk5:
LV NAME               LPs     PPs     DISTRIBUTION          MOUNT POINT
fslv04                499     499     100..100..99..100..100 /data02
# mount /data01
mount: 0506-324 Cannot mount /dev/fslv03 on /data01: There is an input or output error.
# chpv -v a hdisk3
# chpv -v a hdisk4
# varyoffvg datavg
# varyonvg datavg
# mount /data01
Replaying log for /dev/fslv03.
# mount /data02
# df -g
Filesystem    GB blocks      Free %Used    Iused %Iused Mounted on
/dev/hd4           0.50      0.25   51%    10624    16% /
/dev/hd2           7.25      5.14   30%    49315     4% /usr
/dev/hd9var        5.50      3.85   31%     9868     2% /var
/dev/hd3           2.00      1.99    1%      105     1% /tmp
/dev/hd1           0.25      0.25    1%        8     1% /home
/dev/hd11admin      0.25      0.25    1%        5     1% /admin
/proc                 –         –    –         –     –  /proc
/dev/hd10opt       0.50      0.20   60%     8331    15% /opt
/dev/livedump      0.25      0.25    1%        7     1% /var/adm/ras/livedump
/dev/fslv03      250.00    171.20   32%    41784     1% /data01
/dev/fslv04      250.00    249.96    1%        4     1% /data02
#

Leave a Comment

Your email address will not be published. Required fields are marked *

CAPTCHA * Time limit is exhausted. Please reload the CAPTCHA.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Scroll to Top