All posts by Prashant Kumar

Uncategorized

Recreating missing virtual disk (VMDK) Descriptor File Vmware

Leave a comment

Recreating missing virtual disk (VMDK) Descriptor File

Problem-

You just noticed that virtual machine’s VMDK files are missing and lost somehow. The reason, not able to start VM and when you try to add existing HDD – existing VMDK and -flat. VMDK files are not available to attach with Virtual Machine too.

Solution –

We need to recreate missing VMDK files, to add VMDK files and start Virtual Machine.

  1. Login to ESXI PS with Putty. Make sure, SSH service is enabled and running for the PS.
  2. Switch to the directory where your virtual machine is located.

cd /vmfs/volumes/<<datastore>>/VM-Directory

cd /vmfs/volumes/5d406ca3-62654bd0-75fd-e4434b75ed38/APP-U1-63-25

3. Run below command to identify and record the exact size of the existing flat vmdk file for the VM.

ls -ltr *

-rw——-    1 root     root     16106127360 Jun 21 12:20 APP-U1-63-25-flat.vmdk

-rw——-    1 root     root     214748364800 Jun 21 10:49 APP-U1-63-25_1-flat.vmdk

4. Run the vmkfstools to create new virtual disk, i.e VMDK (Descriptor File)

vmkfstools -c 16106127360 -d thin -a lsilogic APP-U1-63-25-OS.vmdk

vmkfstools -c 214748364800 -d thin -a lsilogic APP-U1-63-25-DATA.vmdk

5. As mentioned, there are 2 new files created: APP-U1-63-25-OS.vmdk and APP-U1-63-25-DATA.vmdk are created as a result.

ls -ltr

–rw——-    1 root     root     16106127360 Jun 21 19:21 APP-U1-63-25-OS-flat.vmdk

-rw——-    1 root     root     214748364800 Jun 21 19:21 APP-U1-63-25-DATA-flat.vmdk

-rw——-    1 root     root           546 Jun 21 21:05 APP-U1-63-25-OS.vmdk 

-rw——-    1 root     root           550 Jun 21 21:05 APP-U1-63-25-DATA.vmdk 

-rw——-    1 root     root     214748364800 Jun 22 05:08 APP-U1-63-25_1-flat.vmdk

-rw——-    1 root     root     16106127360 Jun 22 05:08 APP-U1-63-25-flat.vmdk

6. We need to Rename APP-U1-63-25-OS.vmdk  and APP-U1-63-25-DATA.vmdk to the name that is needed to match the orphaned -flat file.

mv APP-U1-63-25-OS.vmdk APP-U1-63-25.vmdk

mv APP-U1-63-25-DATA.vmdk APP-U1-63-25_1.vmdk

7. Now, it is final stage where editing descriptor file (APP-U1-63-25.vmdk and APP-U1-63-25_1.vmdk)

find the line with RW ####### and change the name of the -flat to match the orphaned -flat file you have. So, I changed to APP-U1-63-25-flat.vmdk and APP-U1-63-25_1-flat.vmdk respectively for both disks, as you can see in the image below.

vi APP-U1-63-25.vmdk

vi APP-U1-63-25_1.vmdk

8. You can now delete the below files created earlier. These are not required anymore.

–rw——-    1 root     root     16106127360 Jun 21 19:21 APP-U1-63-25-OS-flat.vmdk

-rw——-    1 root     root     214748364800 Jun 21 19:21 APP-U1-63-25-DATA-flat.vmdk

9. Attach the respective VMDK files to the virtual machine as earlier.

10. Now, All set and good to power on the virtual machine.

Linux

passwd: Authentication token manipulation error | RHEL 6

Leave a comment

Problem described as below-

Getting passwd: Authentication token manipulation error on  RHEL6 Machine.

[root@ip-linuxbox~]# passwd user1
Changing password for user user1.
New password:
Retype new password:
passwd: Authentication token manipulation error

[root@ip-linuxbox~]# passwd -u user1 [Tried to unlock the account password.]
Unlocking password for user user1.
passwd: Libuser error at line: 179 – error creating `/etc/passwd+’: Permission denied.
passwd: Error (password not set?)  [Getting Permission denied error hence passwd is not getting changed/updated. Hence, we need to restore the permission on passwd.]

[root@ip-linuxbox~]# chage -l user1
Last password change : Apr 03, 2019
Password expires : never
Password inactive : never
Account expires : never
Minimum number of days between password change : 0
Maximum number of days between password change : 99999
Number of days of warning before password expires : 7

Solution-

[root@ip-linuxbox~]# rpm –setperms passwd  [Restore permission on passwd file.]

[root@ip-linuxbox~]# which passwd | xargs chmod u+s [Setting required permission on passwd file.]

[root@ip-linuxbox ~]# restorecon /etc/* [ if SELinux context is properly set for, it will fix the same.]

[root@ip-linuxbox~]# passwd user1
Changing password for user user1.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.

[root@ip-linuxbox~]# chage -l user1
Last password change : Jul 28, 2020
Password expires : never
Password inactive : never
Account expires : never
Minimum number of days between password change : 0
Maximum number of days between password change : 99999
Number of days of warning before password expires : 7

Aix, HP-UX, Linux, Solaris

File System shows 100% occupied but du tells different and still has Unused Spaces.

1 Comment

We have faced an issue as below mentioned – found /oracle mount-point showing 100% as Used. But getting different size values for folders under /oracle mount-point, when we executed du -gs .

bash-4.4# df -g
Filesystem GB blocks Free %Used Iused %Iused Mounted on
/dev/hd4 4.00 1.39 66% 20686 6% /
/dev/hd2 4.00 1.02 75% 45679 16% /usr
/dev/hd9var 2.00 1.01 50% 16318 7% /var
/dev/hd3 2.00 1.83 9% 407 1% /tmp
/dev/hd1 1.00 0.16 85% 6124 12% /home
/dev/hd11admin 0.25 0.25 1% 5 1% /admin
/proc – – – – – /proc
/dev/hd10opt 0.50 0.07 87% 12979 44% /opt
/dev/livedump 0.25 0.25 1% 4 1% /var/adm/ras/livedump
/dev/fslv00 198.00 0.01 100% 440205 16% /oracle

bash-4.4# cd /oracle/
bash-4.4# du -gs *
0.00 AutoDeployment
5.45 Oracle
0.03 Patch
7.54 data01
11.67 fmw_12.1.3.0.0_wls
0.01 jboss
0.20 jdk
0.00 lost+found
0.92 wls1221

You may notice 100 % utilization for /oracle mount-point on the “df -g” output , but with “du -gs ” We found the files did not occupy the entire space.

It may be because of open files in the file-system. These open files hold space on the file-system to complete there execution, once the write operation is completed. 

To overcome, from this issue, We need to follow below two steps:-

  1. We need to find all such processes still running but using deleted files, in result – /oracle mount-point is showing 100% as Used.

bash-4.4# fuser -dV /oracle/
/oracle/:
inode=670047 size=5242722 fd=389 5570776
inode=1165305 size=20278 fd=1 8061106
inode=1165313 size=182335565824 fd=1 8716486
inode=1165305 size=20278 fd=1 9044152
inode=1165313 size=182335565824 fd=1 9371672
inode=1165305 size=20278 fd=1 11141354
inode=669981 size=5514335 fd=1 13041898


2. We have to kill all such processes as found in above command.

bash-4.4# kill -9 5570776
bash-4.4# kill -9 8061106
bash-4.4# kill -9 8716486
bash-4.4# kill -9 9044152
bash-4.4# kill -9 9371672
bash-4.4# kill -9 11141354
bash-4.4# kill -9 13041898

bash-4.4# fuser -dV /oracle/
/oracle/:

bash-4.4# df -g
Filesystem GB blocks Free %Used Iused %Iused Mounted on
/dev/hd4 4.00 1.39 66% 20686 6% /
/dev/hd2 4.00 1.02 75% 45679 16% /usr
/dev/hd9var 2.00 1.01 50% 16318 7% /var
/dev/hd3 2.00 1.83 9% 407 1% /tmp
/dev/hd1 1.00 0.16 85% 6124 12% /home
/dev/hd11admin 0.25 0.25 1% 5 1% /admin
/proc – – – – – /proc
/dev/hd10opt 0.50 0.07 87% 12979 44% /opt
/dev/livedump 0.25 0.25 1% 4 1% /var/adm/ras/livedump
/dev/fslv00 198.00 170.01 15% 440205 1% /oracle
bash-4.4# cd /oracle/
bash-4.4# du -gs *
0.00 AutoDeployment
5.45 Oracle
0.03 Patch
7.54 data01
11.67 fmw_12.1.3.0.0_wls
0.01 jboss
0.20 jdk
0.00 lost+found
0.92 wls1221

Solaris

What process is listening on a certain port on Solaris? / How to find out which process listens on certain port on Solaris?

Leave a comment

Scenario: I’m looking for PID for which PORT 2817 is using here in Solaris 11.

-bash-3.2# netstat -an | grep 2817
*.2817 *.* 0 0 49152 0 LISTEN
10.0.50.81.2817 10.0.50.81.37374 49152 0 49152 0 CLOSE_WAIT
10.0.50.81.2817 10.0.50.81.35510 49152 0 49152 0 CLOSE_WAIT
10.0.50.81.2817 10.0.50.81.34478 49152 0 49152 0 CLOSE_WAIT

Here is a script, I found somewhere which actually works. This script worked to find PID from port number, If lsof command/utility is not available with server.

Save this script as Port_check.sh and grant execute permissions using chmod 777 to this script.

————————————————————————————-

#!/bin/ksh

line=’———————————————‘
pids=$(/usr/bin/ps -ef | sed 1d | awk ‘{print $2}’)

if [ $# -eq 0 ]; then
read ans?”Enter port you would like to know pid for: ”
else
ans=$1
fi

for f in $pids
do
/usr/proc/bin/pfiles $f 2>/dev/null | /usr/xpg4/bin/grep -q “port: $ans”
if [ $? -eq 0 ]; then
echo $line
echo “Port: $ans is being used by PID:\c”
/usr/bin/ps -ef -o pid -o args | egrep -v “grep|pfiles” | grep $f
fi
done
exit 0

————————————————————————————-
Now the syntax of using this script is
./Port_check.sh <port number>

For example, to find which process is using TCP port 2817,

-bash-3.2# ./Port_check.sh 2817
———————————————
Port: 2817 is being used by PID:26222 /data01/IBM/WebSphere/AppServer/java_1.7_64/bin/sparcv9/java -XX:+UnlockDiagnos

So, Here is PID 26222 using port 2817.

Thanks…

Aix

AIX: rootvg/disk mirroring

Leave a comment

bash-4.2# bootinfo -s hdisk0
140013
bash-4.2# bootinfo -s hdisk1
140013
bash-4.2# bootinfo -s hdisk2
140013

bash-4.2# lspv
hdisk0 002b012f397c20ce None
hdisk1 002afe4f2b4c3fdb rootvg active
hdisk2 002b016f09313544 ppmvg active
bash-4.2#

bash-4.2# lsvg -p rootvg
rootvg:
PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION
hdisk1 active 546 187 10..00..00..85..92

bash-4.2# extendvg rootvg hdisk0
0516-1398 extendvg: The physical volume hdisk0, appears to belong to
another volume group. Use the force option to add this physical volume
to a volume group.
0516-792 extendvg: Unable to extend volume group.

bash-4.2# extendvg -f rootvg hdisk0

bash-4.2# lsvg -p rootvg
rootvg:
PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION
hdisk1 active 546 187 10..00..00..85..92
hdisk0 active 546 546 110..109..109..109..109

bash-4.2# mirrorvg rootvg hdisk0
0516-1804 chvg: The quorum change takes effect immediately.
0516-1296 lresynclv: Unable to completely resynchronize volume.
The logical volume has bad-block relocation policy turned off.
This may have caused the command to fail.
0516-934 /usr/sbin/syncvg: Unable to synchronize logical volume hd5.
0516-934 /usr/sbin/syncvg: Unable to synchronize logical volume hd2.
0516-934 /usr/sbin/syncvg: Unable to synchronize logical volume dppmiaslv.
0516-934 /usr/sbin/syncvg: Unable to synchronize logical volume data01lv.
0516-932 /usr/sbin/syncvg: Unable to synchronize volume group rootvg.
0516-1126 mirrorvg: rootvg successfully mirrored, user should perform
bosboot of system to initialize boot records. Then, user must modify
bootlist to include: hdisk1 hdisk0.

bash-4.2# bosboot -ad /dev/hdisk0

bosboot: Boot image is 55324 512 byte blocks.

bash-4.2# bootlist -m normal -o
hdisk1 pathid=0

bash-4.2# bootlist -m normal hdisk1 hdisk0

bash-4.2# bootlist -m normal -o
hdisk1 blv=hd5 pathid=0
hdisk0 blv=hd5 pathid=0

bash-4.2# lspv
hdisk0 002b012f397c20ce rootvg active
hdisk1 002afe4f2b4c3fdb rootvg active
hdisk2 002b016f09313544 ppmvg active

To verify, rootvg is mirrored. We should have a 1:2 ratio between LP and PP except for the dumpdevices, like this.

bash-4.2# lsvg -l rootvg
rootvg:
LV NAME TYPE LPs PPs PVs LV STATE MOUNT POINT
hd5 boot 1 2 2 closed/stale N/A
hd6 paging 2 4 2 open/syncd N/A
hd8 jfs2log 1 2 2 open/syncd N/A
hd4 jfs2 2 4 2 open/syncd /
hd2 jfs2 10 20 2 open/stale /usr
hd9var jfs2 1 2 2 open/syncd /var
hd3 jfs2 13 26 2 open/syncd /tmp
hd1 jfs2 1 2 2 open/syncd /home
hd10opt jfs2 1 2 2 open/syncd /opt
hd11admin jfs2 1 2 2 open/syncd /admin
fwdump jfs2 2 4 2 open/syncd /var/adm/ras/platform
lg_dumplv sysdump 4 4 1 open/syncd N/A
dppmiaslv jfs2 120 240 2 open/stale /dppmias
data01lv jfs2 200 400 2 open/stale /data01

And we want to see exactly where a LV is mirrored:-

bash-4.2# lslv -m hd2
hd2:/usr
LP PP1 PV1 PP2 PV2 PP3 PV3
0001 0222 hdisk1 0223 hdisk0
0002 0228 hdisk1 0224 hdisk0
0003 0229 hdisk1 0225 hdisk0
0004 0230 hdisk1 0226 hdisk0
0005 0231 hdisk1 0227 hdisk0
0006 0232 hdisk1 0228 hdisk0
0007 0523 hdisk1 0229 hdisk0
0008 0524 hdisk1 0230 hdisk0
0009 0525 hdisk1 0231 hdisk0
0010 0527 hdisk1 0232 hdisk0

Now, check your bootlist. Does it have both physical disks in the bootlist or not.

bash-4.2# bootlist -m normal -o
hdisk1 blv=hd5 pathid=0
hdisk0 blv=hd5 pathid=0