Category Archives: Solaris

File System shows 100% occupied but du tells different and still has Unused Spaces.

We have faced an issue as below mentioned – found /oracle mount-point showing 100% as Used. But getting different size values for folders under /oracle mount-point, when we executed du -gs .

bash-4.4# df -g
Filesystem GB blocks Free %Used Iused %Iused Mounted on
/dev/hd4 4.00 1.39 66% 20686 6% /
/dev/hd2 4.00 1.02 75% 45679 16% /usr
/dev/hd9var 2.00 1.01 50% 16318 7% /var
/dev/hd3 2.00 1.83 9% 407 1% /tmp
/dev/hd1 1.00 0.16 85% 6124 12% /home
/dev/hd11admin 0.25 0.25 1% 5 1% /admin
/proc – – – – – /proc
/dev/hd10opt 0.50 0.07 87% 12979 44% /opt
/dev/livedump 0.25 0.25 1% 4 1% /var/adm/ras/livedump
/dev/fslv00 198.00 0.01 100% 440205 16% /oracle

bash-4.4# cd /oracle/
bash-4.4# du -gs *
0.00 AutoDeployment
5.45 Oracle
0.03 Patch
7.54 data01
11.67 fmw_12.1.3.0.0_wls
0.01 jboss
0.20 jdk
0.00 lost+found
0.92 wls1221

You may notice 100 % utilization for /oracle mount-point on the “df -g” output , but with “du -gs ” We found the files did not occupy the entire space.

It may be because of open files in the file-system. These open files hold space on the file-system to complete there execution, once the write operation is completed. 

To overcome, from this issue, We need to follow below two steps:-

  1. We need to find all such processes still running but using deleted files, in result – /oracle mount-point is showing 100% as Used.

bash-4.4# fuser -dV /oracle/
/oracle/:
inode=670047 size=5242722 fd=389 5570776
inode=1165305 size=20278 fd=1 8061106
inode=1165313 size=182335565824 fd=1 8716486
inode=1165305 size=20278 fd=1 9044152
inode=1165313 size=182335565824 fd=1 9371672
inode=1165305 size=20278 fd=1 11141354
inode=669981 size=5514335 fd=1 13041898


2. We have to kill all such processes as found in above command.

bash-4.4# kill -9 5570776
bash-4.4# kill -9 8061106
bash-4.4# kill -9 8716486
bash-4.4# kill -9 9044152
bash-4.4# kill -9 9371672
bash-4.4# kill -9 11141354
bash-4.4# kill -9 13041898

bash-4.4# fuser -dV /oracle/
/oracle/:

bash-4.4# df -g
Filesystem GB blocks Free %Used Iused %Iused Mounted on
/dev/hd4 4.00 1.39 66% 20686 6% /
/dev/hd2 4.00 1.02 75% 45679 16% /usr
/dev/hd9var 2.00 1.01 50% 16318 7% /var
/dev/hd3 2.00 1.83 9% 407 1% /tmp
/dev/hd1 1.00 0.16 85% 6124 12% /home
/dev/hd11admin 0.25 0.25 1% 5 1% /admin
/proc – – – – – /proc
/dev/hd10opt 0.50 0.07 87% 12979 44% /opt
/dev/livedump 0.25 0.25 1% 4 1% /var/adm/ras/livedump
/dev/fslv00 198.00 170.01 15% 440205 1% /oracle
bash-4.4# cd /oracle/
bash-4.4# du -gs *
0.00 AutoDeployment
5.45 Oracle
0.03 Patch
7.54 data01
11.67 fmw_12.1.3.0.0_wls
0.01 jboss
0.20 jdk
0.00 lost+found
0.92 wls1221

How to Find Server Public IP Address in Linux Terminal

root@test:/var/log/nginx# wget -qO – icanhazip.com
www.xxx.yyy.zzz
root@test:/var/log/nginx# wget -qO- http://ipecho.net/plain | xargs echo
www.xxx.yyy.zzz
root@test:/var/log/nginx# wget -qO – icanhazip.com
www.xxx.yyy.zzz
root@test:/var/log/nginx# curl icanhazip.com
www.xxx.yyy.zzz
root@test:/var/log/nginx#

sed to play with data or parsing your text

To select all the lines starting from STARTING_PATTERN up to  blank line ^$ and then delete those lines.

# sed ‘/STARTING_PATTERN/,/^$/d’ filename

To edit files in place, use -i option.

# sed -i ‘/STARTING_PATTER/,/^$/d’ filename

Insert multiple lines into a file after specified pattern.

# sed ‘/cdef/r add.txt’ input.txt

# sed ‘/cdef/r add.txt’ input.txt

input.txt:
abcd
accd
cdef
line
web
add.txt:
line1
line2
line3
line4
Output :
abcd
accd
cdef
line1
line2
line3
line4
line
web

If you want to apply the changes in input.txt file. Then, use -i with sed.

# sed -i ‘/cdef/r add.txt’ input.txt

If you want to use a regex as an expression you have to use the -E tag with sed.

# sed -E ‘/RegexPattern/r add.txt’ input.txt

Creating graphs from SAR output

You must know that sar is a very effective tool to collect system activity or performance information from your system.
To collect all information:

sar -o test.log -A 1 3 2>&1 >/dev/null

This command will make a binary file from output, and you can display it with sadf command:

sadf -t -d test.log — -A

Sometimes, you would like to make graphs from these data, because diagrams are clear and better understandable than plain data.
The best way to create graphs from output of sar is kSar that is an Java-based application with gui.
You can download it from the following site: http://ksar.atomique.net/

This program can process sar text output and make diagrams from them, also it can save graphs as picture or export them into a PDF file.
If you have only the binary output of sar, you can convert it into text file with this command:

sar -A -f test.log >> sardata.txt

Now, you can import text file into kSar, with “Data/Load from text file…” menu entry.
Also, kSar has other useful functions, like remote sar running via SSH, for more details read kSar documentation.
Loading text file:sar-graph2-300x187 sar-graph-300x187 sar-main-300x187

What process is listening on a certain port on Solaris? / How to find out which process listens on certain port on Solaris?

Scenario: I’m looking for PID for which PORT 2817 is using here in Solaris 11.

-bash-3.2# netstat -an | grep 2817
*.2817 *.* 0 0 49152 0 LISTEN
10.0.50.81.2817 10.0.50.81.37374 49152 0 49152 0 CLOSE_WAIT
10.0.50.81.2817 10.0.50.81.35510 49152 0 49152 0 CLOSE_WAIT
10.0.50.81.2817 10.0.50.81.34478 49152 0 49152 0 CLOSE_WAIT

Here is a script, I found somewhere which actually works. This script worked to find PID from port number, If lsof command/utility is not available with server.

Save this script as Port_check.sh and grant execute permissions using chmod 777 to this script.

————————————————————————————-

#!/bin/ksh

line=’———————————————‘
pids=$(/usr/bin/ps -ef | sed 1d | awk ‘{print $2}’)

if [ $# -eq 0 ]; then
read ans?”Enter port you would like to know pid for: “
else
ans=$1
fi

for f in $pids
do
/usr/proc/bin/pfiles $f 2>/dev/null | /usr/xpg4/bin/grep -q “port: $ans”
if [ $? -eq 0 ]; then
echo $line
echo “Port: $ans is being used by PID:\c”
/usr/bin/ps -ef -o pid -o args | egrep -v “grep|pfiles” | grep $f
fi
done
exit 0

————————————————————————————-
Now the syntax of using this script is
./Port_check.sh <port number>

For example, to find which process is using TCP port 2817,

-bash-3.2# ./Port_check.sh 2817
———————————————
Port: 2817 is being used by PID:26222 /data01/IBM/WebSphere/AppServer/java_1.7_64/bin/sparcv9/java -XX:+UnlockDiagnos

So, Here is PID 26222 using port 2817.

Thanks…

How to Rescan new LUN’s added in Linux, HP-UX, Aix, Solaris ?

HP-UX

1. Rescan the devices:

ioscan -fnC <disk|tape>

2. Generate device files:

 insf -e

3. Verify the new devices:

 ioscan -funC <disk|tape>

AIX

1. Rescan the devices ):

 cfgmgr -vl fcsx

Where x is FC adapter number

2. Verify the new devices:

 lsdev -Cc <disk|tape>

Linux

The rescan in Linux is HBA-specific.

For QLogic:

echo scsi-qlascan > /proc/scsi/qla<model#>/<adapter instance>

For Emulex:

 sh force_lpfc_scan.sh lpfc<adapter-instance>

For each identified device, run the following:

echo scsi add-single-device <host> <channel> <ID> <lun> >   /proc/scsi/scsi

Solaris

1. Determine the FC channels:

 cfgadm -al

2. Force rescan :

 cfgadm -o force_update -c configure cx

Where x is the FC channel number

3. Force rescan at HBA port level:

 luxadm -e forcelip /dev/fc/fpx

4. Force rescan on all FC devices:

 cfgadm -al -o show_FCP_dev

5. Install device files:

 devfsadm

6. Display all Qlogic HBA ports

 luxadm -e port

7. Display HBA port information

 luxadm -v display <WWPN>

8. Display HBA port information

 luxadm -e dump_map

Notes If one specific SANclient is missing a drive, please verify that your zoning is correct. Please also make sure the host initiator and VTL™s target ports are showing online via the Fibre Channel switch. (Check HBA link light and check the cable.)

 

Configuring X-Server Display For Oracle Solaris 11

What is X Window ?

The X Window System, commonly referred to as X, is a network-based graphical window system.
The X Window System uses a client-server architecture. It enables multiple programs to share and access a common set of hardware.
This hardware includes both input and display devices such as mouse, keyboards, video adapters, and monitors that are connected to the server.

The X Window System consists of X server and X clients.
The X clients are application programs that do not have direct access to the display.
They communicate with the X server which provides the display.

A proper X client-server setup is requires for Oracle’s Universal Installer ,because OUI is a X-window client which
requires a X-Server display to connect to, for its GUI.

What Packages are required ?

 

Packages Description
pkg://solaris/x11/x11-server-utilities X11 server state utilities
pkg://solaris/x11/session/xauth X authority file utility
pkg://solaris/library/motif Used for Motif-based Toolkit
pkg:/system/library/c++-runtime Sun Workshop Compilers Bundled libC
pkg:/system/library/math Math Libraries
pkg:/desktop/window-manager/twm Tab Window Manager for the X Window
pkg:/system/library System Lib files
pkg:/library/zlib The Zip compression library
pkg:/x11/library/libx11 X11 core protocol client library
pkg:/x11/library/libxau X authorization database library
pkg:/x11/library/libxdmcp X Display Manager Control Protocol
pkg:/x11/library/libxext X11 protocol common extensions client
pkg:/x11/library/libxtst libXtst – X Test and Record library
pkg:x11/library/toolkit/libxt X Toolkit Intrinsics library
pkg://solaris/terminal/xterm Terminal emulator for X
pkg://solaris/x11/xclock Displays the time in analog or digital form.

How to configure Xorg included in the Oracle Solaris OS ?

Xorg: Is installed by default .Present in directory /usr/bin/Xorg .
The Xorg server is designed to configure automatically and can run in most situations without the need to edit configuration files.
When configuration is needed, the Xorg server gathers configuration details from xorg.conf the Xorg server configuration file /etc/X11/ directory.

Note :
Xorg packages are included on the Live Media, but not with the text installer.Symbolic links for
X Window System software compatibility with other releases is also present in the directory /usr/X11R6 .

pkg info x11/server/xorg

Name: x11/server/xorg
Summary: Xorg – X11R7 X server
Publisher: solaris
Version: 1.12.2
Build Release: 5.11

How to Configure Xming ?

Download xming, install it on your windows pc system.You can go to http://sourceforge.net/projects/xming/files/ to download.

Do the following changes for Xming configuration on you windows pc:

Add a new entry in C:\Program Files\Xming\X0.hosts

ex. :

localhost
<IP address of server>

Login through putty client to the server :
export DISPLAY=<windows-pc-IP>:0.0
xhost +

To check if the X Window server-client is working You can use either of the applications :

XCLOCK-Displays the time in analog or digital form.

You will then see a clock gui pop up in your windows pc.

Which Toolkit does OUI use ?

Solaris will continue to use the MToolkit (Motif-based Toolkit) as the default in J2SE 5.0, but eventually will be replaced with XToolkit in JDK7.

The new XToolkit implementation provides the following advantages:
-Removes the dependency on Motif and Xt libraries.
-Interoperates better with other GUI Toolkits.
-Provides better performance and quality.

Setting the toolkit for an application:
Using an environment variable. This needs to be set before starting the VM.
csh:
setenv AWT_TOOLKIT XToolkit #selects the XToolkit
setenv AWT_TOOLKIT MToolkit #selects the MToolkit (Motif-based Toolkit)

export AWT_TOOLKIT=XToolkit
export AWT_TOOLKIT=MToolkit

How to configure VNC if the Xorg does not work ?

However for some reason the above method doesnot work the easiest is doing this via VNC as
it removes the requirement of having to have a X-Server installed on your client platform.
Install VNC server on your Oracle server platform as described in next step.
Xvnc:
Runs a VNC session that can be connected by using a VNC client. Virtual Network Computing (VNC) is a
remote software application that enables you to view and interact with one computer desktop,
the Xvnc server, by using the VNC viewer on another computer desktop.
The two computers do not have to be running the same type of operating system. Xvnc provides a guest domain graphical login.

pkg install xvnc

pkg info xvnc – Would list the package installed .
How to Start VNC Manually ?

Run VNC server as the oracle user .
Start the VNC server.
$ /usr/bin/vncserver
Enter the VNC server password.
Password:
Verify:
New ‘myhost:2 ()’ desktop is myhost:2

Creating default startup script /home/oracle/.vnc/xstartup
Creating default startup script /home/oracle/.vnc/xstartup
Starting applications specified in /home/oracle/.vnc/xstartup
Log file is /home/oracle/.vnc/myhost:2.log

From your client (likely Windows), run VNC viewer and connect to oracle’s VNC server display.
# vncviewer hostname:portnumber
For example:

# vncviewer myhost:2
Type the password you provided to the vncserver script.

System Panic During Boot Logging the Error “NOTICE: zfs_parse_bootfs: error 19”

Today while migrating SAN i face this issue, hope it will help others too…

The system panic during boot logging the error:

{0} ok boot 56024-disk
Boot device: /virtual-devices@100/channel-devices@200/disk@1 File and args:
SunOS Release 5.10 Version Generic_147440-01 64-bit
Copyright (c) 1983, 2011, Oracle and/or its affiliates. All rights reserved.
NOTICE: zfs_parse_bootfs: error 19
Cannot mount root on rpool/68 fstype zfs
panic[cpu0]/thread=180e000: vfs_mountroot: cannot mount root

Changes

This issue usually occurs when system is trying to boot a ZFS rpool and the path to the disk changed, or customer is trying to boot the system from a cloned disk (that means the disk is a copy of another boot disks)

Cause

The issue is caused by a mismatch between the current path of the disk you are trying to boot from and the path stored in the ZFS label of the same disk:

ok boot 56024-disk
Boot device: /virtual-devices@100/channel-devices@200/disk@1 File and args:

 

# zdb -l /dev/rdsk/c0d1s0
——————————————–
LABEL 0
——————————————–
version: 29
name: ‘rpool’
state: 0
txg: 1906
pool_guid: 3917355013518575342
hostid: 2231083589
hostname: ”
top_guid: 3457717657893349899
guid: 3457717657893349899
vdev_children: 1
vdev_tree:
type: ‘disk’
id: 0
guid: 3457717657893349899
path: ‘/dev/dsk/c0d0s0
devid: ‘id1,vdc@f85a3722e4e96b600000e056e0049/a’
phys_path: ‘/virtual-devices@100/channel-devices@200/disk@0:a
whole_disk: 0
metaslab_array: 31
metaslab_shift: 27
ashift: 9
asize: 21361065984
is_log: 0
create_txg: 4

As you can see we are trying to boot the path disk@1 but in the ZFS label the path is disk@0.

Solution

To fix the issue you have to boot the system in failsafe mode or from cdrom and import the rpool on that disk to force ZFS to correct the path:

# zpool import -R /mnt rpool
cannot mount ‘/mnt/export’: failed to create mountpoint
cannot mount ‘/mnt/export/home’: failed to create mountpoint
cannot mount ‘/mnt/rpool’: failed to create mountpoint

# zdb -l /dev/rdsk/c0d1s0
——————————————–
LABEL 0
——————————————–
version: 29
name: ‘rpool’
state: 0
txg: 1923
pool_guid: 3917355013518575342
hostid: 2230848911
hostname: ”
top_guid: 3457717657893349899
guid: 3457717657893349899
vdev_children: 1
vdev_tree:
type: ‘disk’
id: 0
guid: 3457717657893349899
path: ‘/dev/dsk/c0d1s0
devid: ‘id1,vdc@f85a3722e4e96b600000e056e0049/a’
phys_path: ‘/virtual-devices@100/channel-devices@200/disk@1:a
whole_disk: 0
metaslab_array: 31
metaslab_shift: 27
ashift: 9
asize: 21361065984
is_log: 0
create_txg: 4

As you can see the path has been corrected, however you have also to remove the zpool.cache file otherwise after boot the ZFS command will still show the disk as c0d0:

# zfs list
NAME USED AVAIL REFER MOUNTPOINT
rpool 5.86G 13.7G 106K /mnt/rpool
rpool/ROOT 4.35G 13.7G 31K legacy
rpool/ROOT/s10s_u10wos_17b 4.35G 13.7G 4.35G /mnt
rpool/dump 1.00G 13.7G 1.00G –
rpool/export 63K 13.7G 32K /mnt/export
rpool/export/home 31K 13.7G 31K /mnt/export/home
rpool/swap 528M 14.1G 114M –

# zfs mount rpool/ROOT/s10s_u10wos_17b
# cd /mnt/etc/zfs
# rm zpool.cache

How to configure Proxy Settings for the Unix / Linux Console

You can use the following methods to configure your console to use a proxy server so that console based programs like wget could get connect to the internet through the proxy.

1 – Set the environment variable
# export http_proxy=http://DOMAIN\USERNAME:PASSWORD@SERVER:PORT/
# export ftp_proxy=http://DOMAIN\USERNAME:PASSWORD@SERVER:PORT/

In the above configuration you can ommit the DOMAIN\USERNAME:PASSWORD@ part if you are not using proxy authentication.

Note: If you get the following error when you try to use wget, you might have to pass the proxy authentication credentials to wget as arguments.

Connecting to SERVER:PORT... connected.
Proxy request sent, awaiting response... 407 Proxy Authentication Required
11:14:45 ERROR 407: Proxy Authentication Required.
1.1 – Passing proxy authentication credentials to wget as arguments
$ wget --proxy-user "DOMAIN\USERNAME" --proxy-passwd "PASSWORD" URL
2 – Configure the proxy settings in the .bashrc
If you want set the proxy for all the users you can do it in the system wide .bashrc file.

nano /etc/bash.bashrc#proxy settings
export http_proxy=http://DOMAIN\USERNAME:PASSWORD@SERVER:PORT/
export ftp_proxy=http://DOMAIN\USERNAME:PASSWORD@SERVER:PORT/

Note: The system wide .bashrc file may not be available in all Linux systems and only can be used if you are using the bash shell

2.1 – Having an alias for wget with proxy
If you don’t want to pass the proxy arguments to wget all the time, you create an alias for wget in the .bashrc file

alias wget 'wget --proxy-user "DOMAIN\USERNAME" --proxy-passwd "PASSWORD"'

How to Unzip Multiple Files from Linux / Unix single Command Line

 
bash-3.2# ls *.zip
118666-47.zip  120830-06.zip  138852-01.zip  142394-01.zip  147217-02.zip
118777-16.zip  120849-04.zip  139520-02.zip  142933-05.zip  148027-03.zip
119081-25.zip  124204-04.zip  142240-01.zip  143506-06.zip
119963-24.zip  126425-01.zip  142251-02.zip  146470-08.zip
bash-3.2#
bash-3.2#

Problem :

bash-3.2# unzip *.zip
Archive:  118666-47.zip
caution: filename not matched:  118777-16.zip
caution: filename not matched:  119081-25.zip
caution: filename not matched:  119963-24.zip
caution: filename not matched:  120830-06.zip
caution: filename not matched:  120849-04.zip
caution: filename not matched:  124204-04.zip
caution: filename not matched:  126425-01.zip
caution: filename not matched:  138852-01.zip
caution: filename not matched:  139520-02.zip
caution: filename not matched:  142240-01.zip
caution: filename not matched:  142251-02.zip
caution: filename not matched:  142394-01.zip
caution: filename not matched:  142933-05.zip
caution: filename not matched:  143506-06.zip
caution: filename not matched:  146470-08.zip
caution: filename not matched:  147217-02.zip
caution: filename not matched:  148027-03.zip
bash-3.2#
bash-3.2#
bash-3.2#

Solution :

Use single quotes [ ‘  ‘ ]
bash-3.2# unzip ‘*.zip’
Archive:  146470-08.zip
inflating: 146470-08/LEGAL_LICENSE.TXT
inflating: 146470-08/prepatch
inflating: 146470-08/SUNWcsu/reloc/usr/share/lib/zoneinfo/EST
inflating: 146470-08/SUNWcsu/reloc/usr/share/lib/zoneinfo/tab/zone_sun.tab
inflating: 146470-08/SUNWcsu/reloc/usr/share/lib/zoneinfo/tab/country.tab
inflating: 146470-08/SUNWcsu/reloc/usr/share/lib/zoneinfo/Australia/Broken_Hill
inflating: 146470-08/SUNWcsu/reloc/usr/share/lib/zoneinfo/Australia/Eucla
inflating: 146470-08/SUNWcsu/reloc/usr/share/lib/zoneinfo/Australia/Melbourne
inflating: 146470-08/SUNWcsu/reloc/usr/share/lib/zoneinfo/Australia/Sydney
inflating: 146470-08/SUNWcsu/reloc/usr/share/lib/zoneinfo/Australia/Adelaide
inflating: 146470-08/SUNWcsu/reloc/usr/share/lib/zoneinfo/Australia/Perth
inflating: 146470-08/SUNWcsu/reloc/usr/share/lib/zoneinfo/Australia/Lord_Howe
inflating: 146470-08/SUNWcsu/reloc/usr/share/lib/zoneinfo/Australia/Hobart
inflating: 146470-08/SUNWcsu/reloc/usr/share/lib/zoneinfo/Australia/Currie