Category Archives: Shell Script

How to Find Server Public IP Address in Linux Terminal

root@test:/var/log/nginx# wget -qO – icanhazip.com
www.xxx.yyy.zzz
root@test:/var/log/nginx# wget -qO- http://ipecho.net/plain | xargs echo
www.xxx.yyy.zzz
root@test:/var/log/nginx# wget -qO – icanhazip.com
www.xxx.yyy.zzz
root@test:/var/log/nginx# curl icanhazip.com
www.xxx.yyy.zzz
root@test:/var/log/nginx#

sed to play with data or parsing your text

To select all the lines starting from STARTING_PATTERN up to  blank line ^$ and then delete those lines.

# sed ‘/STARTING_PATTERN/,/^$/d’ filename

To edit files in place, use -i option.

# sed -i ‘/STARTING_PATTER/,/^$/d’ filename

Insert multiple lines into a file after specified pattern.

# sed ‘/cdef/r add.txt’ input.txt

# sed ‘/cdef/r add.txt’ input.txt

input.txt:
abcd
accd
cdef
line
web
add.txt:
line1
line2
line3
line4
Output :
abcd
accd
cdef
line1
line2
line3
line4
line
web

If you want to apply the changes in input.txt file. Then, use -i with sed.

# sed -i ‘/cdef/r add.txt’ input.txt

If you want to use a regex as an expression you have to use the -E tag with sed.

# sed -E ‘/RegexPattern/r add.txt’ input.txt

How to configure Proxy Settings for the Unix / Linux Console

You can use the following methods to configure your console to use a proxy server so that console based programs like wget could get connect to the internet through the proxy.

1 – Set the environment variable
# export http_proxy=http://DOMAIN\USERNAME:PASSWORD@SERVER:PORT/
# export ftp_proxy=http://DOMAIN\USERNAME:PASSWORD@SERVER:PORT/

In the above configuration you can ommit the DOMAIN\USERNAME:PASSWORD@ part if you are not using proxy authentication.

Note: If you get the following error when you try to use wget, you might have to pass the proxy authentication credentials to wget as arguments.

Connecting to SERVER:PORT... connected.
Proxy request sent, awaiting response... 407 Proxy Authentication Required
11:14:45 ERROR 407: Proxy Authentication Required.
1.1 – Passing proxy authentication credentials to wget as arguments
$ wget --proxy-user "DOMAIN\USERNAME" --proxy-passwd "PASSWORD" URL
2 – Configure the proxy settings in the .bashrc
If you want set the proxy for all the users you can do it in the system wide .bashrc file.

nano /etc/bash.bashrc#proxy settings
export http_proxy=http://DOMAIN\USERNAME:PASSWORD@SERVER:PORT/
export ftp_proxy=http://DOMAIN\USERNAME:PASSWORD@SERVER:PORT/

Note: The system wide .bashrc file may not be available in all Linux systems and only can be used if you are using the bash shell

2.1 – Having an alias for wget with proxy
If you don’t want to pass the proxy arguments to wget all the time, you create an alias for wget in the .bashrc file

alias wget 'wget --proxy-user "DOMAIN\USERNAME" --proxy-passwd "PASSWORD"'

Delete Files Older Than x Days on Linux / Unix

The find utility on linux allows you to pass in a bunch of interesting arguments, including one to execute another command on each file. We’ll use this in order to figure out what files are older than a certain number of days, and then use the rm command to delete them.

Command Syntax

find /path/to/files* -mtime +5 -exec rm {} ;

Note that there are spaces between rm, {}, and ;

Explanation

  • The first argument is the path to the files. This can be a path, a directory, or a wildcard as in the example above. I would recommend using the full path, and make sure that you run the command without the exec rm to make sure you are getting the right results.
  • The second argument, -mtime, is used to specify the number of days old that the file is. If you enter +5, it will find files older than 5 days.
  • The third argument, -exec, allows you to pass in a command such as rm. The {} ; at the end is required to end the command.

Collecting Unix System Information

At the very least, collect the following information for each system that you have:
1. Hostname:
% hostname
2. Hostname aliases:
% grep `hostname` /etc/hosts | awk ‘{ print $3 }’
3. Host network addresses:
% grep `hostname` /etc/hosts | awk ‘{ print $1 }’
4.   Host ID:
% hostid
5. System serial number:
On the back of most all computers.
6. Manufacturer of the system’s hardware:
On the front of most computers
7. System model name:
On the front of most computers
8. CPU type:
% uname -a
9. Application architecture:
% uname -a
10.   Kernel architecture:
% uname -a
11.  Amount of main memory:
Can be found at boot time
% dmesg
12.  Operating system name:
% uname -a
13.  Operating system version:
% uname -a
14.  Kernel version:
% uname -a
15.  Disk configuration:
% df

How to check and install missing perl modules

  • Check if module is installed. Errors mean missing module.

    # perl -MModule::Name -e 1
    
  • See documentation of the module if installed.
    # perldoc Module::Name
    
  • Open CPAN shell
    # perl -MCPAN -e shell
    
  • To reconfigure the shell if needed.
    cpan>o conf init
    
  • Install an available module.
    cpan> install HTML::Template
  • You can run the Perl CPAN module via command line perl and get it installed in a single line:

# perl -MCPAN -e ‘install HTML::Template

  • Force install if test fails.
    cpan> force install Module::Name
    
  • To manual install perl modules. Unzip and go to module directory.

# tar -zxvf HTML-Template-2.8.tar.gz

  • # perl Makefile.PL
    # make
    # make test
    # make install
    

How to use Unix Editor Vi (Vim)

Today vi is considered the standard. It is the only editor that will be  installed by default on any UNIX system.

It is small, fast and efficient; useful to make some minor editing of system files, editing of huge data files or when a slow network link is involved.

It is important to learn at least the basics of vi.

Vi: Start-up, Modes, Save and Quit

To enter Vi type:

 vi FILENAME

Vi has two modes: Normal (command) mode and edit mode.

 Switch to normal mode:     <ESC>
 Switch to edit mode:       i or a

Getting out of Vi (change to normal mode <ESC>):

 Exit Vi:                       :q
 Exit Vi (ignore changes):      :q!
 Save:                          :w
 Save and Exit:                 :wq

Switch to (edit) an other file:

:edit FILENAME

Getting help:

:help topic

Vi: Move, Delete and Paste

Change to normal mode with <ESC>.

Move: Use either the arrow keys or the hjkl keys:

 h (left)    j (down)    k (up)    l (right)

 Getlocation and file status:   Ctrl-g
 Moves to end of the file:      Shift-G
 Moves line NUMBER:             NUMBER Shift-G

Delete:

 Delete a character:     x
 Delete a line:          dd

For multiple deletion precede command with a number.

 Delete 5 characters:     5 x
 Delete 7 lines:          7 dd

Paste: inserts all you deleted with the preceding delete.

 Paste:	                 p

Vi: Search and Replace

Change to normal mode with <ESC>.

Search (Wraped around at end of file):

  Search STRING forward :   / STRING.
  Search STRING backward:   ? STRING.

  Repeat search:   n
  Repeat search in opposite direction:  N  (SHIFT-n)

Replace: Same as with sed, Replace OLD with NEW:

 First occurrence on current line:      :s/OLD/NEW

 Globally (all) on current line:        :s/OLD/NEW/g

 Between two lines #,#:                 :#,#s/OLD/NEW/g

 Every occurrence in file:              :%s/OLD/NEW/g

 VIM QUICK REFERENCE CARD or VI Cheat Sheet

What is Shell Scripts

Shell scripts are text files that automate a series of UNIX environment-based commands that otherwise must be performed one at a time. Shell scripts are often used to automate command sequences that repeat, such as services that start or stop on system start up or shut down.

Any command that can be performed from the command line, such as ls, can be included in a shell script. Similarly, any command that can be included in a shell script can be performed on the UNIX environment command line.

Users with little or no programming experience can create and run shell scripts. You initiate the sequence of commands in the shell script by simply entering the name of the shell script on a command line.

Determining the Type of Shell to Run a Shell Script

There are several different shells available in the Solaris OS. Two of the most commonly used shells are the Bourne shell and the Korn shell.

To ensure that the correct shell is used to run a shell script, the first line of the script should always begin with the characters #!, followed immediately by the absolute path name of the shell required to run the script. These must be the only characters on the first line of the file.

#!/full-pathname-of-shell

For example:

#!/bin/sh

or

#!/bin/ksh

Comments

Comments are text entries that often provide information about a shell script. They are inserted into a shell script but have no effect on the script itself. Comments are ignored by the shell and are solely for the benefit of the user.

Comments are preceded by the hash (#) character. Whenever the shell encounters a word beginning with the # character it ignores all text on that line.

For example:

# this is a comment ls -l # list the files in a directory

Setting Korn Shell Options

Options are switches that control the behavior of the Korn shell. Options are boolean, meaning that they can be either on or off.

To switch an option on, type:

$ set -o option_name

To switch an option off, type:

$ set +o option_name

To show current option settings, type:

$ set -o
Note: The set -o and set +o options can only change a single option setting at a time.

Protecting File Content During I/O Redirection

Redirecting standard output to an existing file overwrites the previous file content, which results in data loss. This process of overwriting existing data is known as clobbering. To prevent an overwrite from occurring, the shell supports a noclobber option.

When the noclobber option is set, the shell refuses to redirect standard output to the existing file and displays an error message to the screen.

The noclobber option is activated in the shell using the set command. For example:

$ set -o noclobber
$ set -o | grep noclobber
noclobber        on
$ ps -ef > file_new
$ cat /etc/passwd > file_new
ksh: file_new: file already exists
$

Deactivating the noclobber Option

To temporarily deactivate the noclobber option, use the >| deactivation syntax on the command line. The noclobber option is ignored for this command line only, and the contents of the file are overwritten.

$ ls -l >| file_new

Note: There is no space between the > and | on the command line.

To deactivate the noclobber option, perform the following commands:

$ set +o noclobber $ set -o | grep noclobber
noclobber off
$ ls -l > file_new $

Using Korn Shell Functions

Functions are a powerful feature of shell programming used to construct customized commands. A function is a group of UNIX commands organized as separate routines. Using a function involves two steps:

  1. Define the function.
  2. Invoke the function.

Defining a Function

A function is defined using the general format:

function_name { command; . . . command; }
Note: A space must appear after the first brace and before the closing brace.

Function Examples

The following example creates a function called num to perform the who command, directing the output to the wc command to display the total number of users currently logged on the system:

$ function num { who | wc -l; }
$ num
	 9

The following example creates a function called list to perform the ls command, directing the output to the wc command to display the total number of subdirectories and files in the current directory:

$ function list { ls -al | wc -l; }
$ list
	 34
Note: If a command name is defined as a function and an alias, the alias takes precedence.

To display a list of all functions, use the following command:

$ typeset -f
function list
{
ls -al | wc -l; }
function num
{
who | wc -l; }

To display just the function names, use the following command:

$ typeset +f
list
num

Configuring the Shell Environment

The shell secondary prompt string is stored in the PS2 shell variable, and you can customize it as needed.

$ PS2="somethings missing >" $ echo $PS2 somethings missing > $ In this example, the secondary prompt displays the message that the command line is incomplete.
Note: To have this secondary shell prompt appear in every shell, it must be included in the user's Korn shell initialization file (usually named .kshrc).