Category Archives: Shell Script

How to configure Proxy Settings for the Unix / Linux Console

You can use the following methods to configure your console to use a proxy server so that console based programs like wget could get connect to the internet through the proxy.

1 – Set the environment variable
# export http_proxy=http://DOMAIN\USERNAME:PASSWORD@SERVER:PORT/
# export ftp_proxy=http://DOMAIN\USERNAME:PASSWORD@SERVER:PORT/

In the above configuration you can ommit the DOMAIN\USERNAME:PASSWORD@ part if you are not using proxy authentication.

Note: If you get the following error when you try to use wget, you might have to pass the proxy authentication credentials to wget as arguments.

Connecting to SERVER:PORT... connected.
Proxy request sent, awaiting response... 407 Proxy Authentication Required
11:14:45 ERROR 407: Proxy Authentication Required.
1.1 – Passing proxy authentication credentials to wget as arguments
$ wget --proxy-user "DOMAIN\USERNAME" --proxy-passwd "PASSWORD" URL
2 – Configure the proxy settings in the .bashrc
If you want set the proxy for all the users you can do it in the system wide .bashrc file.

nano /etc/bash.bashrc#proxy settings
export http_proxy=http://DOMAIN\USERNAME:PASSWORD@SERVER:PORT/
export ftp_proxy=http://DOMAIN\USERNAME:PASSWORD@SERVER:PORT/

Note: The system wide .bashrc file may not be available in all Linux systems and only can be used if you are using the bash shell

2.1 – Having an alias for wget with proxy
If you don’t want to pass the proxy arguments to wget all the time, you create an alias for wget in the .bashrc file

alias wget 'wget --proxy-user "DOMAIN\USERNAME" --proxy-passwd "PASSWORD"'

Delete Files Older Than x Days on Linux / Unix

The find utility on linux allows you to pass in a bunch of interesting arguments, including one to execute another command on each file. We’ll use this in order to figure out what files are older than a certain number of days, and then use the rm command to delete them.

Command Syntax

find /path/to/files* -mtime +5 -exec rm {} ;

Note that there are spaces between rm, {}, and ;

Explanation

  • The first argument is the path to the files. This can be a path, a directory, or a wildcard as in the example above. I would recommend using the full path, and make sure that you run the command without the exec rm to make sure you are getting the right results.
  • The second argument, -mtime, is used to specify the number of days old that the file is. If you enter +5, it will find files older than 5 days.
  • The third argument, -exec, allows you to pass in a command such as rm. The {} ; at the end is required to end the command.

Collecting Unix System Information

At the very least, collect the following information for each system that you have:
1. Hostname:
% hostname
2. Hostname aliases:
% grep `hostname` /etc/hosts | awk ‘{ print $3 }’
3. Host network addresses:
% grep `hostname` /etc/hosts | awk ‘{ print $1 }’
4.   Host ID:
% hostid
5. System serial number:
On the back of most all computers.
6. Manufacturer of the system’s hardware:
On the front of most computers
7. System model name:
On the front of most computers
8. CPU type:
% uname -a
9. Application architecture:
% uname -a
10.   Kernel architecture:
% uname -a
11.  Amount of main memory:
Can be found at boot time
% dmesg
12.  Operating system name:
% uname -a
13.  Operating system version:
% uname -a
14.  Kernel version:
% uname -a
15.  Disk configuration:
% df

How to check and install missing perl modules

  • Check if module is installed. Errors mean missing module.

    # perl -MModule::Name -e 1
    
  • See documentation of the module if installed.
    # perldoc Module::Name
    
  • Open CPAN shell
    # perl -MCPAN -e shell
    
  • To reconfigure the shell if needed.
    cpan>o conf init
    
  • Install an available module.
    cpan> install HTML::Template
  • You can run the Perl CPAN module via command line perl and get it installed in a single line:

# perl -MCPAN -e ‘install HTML::Template

  • Force install if test fails.
    cpan> force install Module::Name
    
  • To manual install perl modules. Unzip and go to module directory.

# tar -zxvf HTML-Template-2.8.tar.gz

  • # perl Makefile.PL
    # make
    # make test
    # make install
    

How to use Unix Editor Vi (Vim)

Today vi is considered the standard. It is the only editor that will be  installed by default on any UNIX system.

It is small, fast and efficient; useful to make some minor editing of system files, editing of huge data files or when a slow network link is involved.

It is important to learn at least the basics of vi.

Vi: Start-up, Modes, Save and Quit

To enter Vi type:

 vi FILENAME

Vi has two modes: Normal (command) mode and edit mode.

 Switch to normal mode:     <ESC>
 Switch to edit mode:       i or a

Getting out of Vi (change to normal mode <ESC>):

 Exit Vi:                       :q
 Exit Vi (ignore changes):      :q!
 Save:                          :w
 Save and Exit:                 :wq

Switch to (edit) an other file:

:edit FILENAME

Getting help:

:help topic

Vi: Move, Delete and Paste

Change to normal mode with <ESC>.

Move: Use either the arrow keys or the hjkl keys:

 h (left)    j (down)    k (up)    l (right)

 Getlocation and file status:   Ctrl-g
 Moves to end of the file:      Shift-G
 Moves line NUMBER:             NUMBER Shift-G

Delete:

 Delete a character:     x
 Delete a line:          dd

For multiple deletion precede command with a number.

 Delete 5 characters:     5 x
 Delete 7 lines:          7 dd

Paste: inserts all you deleted with the preceding delete.

 Paste:	                 p

Vi: Search and Replace

Change to normal mode with <ESC>.

Search (Wraped around at end of file):

  Search STRING forward :   / STRING.
  Search STRING backward:   ? STRING.

  Repeat search:   n
  Repeat search in opposite direction:  N  (SHIFT-n)

Replace: Same as with sed, Replace OLD with NEW:

 First occurrence on current line:      :s/OLD/NEW

 Globally (all) on current line:        :s/OLD/NEW/g

 Between two lines #,#:                 :#,#s/OLD/NEW/g

 Every occurrence in file:              :%s/OLD/NEW/g

 VIM QUICK REFERENCE CARD or VI Cheat Sheet

What is Shell Scripts

Shell scripts are text files that automate a series of UNIX environment-based commands that otherwise must be performed one at a time. Shell scripts are often used to automate command sequences that repeat, such as services that start or stop on system start up or shut down.

Any command that can be performed from the command line, such as ls, can be included in a shell script. Similarly, any command that can be included in a shell script can be performed on the UNIX environment command line.

Users with little or no programming experience can create and run shell scripts. You initiate the sequence of commands in the shell script by simply entering the name of the shell script on a command line.

Determining the Type of Shell to Run a Shell Script

There are several different shells available in the Solaris OS. Two of the most commonly used shells are the Bourne shell and the Korn shell.

To ensure that the correct shell is used to run a shell script, the first line of the script should always begin with the characters #!, followed immediately by the absolute path name of the shell required to run the script. These must be the only characters on the first line of the file.

#!/full-pathname-of-shell

For example:

#!/bin/sh

or

#!/bin/ksh

Comments

Comments are text entries that often provide information about a shell script. They are inserted into a shell script but have no effect on the script itself. Comments are ignored by the shell and are solely for the benefit of the user.

Comments are preceded by the hash (#) character. Whenever the shell encounters a word beginning with the # character it ignores all text on that line.

For example:

# this is a comment ls -l # list the files in a directory

Setting Korn Shell Options

Options are switches that control the behavior of the Korn shell. Options are boolean, meaning that they can be either on or off.

To switch an option on, type:

$ set -o option_name

To switch an option off, type:

$ set +o option_name

To show current option settings, type:

$ set -o
Note: The set -o and set +o options can only change a single option setting at a time.

Protecting File Content During I/O Redirection

Redirecting standard output to an existing file overwrites the previous file content, which results in data loss. This process of overwriting existing data is known as clobbering. To prevent an overwrite from occurring, the shell supports a noclobber option.

When the noclobber option is set, the shell refuses to redirect standard output to the existing file and displays an error message to the screen.

The noclobber option is activated in the shell using the set command. For example:

$ set -o noclobber
$ set -o | grep noclobber
noclobber        on
$ ps -ef > file_new
$ cat /etc/passwd > file_new
ksh: file_new: file already exists
$

Deactivating the noclobber Option

To temporarily deactivate the noclobber option, use the >| deactivation syntax on the command line. The noclobber option is ignored for this command line only, and the contents of the file are overwritten.

$ ls -l >| file_new

Note: There is no space between the > and | on the command line.

To deactivate the noclobber option, perform the following commands:

$ set +o noclobber $ set -o | grep noclobber
noclobber off
$ ls -l > file_new $

Using Korn Shell Functions

Functions are a powerful feature of shell programming used to construct customized commands. A function is a group of UNIX commands organized as separate routines. Using a function involves two steps:

  1. Define the function.
  2. Invoke the function.

Defining a Function

A function is defined using the general format:

function_name { command; . . . command; }
Note: A space must appear after the first brace and before the closing brace.

Function Examples

The following example creates a function called num to perform the who command, directing the output to the wc command to display the total number of users currently logged on the system:

$ function num { who | wc -l; }
$ num
	 9

The following example creates a function called list to perform the ls command, directing the output to the wc command to display the total number of subdirectories and files in the current directory:

$ function list { ls -al | wc -l; }
$ list
	 34
Note: If a command name is defined as a function and an alias, the alias takes precedence.

To display a list of all functions, use the following command:

$ typeset -f
function list
{
ls -al | wc -l; }
function num
{
who | wc -l; }

To display just the function names, use the following command:

$ typeset +f
list
num

Configuring the Shell Environment

The shell secondary prompt string is stored in the PS2 shell variable, and you can customize it as needed.

$ PS2="somethings missing >" $ echo $PS2 somethings missing > $ In this example, the secondary prompt displays the message that the command line is incomplete.
Note: To have this secondary shell prompt appear in every shell, it must be included in the user's Korn shell initialization file (usually named .kshrc).

Korn Shell Alias Utility

An alias is a shorthand notation in the Korn shell to enable you to customize and abbreviate UNIX commands. An alias is defined by using the alias command.

3>Command Format

The alias command syntax is as follows:

alias name=command_string

For example:

$ alias dir='ls -lF'

The shell maintains a list of aliases that it searches when a command is entered. If the first word on the command line is an alias, the shell replaces that word with the text of the alias. When an alias is created, the following rules apply:

  • There can be no space on either side of the equal sign.
  • The command string must be quoted if it includes any options, metacharacters, or spaces.
  • Each command in a single alias must be separated with a semicolon.

Predefined Korn Shell Aliases

The Korn shell contains several predefined aliases, which you can display by using the alias command. User-defined aliases are also displayed.

$ alias
autoload='typeset -fu'
command='command '
functions='typeset -f'
history='fc -l'
integer='typeset -i'
local=typeset
nohup='nohup '
r='fc -e -'
stop='kill -STOP'
suspend='kill -STOP $$'

User-Defined Aliases

Aliases are commonly used to abbreviate or customize frequently used commands. For example:

$ alias h=history $ $ h 278 cat /etc/passwd 279 pwd 280 cp /etc/passwd /tmp 281 ls ~ 282 alias h=history 283 h

Using the rm, cp, and mv commands can inadvertently result in loss of data. As a precaution, you can alias these commands with the interactive option.

For example:
$ alias rm='rm -i'
$ rm dat1
rm: remove  dat1: (yes/no)? no
$

By creating a cp -i and mv -i alias, the shell prompts you before overwriting existing files.

You can deactivate an alias temporarily by placing a backslash () in front of the alias on the command line. The backslash prevents the shell from looking in the alias list, thereby causing it to perform the original rm command.

For example:

$ rm file1 rm: remove file1 (yes/no)? no $ $ rm file1 $ ls file1 file1: No such file or directory

Command Sequences

You can group several commands together under a single alias name. Individual commands are separated by semicolons. For example:

$ alias info='uname -a; id; date' $ info SunOS host1 5.8 Generic sun4u sparc SUNW,Ultra-5_10 uid=102(user2) gid=10(staff) Fri Jun 30 15:22:47 MST 2000 $
In the next example, an alias is created using a pipe (|) to direct the output of the ls -l command to the more command. When the new alias is invoked, a directory listing appears. If the listing is longer than the available space on the screen, the --More-- line item appears at the end of the list, indicating that additional directory contents are displayed on the subsequent screen.

For example:

$ alias ll='ls -l | more' $ cd /usr $ ll total 136 drwxrwxr-x 2 root bin 1024 May 13 18:33 4lib drwx------ 8 root bin 512 May 13 18:14 aset drwxrwxr-x 2 root bin 7168 May 13 18:23 bin drwxr-xr-x 4 bin bin 512 May 13 18:13 ccs drwxrwxr-x 5 root bin 512 May 13 18:28 demo

Removing Aliases

Use the unalias command to remove aliases from the alias list. The unalias command syntax is as follows:
$ unalias alias_name

For example:

$ unalias h
$ h
ksh: h:  not found

Note: To pass the new aliases to every shell invoked, place it in your Korn shell initialization file (usually called .kshrc).

Other User Korn Shell Variables

Other Korn shell variables are described in the table.

Other Korn Shell Variables
Variable Use
LPDEST
Set your default printer. Use this value if using the /usr/bin/lp command.
PRINTER
Set your default printer. Use this variable if using the lpd-based compatibility commands (lpr, lpq, and lprm)
TERM,
Define the terminal. This variable should be reset in the /etc/profile file. When the user invokes an editor, the system looks for a file with the same name that is defined in this environment variable. The system searches the directory referenced by TERMINFO to determine the terminal characteristics.
MANPATD
Set the hierarchies of man pages that are available.

Managing Jobs in the Korn Shell

The Korn shell enables you to run jobs in the background, which frees your terminal to perform other commands. You can also run jobs in the foreground. You can use Korn shell commands to list current jobs as well as to stop a job.

A job is a process that the shell can manage. Shells start and control jobs. Because jobs are processes, each job has an associated PID. The shell also assigns each job a sequential job ID number.

The shell enables you to run multiple jobs at the same time. Job control commands enable you to manage multiple jobs within a shell. There are three types of jobs that shells manage: foreground jobs, background jobs, and stopped jobs.

When you perform a command in a terminal window, the command occupies that terminal window until it completes. This type of job is called a foreground job.

When you enter an ampersand (&) symbol at the end of a command line, the command runs without occupying the terminal window. The shell prompt is displayed immediately after you press Return. This type of job is called a background job.

If you press Control-Z for a foreground job, or perform the stop command for a background job, the job stops. This job is called a stopped job.

Job control commands enable you to place jobs in the foreground or background, and to start or stop jobs. The table describes the commands you can use for job control.

Job Control Commands
Command Value
jobs
Lists all jobs that are currently running or are stopped in the background
bg %n
Runs the current or specified job in the background (n is the job ID)
fg %n
Brings the current or specified job into the foreground (n is the job ID)
Control-Z
Stops the foreground job and places it in the background as a stopped job
stop %n
Stops a job that is running in the background (n is the job ID)


Note: You can control a job by using these commands only in the shell in which the job started.

Running a Job in the Background

To run a job in the background, enter the command you want to run along with an ampersand (&) symbol at the end of the command line. For example, you can run the sleep command in the background as follows:

$ sleep 500 &
[1]     3028
$
Note: The sleep command suspends execution of a program for n seconds.

The shell returns the job ID number it assigned to the command, contained in brackets, and the PID associated with the command. You use the job ID number to manage the job with job control commands. The kernel can use the PID number to manage the job.

When a background job has finished and you press Return, the shell displays a message indicating that the job is done.

[1] + Done
sleep 500 &
$

Listing Current Jobs

 You can use the jobs command to display the list of jobs that are running or stopped in the background. For example:$ jobs
[1] + Running
sleep 500 &
$

Bringing a Background Job Into the Foreground

 You can use the fg command to bring a background job to the foreground. For example:


$ fg %1
sleep 500

Note: The foreground job occupies the shell until the job is completed, stopped, or stopped and placed into the background.


Sending a Foreground Job to the Background

You can use the Control-Z keys and bg command to return a job to the background. The Control-Z keys suspend the job, and place it in the background as a stopped job. The bg command runs the job in the background. For example:

$ sleep 500
^Z[1] + Stopped (SIGTSTP)        sleep 500
$ jobs
[1] + Stopped (SIGTSTP)        sleep 500
$ bg %1
[1]     sleep 500&
$ jobs
[1] +  Running                 sleep 500
$

Note: When you place a stopped job in either the foreground or the background the job restarts.

command1 $(command2)

To ignore the special meaning of the dollar sign metacharacter, use the following command:

$ echo '$SHELL'
$SHELL