root@test:/var/log/nginx# wget -qO – icanhazip.com
www.xxx.yyy.zzz
root@test:/var/log/nginx# wget -qO- http://ipecho.net/plain | xargs echo
www.xxx.yyy.zzz
root@test:/var/log/nginx# wget -qO – icanhazip.com
www.xxx.yyy.zzz
root@test:/var/log/nginx# curl icanhazip.com
www.xxx.yyy.zzz
root@test:/var/log/nginx#
Category Archives: Shell Script
sed to play with data or parsing your text
To select all the lines starting from STARTING_PATTERN up to blank line ^$ and then delete those lines.
# sed ‘/STARTING_PATTERN/,/^$/d’ filename
To edit files in place, use -i option.
# sed -i ‘/STARTING_PATTER/,/^$/d’ filename
Insert multiple lines into a file after specified pattern.
# sed ‘/cdef/r add.txt’ input.txt
# sed ‘/cdef/r add.txt’ input.txt
input.txt: abcd accd cdef line web | add.txt: line1 line2 line3 line4 |
Output : abcd accd cdef line1 line2 line3 line4 line web |
If you want to apply the changes in input.txt file. Then, use -i with sed.
# sed -i ‘/cdef/r add.txt’ input.txt
If you want to use a regex as an expression you have to use the -E tag with sed.
# sed -E ‘/RegexPattern/r add.txt’ input.txt
How to configure Proxy Settings for the Unix / Linux Console
You can use the following methods to configure your console to use a proxy server so that console based programs like wget could get connect to the internet through the proxy.
- 1 – Set the environment variable
-
# export http_proxy=http://DOMAIN\USERNAME:PASSWORD@SERVER:PORT/
# export ftp_proxy=http://DOMAIN\USERNAME:PASSWORD@SERVER:PORT/In the above configuration you can ommit the DOMAIN\USERNAME:PASSWORD@ part if you are not using proxy authentication.
Note: If you get the following error when you try to use wget, you might have to pass the proxy authentication credentials to wget as arguments.
-
Connecting to SERVER:PORT... connected.
Proxy request sent, awaiting response... 407 Proxy Authentication Required
11:14:45 ERROR 407: Proxy Authentication Required. - 1.1 – Passing proxy authentication credentials to wget as arguments
-
$ wget --proxy-user "DOMAIN\USERNAME" --proxy-passwd "PASSWORD" URL
-
- 2 – Configure the proxy settings in the .bashrc
- If you want set the proxy for all the users you can do it in the system wide .bashrc file.
#proxy settingsnano /etc/bash.bashrc
export http_proxy=http://DOMAIN\USERNAME:PASSWORD@SERVER:PORT/
export ftp_proxy=http://DOMAIN\USERNAME:PASSWORD@SERVER:PORT/Note: The system wide .bashrc file may not be available in all Linux systems and only can be used if you are using the bash shell
- 2.1 – Having an alias for wget with proxy
- If you don’t want to pass the proxy arguments to wget all the time, you create an alias for wget in the .bashrc file
alias wget 'wget --proxy-user "DOMAIN\USERNAME" --proxy-passwd "PASSWORD"'
Delete Files Older Than x Days on Linux / Unix
The find utility on linux allows you to pass in a bunch of interesting arguments, including one to execute another command on each file. We’ll use this in order to figure out what files are older than a certain number of days, and then use the rm command to delete them.
Command Syntax
find /path/to/files* -mtime +5 -exec rm {} ;
Note that there are spaces between rm, {}, and ;
Explanation
- The first argument is the path to the files. This can be a path, a directory, or a wildcard as in the example above. I would recommend using the full path, and make sure that you run the command without the exec rm to make sure you are getting the right results.
- The second argument, -mtime, is used to specify the number of days old that the file is. If you enter +5, it will find files older than 5 days.
- The third argument, -exec, allows you to pass in a command such as rm. The {} ; at the end is required to end the command.
Collecting Unix System Information
At the very least, collect the following information for each system that you have:
1. Hostname:
% hostname
2. Hostname aliases:
% grep `hostname` /etc/hosts | awk ‘{ print $3 }’
3. Host network addresses:
% grep `hostname` /etc/hosts | awk ‘{ print $1 }’
4. Host ID:
% hostid
5. System serial number:
On the back of most all computers.
6. Manufacturer of the system’s hardware:
On the front of most computers
7. System model name:
On the front of most computers
8. CPU type:
% uname -a
9. Application architecture:
% uname -a
10. Kernel architecture:
% uname -a
11. Amount of main memory:
Can be found at boot time
% dmesg
12. Operating system name:
% uname -a
13. Operating system version:
% uname -a
14. Kernel version:
% uname -a
15. Disk configuration:
% df
How to check and install missing perl modules
- Check if module is installed. Errors mean missing module.
# perl -MModule::Name -e 1
- See documentation of the module if installed.
# perldoc Module::Name
- Open CPAN shell
# perl -MCPAN -e shell
- To reconfigure the shell if needed.
cpan>o conf init
- Install an available module.
cpan> install HTML::Template
- You can run the Perl CPAN module via command line perl and get it installed in a single line:
# perl -MCPAN -e ‘install HTML::Template
- Force install if test fails.
cpan> force install Module::Name
- To manual install perl modules. Unzip and go to module directory.
# tar -zxvf HTML-Template-2.8.tar.gz
-
# perl Makefile.PL # make # make test # make install
How to use Unix Editor Vi (Vim)
Today vi is considered the standard. It is the only editor that will be installed by default on any UNIX system.
It is small, fast and efficient; useful to make some minor editing of system files, editing of huge data files or when a slow network link is involved.
It is important to learn at least the basics of vi.
Vi: Start-up, Modes, Save and Quit
To enter Vi type:
vi FILENAME
Vi has two modes: Normal (command) mode and edit mode.
Switch to normal mode: <ESC> Switch to edit mode: i or a
Getting out of Vi (change to normal mode <ESC>):
Exit Vi: :q Exit Vi (ignore changes): :q! Save: :w Save and Exit: :wq
Switch to (edit) an other file:
:edit FILENAME
Getting help:
:help topic
Vi: Move, Delete and Paste
Change to normal mode with <ESC>.
Move: Use either the arrow keys or the hjkl keys:
h (left) j (down) k (up) l (right) Getlocation and file status: Ctrl-g Moves to end of the file: Shift-G Moves line NUMBER: NUMBER Shift-G
Delete:
Delete a character: x Delete a line: dd
For multiple deletion precede command with a number.
Delete 5 characters: 5 x Delete 7 lines: 7 dd
Paste: inserts all you deleted with the preceding delete.
Paste: p
Vi: Search and Replace
Change to normal mode with <ESC>.
Search (Wraped around at end of file):
Search STRING forward : / STRING. Search STRING backward: ? STRING. Repeat search: n Repeat search in opposite direction: N (SHIFT-n)
Replace: Same as with sed, Replace OLD with NEW:
First occurrence on current line: :s/OLD/NEW Globally (all) on current line: :s/OLD/NEW/g Between two lines #,#: :#,#s/OLD/NEW/g Every occurrence in file: :%s/OLD/NEW/g VIM QUICK REFERENCE CARD or VI Cheat Sheet
What is Shell Scripts
Shell scripts are text files that automate a series of UNIX environment-based commands that otherwise must be performed one at a time. Shell scripts are often used to automate command sequences that repeat, such as services that start or stop on system start up or shut down.
Any command that can be performed from the command line, such as ls, can be included in a shell script. Similarly, any command that can be included in a shell script can be performed on the UNIX environment command line.
Users with little or no programming experience can create and run shell scripts. You initiate the sequence of commands in the shell script by simply entering the name of the shell script on a command line.
Determining the Type of Shell to Run a Shell Script
There are several different shells available in the Solaris OS. Two of the most commonly used shells are the Bourne shell and the Korn shell.
To ensure that the correct shell is used to run a shell script, the first line of the script should always begin with the characters #!, followed immediately by the absolute path name of the shell required to run the script. These must be the only characters on the first line of the file.
#!/full-pathname-of-shell
For example:
#!/bin/sh
or
#!/bin/kshComments
Comments are text entries that often provide information about a shell script. They are inserted into a shell script but have no effect on the script itself. Comments are ignored by the shell and are solely for the benefit of the user.
Comments are preceded by the hash (#) character. Whenever the shell encounters a word beginning with the # character it ignores all text on that line.For example:
# this is a comment ls -l # list the files in a directory
Setting Korn Shell Options
Options are switches that control the behavior of the Korn shell. Options are boolean, meaning that they can be either on or off.
To switch an option on, type:
$ set -o option_name
To switch an option off, type:
$ set +o option_name
To show current option settings, type:
$ set -o
Note: The set -o and set +o options can only change a single option setting at a time. |
Protecting File Content During I/O Redirection
Redirecting standard output to an existing file overwrites the previous file content, which results in data loss. This process of overwriting existing data is known as clobbering. To prevent an overwrite from occurring, the shell supports a noclobber option.
When the noclobber option is set, the shell refuses to redirect standard output to the existing file and displays an error message to the screen.
The noclobber option is activated in the shell using the set command. For example:
$ set -o noclobber $ set -o | grep noclobber noclobber on $ ps -ef > file_new $ cat /etc/passwd > file_new ksh: file_new: file already exists $Deactivating the noclobber Option
To temporarily deactivate the noclobber option, use the >| deactivation syntax on the command line. The noclobber option is ignored for this command line only, and the contents of the file are overwritten.
$ ls -l >| file_new
Note: There is no space between the > and | on the command line. |
To deactivate the noclobber option, perform the following commands:
$ set +o noclobber $ set -o | grep noclobber
noclobber off
$ ls -l > file_new $
Using Korn Shell Functions
Functions are a powerful feature of shell programming used to construct customized commands. A function is a group of UNIX commands organized as separate routines. Using a function involves two steps:
- Define the function.
- Invoke the function.
Defining a Function
A function is defined using the general format:
function_name { command; . . . command; }
Note: A space must appear after the first brace and before the closing brace. |
Function Examples
The following example creates a function called num to perform the who command, directing the output to the wc command to display the total number of users currently logged on the system:
$ function num { who | wc -l; }
$ num
9
The following example creates a function called list to perform the ls command, directing the output to the wc command to display the total number of subdirectories and files in the current directory:
$ function list { ls -al | wc -l; }
$ list
34
Note: If a command name is defined as a function and an alias, the alias takes precedence. |
To display a list of all functions, use the following command:
$ typeset -f
function list
{
ls -al | wc -l; }
function num
{
who | wc -l; }
To display just the function names, use the following command:
$ typeset +f list numConfiguring the Shell Environment
The shell secondary prompt string is stored in the PS2 shell variable, and you can customize it as needed.
$ PS2="somethings missing >" $ echo $PS2 somethings missing > $ In this example, the secondary prompt displays the message that the command line is incomplete.
Note: To have this secondary shell prompt appear in every shell, it must be included in the user's Korn shell initialization file (usually named .kshrc). |