Introduction to Linux shell Scripting

[Aerosol]

Unix shell", "Bash" or "shell" is an interpreter or a command line interface run in a text window mode on a Linux or Unix like machines. It is used to interpret users commands, process them and forward to the lower levels of Operating System where they can be executed and perform desired action. Althought there are more shells like Korn shell, Perl shell, zsh, probably the most popular are Csh and Bash. Interpreter or shell also exist on Microsoft Windows and in the past was MS-DOS Prompt, now there are Command promt or Windows PowerShell.

When the linux bash is invoked as an interactive shell, first it reads and executes commands from /etc/profile file. If /etc/profile doesn’t exist, commands from ~/.bash_profile, ~/.bash_login and ~/.profile are executed in the given order. Typically bash_profile executes ~/.bashrc. This file only runs during shell log in, and when a login shell exits, Bash reads and executes commands from the ~/.bash_logout file.

Shell scripting

Instead of running a single command, shell also has the ability of executing multiple or an entire set of complex sequences of commands from the external file known as a "shell script" or simply "script". A script might contain just a single command or large list of commands. It might contain functions, conditional constructs, loops and other logical structures common in programming. Therefore, a Bash shell script is a computer program written in the Bash programming language and Shell scripting is the art of creating and maintaining those scripts. Shell scripting may come from the desire to automate some routine procedure to make things easier for end users, or from a real need of automating many system administration tasks, such as performing disk backups or evaluating system logs.

Writing a Simple Bash Script

Shell script is basically a simple text file with sequences of commands and file permissions set for execution. Once interpreter "shell" calls a file, commands in the script are processed and executed.

Every shell script starts with the "#!" sequence often called "shebang". Shebang represents a shell by which script will be executed. It is an absolute path where interpreter can be found. So for example if we would like to use Bourne shell as an interpreter, we can use sequence #!/bin/sh, or #!/bin/bash:

LinuxBox#vi my_script.sh
 #!/bin/bash
 # This is my 1st script
 echo "Hi, this is hello from shell script !" 

 :wq!

(Other lines of code in script that start with sign "#" are comments. Comments are lines of code that are not processed by shell. Notice that in comments, "#" signs are not followed by the "!" exclamation mark sign - this is only reserved for "shebang" syntax. A good practice when writing a script is to make a simple name or explanation and put it in the comment line near the beginning of script.)

After creating and saving my_script.sh in VI text editor, we have to change file's premissions to make it executable. To make a script executable we can set premissions something like 700 or similar:

LinuxBox# chmod 700 my_script.sh
 LinuxBox# ls -l
 LinuxBox# -rwx------ 1 root root 29 Apr 29 13:07 my_script.sh 

After we have set executable permissions, script can be called and start doing it's job:

LinuxBox#./my_script.sh 
 Hi, this is hello from shell script ! 
 LinuxBox#

Once we know the basics, we can make some sripts and use commands that actualy do something usefull. For examaple if we want to make a simple backup script which syncs our data, we could write a script like this:

LinuxBox# vi my_backup.sh 
 #!/bin/bash
 # my simple backup script that uses rsync and tar 
 tar -czf /home/ittutorials/Documents/my_pictures.tar.gz /home/ittutorials/Pictures
 rsync -avh --exclude="*.bak" /home/ittutorials/Documents/ /media/sdb/my_backup/
 LinuxBox# 

( The script could be called every day via CronJob. That way we could have automatic backup procedure. See CroonJob tutorial for more info on task scheduling and automation.)

Use of Variables

Simple batch jobs might be usefull for isolated tasks, but use of variables provides much more flexibility. Althought above script is useful, it has hard-coded paths. That might not be a problem, but when writing long scripts that reference paths often, better idea would be to utilize variables. That way we can put long path names into "memory boxes" (variables), and call them from variables whenever we want:

LinuxBox# vi my_backup.sh 
 #!/bin/bash
 # my simple backup script that uses variables and rsync

 SOURCEDIR=/home/ittutorials/Documents/MyPictures
 DESTDIR=/media/sdb/userdata/my_backup/

 echo "Starting backup..."
 echo "Backuping" $SOURCEDIR 
 rsync -avh --exclude="*.bak" $SOURCEDIR $DESTDIR
 echo "Backuping to destination" $DESTDIR
 echo "Backup is successful !"  

 LinuxBox# 

( This way, if we change variable for directory path in only one location, change will take effect for whole script where this variable is used, which makes maintenance of scripts a lot easier.)

Global vs. Local variables

Global variables can be used anywhere in this bash script. Once the global variable is defined and it's value is set, it can be used in functions, loops and any other logical structure inside the script in which it is defined.

Local variables, are defined and used only inside the functions. It's life is limited to specific function in which it is defined, and it's value can't be used outside that function. This is often called the scope of variable.

Local variables are define with the reserved word "local" used in front of variable name:

 LinuxBox# vi use_of_variables.sh 
 #!/bin/bash
 # Local vs. Global variables

 VAR="Global variable"
 function my_function {
    #This variable is local to my_function function 
	local VAR="Local variable"
	echo $VAR
 }

 echo $VAR 
 my_function
 echo $VAR 
 # Notice that global variable did not change
 # "local" is shell reserved word for local variables

 LinuxBox# ./use_of_variables.sh 
 Global variable
 Local variable
 Global variable

 LinuxBox#

(Notice that even though Global and Local variable have the same name "$VAR", value of global variable did not change once we defined and called Local variable from function.)

Environment variables

Environment variables are a set of dynamic named values that can affect the way running processes will behave. For example, an environment variable with a standard name can store the location that a particular computer system uses to store temporary files.

Some Common and often used environment variables are:

• PATH - Sets the search path for any executable command. Similar to the PATH variable in MSDOS.
  
• HOME - Home directory of the user.
  
• MAIL - Contains the path to the location where mail addressed to the user is stored.
  
• PS1 - Primary prompts in bash. PS1 is set to $ by default.
  
• PS2 - Secondary prompts in bash. PS2 is set to '>' by default.
  
• USER - User login name.
  
• HOSTNAME - The system's host name.
  
• MACHTYPE - The CPU architecture that the system is running on.
  
• TERM - indicates the terminal type being used. This should be set correctly for editors like vi to work correctly.
  
• SHELL - Determines the type of shell that the user sees on logging in.
  

To see what are the values of the above environment variables, just do an echo of the name of the variable preceeded with a $. For example, to determine the type of shell enter:

LinuxBox# echo $SHELL
 /bin/bash 
 LinuxBox# 

To see all environment variables and their values under UNIX-like operating systems, use set or printenv command:

LinuxBox# set
 BASH=/bin/bash
 BASHOPTS=checkwinsize:cmdhist:expand_aliases:extquote:force_fignore:histappend:hostcomplete:interactive_comments:login_shell:progcomp:promptvars:sourcepath
 BASH_ALIASES=() 
 BASH_SOURCE=()
 BASH_VERSINFO=([0]="4" [1]="1" [2]="5" [3]="1" [4]="release" [5]="i486-pc-linux-gnu")
 BASH_VERSION='4.1.5(1)-release'
 COLUMNS=80
 DIRSTACK=()
 EUID=0
 GROUPS=()
 HISTCONTROL=ignoreboth
 HISTFILE=/root/.bash_history
 HISTFILESIZE=500
 HISTSIZE=500
 HOME=/root
 HOSTNAME=ittutorials
 HOSTTYPE=i486
 IFS=$' \t\n'
 LINES=24
 LOGNAME=root
 ...

 LinuxBox# 

To set environment variable in Bourne shell (sh and bash), simply use "export" command with the name and value of environment variable (export var=value):

LinuxBox# export PATH=/home/JohnnyBoy/bin 
 LinuxBox# 

Or, to add a folder in the current PATH without overwriting the current path:

LinuxBox# export PATH=$PATH:/home/JohnnyBoy/bin 
 LinuxBox# 

(Changing env variable like PATH will enable You to run script located in the folder You have just added to the path, without providing the full path of the script. After the change, scripts will be able to run from anywhere in the system path by simply calling the name of the script.)

Reading user's input

Non-interactive scripts are useful, but we have to provide new information from the start when the script gets iniciated. Better way is to make interactive script that can catch user's input as script's arguments while the script is running. Catching user's input is usualy done from the command line interface during script's run-time.

To read user input and put the value in variable of a script, we'll use "read" syntax:

LinuxBox# vi my_script.sh

 #!/bin/bash
 # Reading of user's input
 echo "Please enter your name: "
 read user_name
 echo "Nice to meet you $user_name" 
 echo "Can you please enter Your 2 favorite colors? "
 read color1 color2
 echo "Your favorite colors are $colo1 and $color2."

 LinuxBox# ./my_script.sh
 Please enter your name:
 John
 Nice to meet you John
 Can you please enter Your 2 favorite colors?
 Black White
 Your favorite colors are Black and White.

 LinuxBox#  

Or we could simply use "$" sign with the number of argument we want to catch. So if we would like to catch 1st argument we can use $1 as in example below:

LinuxBox# vi my_script.sh 
 #!/bin/bash 
 # Reading and displaying of arguments
 echo "You just entered the words: $1 $2"

 LinuxBox# ./my_script.sh John Malkovich
 You just entered the words: John Malkovich
 LinuxBox#

(Notice that catching data this way doesn't wait users input. We have to provide input along with the script name, as in the example above, where input is "Jonh Malkovich".)

Source