Learn Shell Scripting In One Day

Let’s learn “Shell Scripting". So, in this post I wanted to talk about “What is a shell script?" & "learner guide  to shell scripting with examples". Before jumping into learning on how to code and execute shell scripts, let’s talk a bit about “Linux” and history of Linux.

Linux is technically the kernel, so, there are other utilities such as GNU tools, GCC compiler etc,. which would ideally make a complete operating system (OS). That is the reason it is called GNU (GNUs not Unix) Linux not simply Linux. Ideally one must say 'GNU Linux' not just 'Linux'.  GNU Linux is one of the widely used OS in market, because it is open-source, so anyone could download/re-use the kernel, publish a new Linux distribution under the guidelines of GPL & Free Software Foundation.

Topics

Part 1

• A Brief History Of Linux
• Linux OS Introduction (A Brief Note)
• What is a Shell in Linux environment?
• Let’s create first shell script
• How to execute/run a script file?
• How to declare a variable and recall it?
• Let’s perform numerical operations using variables
• Let’s perform logical operations using variables

Part 2

• Conditional operations using “if…else…fi” statement
• Conditional operations using “case..esac” statement
• Usage of “for” loop (iteration) statement
• Usage of “while” loop
• Let’s look into some of the commonly used standard variables

Part 3

• Let’s look into some of the commonly used built-in standard shell variables
• Shell redirection and usage of “*” (wildcard)
• Regular expression commands (regex)
• Usage of grep
• Let’s talk about the powerful language “awk”

A Brief History Of Linux. . .

In 1960’s MIT (Massachusetts Institute of Technology) developed an experimental operating system called “Multics” (Multiplexed Information and Computing Service).

At later stage, one of the developers (AT & T) of the Multics project dropped out of it and developed their own OS called “Unics” (Unix) in 1970’s. This was originally written in “C” language.

The GNU Project, started in 1983 by “Richard Stallman”, had the goal of creating a "complete Unix-compatible software system" composed entirely of free software. Later in the year 1985, Stallman started the Free Software Foundation and wrote the GNU General Public License (GNU GPL) in 1989.

Down the line, after 20 years of Unix development, “Andrew Tanenbaum” developed lighter version of Unix called “MINIX” (Minimal Unix) which could run on small computers. This inspired “Linus Benedict Torvalds” (Father of Linux) to start research and work on development of Linux in early 1990s. That’s how the “Linux” kernel was born.

After that we all know, there are many variants/version/flavors/distribution of Linux available in market such as Red Hat Enterprise Linux (by Red Hat), SUSE Linux Enterprise Server (SLES by Novell), Oracle Enterprise Linux (OEL by Oracle), Ubuntu (by Canonical), Scientific Linux (produced by Fermilab, CERN, DESY, ETH Zurich), Fedora (by Red Hat, community based), Linux Mint, ArchLinux (developed by Aaron Griffin and others), Debian (by Debian Project), CentOS(Community Enterprise Operating System – now part of Red Hat), Gentoo, Slackware, and a lot many.

Linux OS Introduction (A Brief Note)

At the core level, there is “kernel” and GNU utilities which makes a complete operating system. At broader level there is “shell” (command interpreter) which talks to kernel to get the task done, so kernel would actually manage IO, hardware and other core activities of an operating system.

By default, every operating system got a file system which actually provides a framework to store data on storage devices. Hence, in Linux (RHEL variant), we got Ext2/3/4 & XFS being used as default (RHEL7/8 uses XFS as the default file system).

At minimal, we need ‘/’(root), ‘/boot’ and ‘Swap’ file systems as recommended by Red Hat to get started and to install other applications on it. So, ‘/’(root) is the top-level file system path where further installations and other files gets stored/created. The below snap shows the inverted tree structure of the XFS file system on a RHEL7/8 version:

What is a Shell in Linux environment?

Simply put, a shell in Unix/Linux environment is a command interpreter which could understand & processes user or system commands/inputs and provides the respective output. Whenever a user types a command and hit “Enter” key, it would search in the desired path (PATH) to see if this is a binary executable (command) and once found processes it accordingly, otherwise, throws out an error.

Following are the most popular shell interpreters being used:

→  Bourne Again Shell (bash)

→  C Shell (csh : improved version of tcsh)

→  Korn Shell (ksh)

How to check the default shell being used?

-- Run the command “echo $SHELL":

NOTE: In this page, I've assumed that the reader has a Linux ( I've used RHEL6.x VM for all demonstrations) box (minimal mode) ready to learn/practice "examples", and login as either root or as a regular user.

There are a few third party websites which offers online terminal which doesn't need any installation or workstation, this can be accessed over a browser and one such example is given here :  http://bellard.org/jslinux/

Let’s create first shell script file "welcome.sh"

Let’s write a simple shell script which would print a message “Welcome to Linux Shell Scripting.  Session by : www.SimplyLinuxFAQ.com” on the console/terminal when executed.

So, let’s create a file called “welcome.sh” file with the following text:

#!/bin/bash

# This is a comment & this is my first script file.

echo “Welcome to Linux Shell Scripting. Session by : www.simplylinuxfaq.com”

To create the above file, you could use any text editor such as “vi” ( there are many text editors such as vi, gedit, pico, emacs etc.,) and create the file with the above contents. By default "vi" editor is available in most Linux variants. Otherwise one could create the file using redirection operators as shown below:

This is the standard being practiced with scripts, so it should always begin with “#!” symbol which is popularly called as “shebang”. Also, the file name should end with “.sh” as a good practice. So, this basically indicates that this is a shell script file.  Not only that, whenever someone wants to check what type of file is this by using the "file" command it would be interpreted as a "Shell Script Executable" file as shown above ('file' command is used to determine the type of file).

NOTE: If you are new to Linux/Unix then you could use basic commands such as "ls" (to list files/directories), "ls -l" (long listing of files/directories), "ls -al" (long list of files/directories including hidden), "cd <DirectoryName>" (to change the current working directory), "pwd" (print current working directory) as such on the terminal.

How to execute/run a script file?

To execute a script or to run it, it would need to have an execute permission set for the owner.

A file/directory would normally hold 3 permission bits (read-r, write-w, execute-x) set on 3 levels such as user (u - owner), group (g) & others (o). These permission bits are also represented in octal values such as 4-read, 2-write, 1-execute, and special bits such as 4 (SUID-Set User ID), 2 (SGID-Set Group ID), & 1 (Sticky Bit). So, numerically (in octal representation) permission could be set in 4-digit format, something like 0755 using "chmod" command otherwise, using permission bit set directly on user/group/others. The first digit is a special bit, second number shows what permission does a user/owner got, third digit shows for group and last one for others.

Let’s set the execute permission or rather first check what permission bits are already set using "ls -l" command on our first script that we had created “welcome.sh”:

As you could see the owner got only "rw" (read & wright) permission set and no "execute" permission.

Now, let’s try to execute this script file. You could use “bash” command line or execute in-line. Using “bash” would execute the file and it doesn’t need execute permission as shown below:

Let’s set the "execute" permission on the user/owner using "chmod" command for the file and then try to execute it as shown below "./welcome.sh" :

Let’s look at the three statements used in this first script file:

How to declare a variable and recall it?

A variable is usually a string/word which must begin with an alphabet and could be any combination of alphabets and numbers. So, in a shell environment we could use “VARIABLE_NAME=VALUE” format to declare a variable. Actually, a variable is referencing to a memory location to store some data.

NOTE: There should not be any space between 'VARIABLE_NAME=' and 'VALUE'.

Practice Example 1:

--------------------------

Let’s declare a variables and print the value in a new program called “var1.sh”. So, the script file "var1.sh" looks as shown below:

So, let’s execute this script file after setting the execute permission and check out what it shows on terminal. You would get similar output as shown below:

In the above script file "var1.sh", there are three variables declared:

- NAME, BLOG & NUM

- NAME & BLOG are character/string/word variables.

- NUM is a numerical variable.

The "echo" command could be used to print the value of a variable and variable should be prefixed with "$", so shell would understand that and replaces it with actual value of that variable.

Let’s perform numerical operations using variables

Now, let’s see how numerical operations are being performed using variables. So, to perform these operations we need to use the below format :

$(( VARIABLE1 Operator VARIABLE2 ))

Practice Example 1:

--------------------------

To demonstrate this, let’s create a new file called “num_operation.sh” file as shown below:

In the above demonstration, I’ve used “echo -e” this enables interpretation of backslash (\) which could be used to print “new lines”, “tab space”, and other special characters. So, we could use echo -e “\n” to print a new line, and echo -e “\t” to print a tab space. Likewise “echo -n” would print everything on the same line.

By default, each "echo" command would print text on new line, if you wish to print out additional new line using "echo" you could use "echo -e "\n" as I've done before.

So, let’s turn on the execute bit on this file and check what it prints when executed.

Let’s perform logical operations using variables

Logical operations are nothing but logical comparisons between variables, such as given below which are more commonly performed on numerical variables :

-- less-than (lt)

-- greater-than (gt)

--  equal-to (eq)

-- not-equal-to (ne)

-- less-than or equal to (le) &

-- greater-than or equal to (ge)

Operations such as “equal-to” (==) and “not-equal-to” (!=) are usually performed on character variables.

Other logical operations such as “&&” (logical AND), “||” (logical OR) are usually performed along with a conditional operator, and they are Boolean operators. There is also "-a" (logical AND) and "-o" (logical OR) available, these operators are mostly used inside if or conditional statements, whereas "&&" and "||" would be used outside conditional statements. We'll find out the usage as I demonstrate in the below section.

Logical AND (&&) or (-a) would evaluate to "1" (true) if all conditions are passed, otherwise evaluates to "0" (false). Like-wise logical OR or (-o) would evaluates to "1" (true) if any one condition is passed, otherwise "0" (false).

Practice Example 1:

--------------------------

Let's perform a simple logical operation where we'd compare a numerical variable with another variable and print which is greater or lesser & evaluate a character variable if it matches with character "y".

In this script file I've used "if..." loop (let’s talk about loops later). So, this is the script that I've created:

When I execute the script it would produce output as shown below:

So, the script does a logical comparison :

-  It checks to see if NUM1 variable is less than NUM2 and prints a message if so.

- In second "if" condition, it checks to see if variable ANS holds "y" and prints a message if it is true.

Practice Example 2:

--------------------------

Let’s demonstrate the usage of logical AND & OR operations now using an example script file “logical_operation.sh” as shown below:

Output after executing this script:

So, this is how the script performs a logical comparison :

- The first "if" condition prints the message only if variable "N1 is less than N2 & N3 is greater than N2".

- In second one, it checks to see if "N2 is greater than N3 Or N1 is less than N3", if so print message.

Practice Example 3:

--------------------------

Let's see the usage “&&” and “||” now. To demonstrate this, I’d declare 3 variables as “NUM1=10”, “NUM2=20” & “ KEY=’Y’ ” and compare them as shown below:

As you could see above when I execute [ $NUM1 -le $NUM2 -a “$KEY” == “Y” ] this statement it produces the output as “Both conditions are true”, this is because both the conditions are passed (1) hence it executes the AND (&&) function.

To further simply:

- 10 is lesser than 20 AND KEY is "Y", both are true, hence, logical AND (&&) executes the echo statement to print the message
"Both conditions are true".

Likewise, in the next statement [ $NUM1 -gt $NUM2 ] produces the out “10 is smaller”, because the condition is not passed (0) so it would execute the OR (||) function which would print the message “10 is smaller” on the console.

To further simply:

- 10 is not greater than 20 (conditions becomes false), hence, logical OR (||) executes the echo statement to print the message

  "10 is smaller".

The operator being used “[” is a special one which is also called “test” function. This could be used directly without “if” statement to check if condition evaluates to true (Boolean :1) or false (Boolean : 0) as shown above. If you run the command “which [“ you could get to see it is a binary file as shown below:

Practice Example 4:

--------------------------

Likewise, there is post-increment and post-decrement which can be performed on a numerical variable using the format “VAR++” & “VAR--” which is as good as “VAR=VAR+1” & “VAR=VAR-1”.

In above screen image, I’ve incremented N1 & decremented N2 initially and their values got changed from 10,10 to 11,9 as shown. Later, decremented N1 and incremented N2 which made it to reset it back to its original value of 10, 10. Same way there is pre-increment and pre-decrement which can be done.

Continue reading....