OS Module 2
OS Module 2
Introduction
Shell programming is a basic skill that every UNIX System administrator should
have. The Systems Administrator must be able to read and write shell programs
because
There are many tasks that can and should be quickly automated by using
shell programs
Many software products come with install scripts that have to be modified
for our system before they will work.
When we run the terminal, the Shell issues a command prompt (usually
$), where we can type our input, which is then executed when we hit the Enter
key. The output or the result is thereafter displayed on the terminal. The Shell
wraps around the delicate interior of an Operating system protecting it from
accidental damage. Hence the name Shell.
Kernel
The kernel is a computer program that is the core of a computer’s operating system,
with complete control over everything in the system. It manages following resources
of the Linux system –
File management
Process management
I/O management
Memory management
Device management etc.
It is often mistaken that Linus Torvalds has developed Linux OS, but
actually he is only responsible for development of Linux kernel. Complete Linux
system = Kernel + GNU system utilities and libraries + other management scripts
+ installation scripts.
Shell
A shell is special user program which provide an interface to user to
use operating system services. Shell accept human readable commands from user
and convert them into something which kernel can understand. It is a command
language interpreter that execute commands read from input devices such as
keyboards or from files. The shell gets started when the user logs in or start the
terminal.
Linux Shell
There are several shells are available for Linux systems like –
BASH (Bourne Again SHell) – It is most widely used shell in Linux
systems. It is used as default login shell in Linux systems and in macOS.
It can also be installed on Windows OS.
CSH (C SHell) – The C shell’s syntax and usage are very similar to the C
programming language.
KSH (Korn SHell) – The Korn Shell also was the base for the POSIX
Shell standard specifications etc.
Each shell does the same job but understands different commands and provide
different built in functions.
1. Bourne Shell (sh) – Bourne Shell is the original unix shell developed at
AT&T by Stephen Bourne. Bourne shell also named as (sh) programming
name. It used the symbol $. Bourne shell’s family is bourne, korn shells, bash
and zsh .
2. Korn shell (ksh) – korn shell is the unix shell developed by David korn of
Bell labs. Is is considered as the family member of Bourne shell as it uses the
$ symbol of Bourne shell. It is also names as ksh programmatic ally and it
most widely used shell.
3. Bourne Again Shell (bash) – It is the free version of Bourne shell and comes
with all UNIX/Linux systems as free with some additional features like
command line editing. Its program name is bash. It can read commands from
file called scripts.
4. C Shell (sh) – C shell is the UNIX shell created by Bill joy at California
university as an alternative to Bourne shell – Unix original shell. C shell
along with Bourne and Korn, are there most popular and commonly used shells.
csh is the program name for C shell.
Examples:
$ echo "I am $LOGNAME"
I am lhunath
$ echo 'I am $LOGNAME'
I am $LOGNAME
$ # boo
$ echo An open\ \ \ space
An open space
$ echo "My computer is $(hostname)"
My computer is Lyndir
$ echo boo > file
$ echo $(( 5 + 5 )) 10
$ (( 5 > 0 )) && echo "Five is greater than zero."
Five is greater than zero.
GETTING HELP
Type help after almost any command to bring up a help menu for that command:
help:- The help command provides information on built-in commands.
help pwd:- At the prompt, enter ‘help’ followed by the command we wish to learn
more about.
help –d pwd:- With the ‘-d’ option, the help command will only return a short
description of the specified command.
help –s pwd:- With the -s option, help will return a short usage synopsis for the
specified command.
help –m pwd:- If we prefer the look of manpages, then we’ll be happy to know
that the ‘-m’ option formats the help command output as a pseudo- manpage.
MAN PAGES
Type man before almost any command to bring up a manual for that
command. The man command in Linux is used to display the user manual of any
command that we can run on the terminal. It provides a detailed view of the
command which includes NAME, SYNOPSIS, DESCRIPTION, OPTIONS,
EXIT STATUS, RETURN VALUES, ERRORS, FILES, VERSIONS,
EXAMPLES, and AUTHORS.
Syntax :
$man [OPTION]... [COMMAND NAME]...
Example
$ man printf
$ man 2 intro
$ man -a intro
$ man -k cd
$ man -f ls
In the FHS, all files and directories appear under the root directory /, even if
they are stored on different physical or virtual devices.
Some of these directories only exist on a particular system if certain
subsystems, such as the X Window System, are installed.
Most of these directories exist in all UNIX operating systems and are generally
used in much the same way; however, the descriptions here are those used
specifically for the FHS, and are not considered authoritative for platforms
other than Linux.
Each of the above directories (which is a file, at the first place) contains
important information, required for booting to device drivers, configuration files, etc.
Describing briefly the purpose of each directory, we are starting hierarchically.
/bin : All the executable binary programs (file) required during booting,
repairing, files required to run into single-user-mode, and other
important, basic commands viz., cat, du, df, tar, rpm,
wc, history, etc.
/boot : Holds important files during boot-up process, including Linux
Kernel.
/dev : Contains device files for all the hardware devices on the machine
e.g., cdrom, cpu, etc
/etc : Contains Application’s configuration
files, startup, shutdown, start, stop script for every individual program.
/home : Home directory of the users. Every time a new user is created, a
directory in the name of user is created within home directory which
contains other directories like Desktop, Downloads, Documents, etc.
/lib : The Lib directory contains kernel modules and shared library
images required to boot the system and run commands in root file system.
/lost+found : This Directory is installed during installation of Linux, useful
for recovering files which may be broken due to unexpected shut-
down.
/media : Temporary mount directory is created for removable devices viz.,
media/cdrom.
/mnt : Temporary mount directory for mounting file system.
/opt : Optional is abbreviated as opt. Contains third party application
software. Viz., Java, etc.
/proc : A virtual and pseudo file-system which contains information about
running process with a particular Process-id aka pid.
/root : This is the home directory of root user and should never be
confused with ‘/‘
/run : This directory is the only clean solution for early-runtime- dir
problem.
/sbin : Contains binary executable programs, required by System
Administrator, for Maintenance. Viz., iptables, fdisk, ifconfig, swapon,
reboot, etc.
/srv : Service is abbreviated as ‘srv‘. This directory contains server specific
and service related files.
/sys : Modern Linux distributions include a /sys directory as a virtual
filesystem, which stores and allows modification of the devices connected
to the system.
/tmp :System’s Temporary Directory, Accessible by users and root. Stores
temporary files for user and system, till next boot.
/usr : Contains executable binaries, documentation, source
code, libraries for second level program.
/var : Stands for variable. The contents of this file is expected to grow.
This directory contains log, lock, spool, mail and temp files
Linux Commands
1. pwd
When you log into your server, you are typically dropped into your user account’s home
directory. A home directory is a directory set aside for your user to store files and create
directories. To find out where your home directory is in relationship to the rest of the
filesystem, you can use the pwd command. This command displays the directory that we are
currently in
Syntax: $pwd
You should get back some information that looks like this:
/home/demo
2. cd
To move around the file system use the cd command. The Linux file system is a tree
structure. The top of the tree is denoted by a slash (/). Under the root directory, you will find
all otherfolders.
To move to a particular directory:
Syntax : cd directory_name
To move to a particular folder use the cd command as follows:
Syntax: cd /home/username/Documents
3. ls
It display the contents( file names) in a directory. To use this command move to the
directory for which you wish to apply the ls command.
Syntax: /usr/share$ ls
4. file
file command is used to determine the type of a file. This command tests each argument in
an attempt to categorize [Link] has three sets of tests as follows:
filesystem test: The program verifies that if the file is empty, or if it’s some sort of
special file. This test causes the file type to be printed.
magic test: These tests are used to check for files with data in particular fixed formats.
language test: This test search for particular strings which can appear anywhere in the
first few blocks of a file.
Syntax:
file [option] [filename]
5. cp
To copy a file using Linux, use the cp command as follows:
cp filename1 filename2
The above command will copy the contents in the file filename1 to filename2.
6. mv
The simplest way to rename a file is to use the mv command.
Syntax:
mvoldfilenamenewfilename
Use the mv command to move a file from one folder to another as well.
mv /path/of/original/file /path/of/target/folder
7. mkdir
To create a new folder you can use the mkdir command:
mkdirfoldername
8. rmdir
rmdir command is used remove empty directories from the filesystem in Linux. The rmdir
command removes each and every directory specified in the command line only if these
directories are empty. So if the specified directory has some directories or files in it then this
cannot be removed by rmdir command.
Syntax:
rmdirdirectory_name
9. whereis
whereis command is used to find the location of source/binary file. whereis doesn’t require
any root privilege to execute.
Syntax:
whereis [options] filename
10. More
more command is used to display a file with pausing. cat command will dump the entire
content of a file on the screen whereas more command will display content that would fit
your screen and you can press enter to see rest of the content line by line.
Syntax : $more [Link]
11. Ps
The ps (i.e., process status) command is used to provide information about the currently
running processes, including their process identification numbers
Syntax: $ps
12. w
The command w on many Unix-like operating systems provides a quick summary of every
user logged into a computer, what each user is currently doing.
Syntax: $w
13. id
id command is command which can print real and effective User ID (UID) and
Group ID (GID). An UID is a single identity for a user. While Group ID (GID) can consist
of more than one UID.
Syntax: $id user_name
14. free
you can use the free command to get a detailed report on the system's memory usage.
The free command provides information about the total amount of physical and swap
memory as well as the free and used memory and swap space in the system
15. clear
clear the terminal screen.
16. echo
echo command in linux is used to display line of text/string that are passed as an argument .
17. test
test is used as part of the conditional execution of shell commands. test exits with the status
determined by EXPRESSION. Placing the EXPRESSION between square brackets ([ and ])
is the same as testing the EXPRESSION with test.
18. expr
The expr command in Unix evaluates a given expression and displays its corresponding
output. It is used for:
Basic operations like addition, subtraction, multiplication, division, and modulus on
integers.
Evaluating regular expressions, string operations like substring, length of strings etc.
Syntax:
$expr expression
Example:
a=`expr $a+ 1`
19. Read
read reads a single line from standard input.
Read –p “Enter your name:”
Here we read the data along with some hint text . The hint text helps the user in what he/she
has to enter . -p here stands for the prompt . The hint text also called the prompt text.
Filter Commands
Filters are programs that take plain text(either stored in a file or produced by another
program) as standard input, transforms it into a meaningful format, and then returns it as
standard output. Linux has a number of filters. Some of the most commonly used filters are
explained below:
1. cat : Displays the text of the file line by line.
Syntax:
cat [path]
2. head : Displays the first n lines of the specified text files. If the number of lines is not
specified then by default prints first 10 lines.
Syntax:
head [-number_of_lines_to_print] [path]
3. tail : It works the same way as head, just in reverse order. The only difference in tail is, it
returns the lines from bottom to up.
Syntax:
tail [-number_of_lines_to_print] [path]
4. sort : Sorts the lines alphabetically by default but there are many options available to
modify the sorting mechanism. Be sure to check out the man page to see everything it can
do.
Syntax:
sort [-options] [path]
[Link] : Removes duplicate lines. uniq has a limitation that it can only remove continuous
duplicate lines(although this can be fixed by the use of piping). Assuming we have the
following data.
Syntax:
uniq [options] [path]
6. wc : wc command gives the number of lines, words and characters in the data.
Syntax:
wc [-options] [path]
7. grep : grep is used to search a particular information from a text file.
Syntax:
grep [options] pattern [path]
8. nl : nl is used to number the lines of our text data.
Syntax:
nl [-options] [path]
Here is an example which will list all the files, with details (-la) in the
/dev directory and pipe the output to more. The /dev directory should have dozens
of files and hence ensure that more needs to paginate.
Notice the --More-- prompt at the bottom of the screen. Press SPACE to see the
next page and keep pressing SPACE until the output is finished.
Here is another pipe example, this time using the “wc” (word count) tool.
ls -1 /dev | wc
wc counts the numbers of lines, words and characters in the standard input. If
we use the -l parameter it will display only the number of lines, which is good
way to see how many files are in a directory!
There won’t be anything displayed on the terminal as everything was sent to the
file. We can take a look at the file using the cat command (which can be piped into
more) or for convenience we can just use the more command on its own:
more [Link]
The first redirection will overwrite the file [Link] while the second
will append to it. Now whatever text we type will be sent to the file
[Link] until we press Control-D, at which point the file will be closed and
we will be returned to the command prompt. If we want to add more text to the file
use the same command but with two greater than signs (>>).
INFORMATIONAL COMMANDS
ps:- ps shows process status. Use this command to displays a table of all
our own running programs or processes
w:- Used to show who is logged on and what they are doing
id:- Used to find out user and group names and numeric ID’s (UID or group
ID) of the current user or any other user in the server
free:- Displays the total amount of free space available along with the amount
of memory used and swap memory in the system, and also the buffers used by
the kernel
clear:- Used to clear the terminal screen
echo:- Used to display line of text/string that are passed as an argument
more:- Used to view the text files in the command prompt, displaying one
screen at a time in case the file is large (For example log files)
FILE PERMISSIONS
Every file and directory in our UNIX/Linux system has following 3
permissions defined for all the 3 owners (User, Group, Other).
Read: This permission give we the authority to open and read a file. Read
permission on a directory gives we the ability to lists its content.
Write: The write permission gives we the authority to modify the contents of
a file. The write permission on a directory gives we the authority to add,
remove and rename files stored in the directory. Consider a scenario where we
have to write permission on file but do not have write permission on the
directory where the file is stored. We will be able to modify the file contents.
But we will not be able to rename, move or remove the file from the
directory.
Execute: In Windows, an executable program usually has an extension
".exe" and which we can easily run. In Unix/Linux, we cannot run a
program unless the execute permission is set. If the execute permission is
not set, we might still be able to see/modify the program code (provided
read & write permissions are set), but not run it.
Setting Permissions
chmod +rwx filename to add permissions.
chmod -rwx directoryname to remove permissions.
chmod +x filename to allow executable permissions.
chmod -wx filename to take out write and executable permissions.
COMMENTS
When we are writing shell programs as a Systems Administrator comments
are doubly important. Chances are won't be at the site forever. At some stage
another Systems Administrator is going to come along and have to decipher what
our shell programs are doing.
VARIABLES
A variable is declared without a $, but has a $ when invoked. Let's edit
our hello-world example to use a variable for the entity being greeted, which is
World.
#!/bin/bash who="World"
echo Hello, $who!
OPERATORS
1. Arithmetic Operators
Example
#!/bin/sh
a=5
b=10
val=`expr $a + $b`
val=`expr $a - $b`
echo "a - b : $val"
val=`expr $a \* $b`
echo "a * b : $val"
Relational operators
Relational operators are those operators which defines the relation between
two operands. They give either true or false depending upon the relation. They are
of 6 types:
‘==’ Operator : Double equal to operator compares the two operands. Its returns
true is they are equal otherwise returns false.
‘!=’ Operator : Not Equal to operator return true if the two operands are not equal
otherwise it returns false.
‘<' Operator : Less than operator returns true if first operand is lees than second
operand otherwse returns false.
‘<=' Operator : Less than or equal to operator returns true if first operand is less
than or equal to second operand otherwise returns false
‘>’ Operator : Greater than operator return true if the first operand is greater
than the second operand otherwise return false.
‘>=’ Operator : Greater than or equal to operator returns true if first operand is
greater than or equal to second operand otherwise returns false
Example
#!/bin/bash
#reading data from the userread -p
'Enter a : ' a
read -p 'Enter b : ' b
if(( $a!=$b ))
then
echo a is not equal to b.
else
echo a is equal to b.
fi
if(( $a<=$b ))
then
echo a is less than or equal to b.
else
echo a is not less than or equal to b.
fi
LOGICAL OPERATORS
They are also known as boolean operators. These are used to perform logical
operations. They are of 3 types:
1. Logical AND (&&) : This is a binary operator, which returns true if both
the operands are true otherwise returns false.
2. Logical OR (||) : This is a binary operator, which returns true is
either of the operand is true or both the operands are true and returns false
if none of then is false.
3. Not Equal to (!) : This is a uninary operator which returns true if the operand
is false and returns false if the operand is true.
Example
#!/bin/bash
#reading data from the userread -p
'Enter a : ' a
read -p 'Enter b : ' b
$ echo 'Hostname=$HOSTNAME’ ;
Current User=`whoami` ;
Message=\$ is USD' Hostname=$HOSTNAME’ ;
Current User=`whoami` ;
Message=\$ is USD
DOUBLE QUOTES
Use double quotes when we want to display the real meaning of special
variables.
$ echo "Hostname=$HOSTNAME” ;
Current User=`whoami` ;
Message=\$ is USD" Hostname=dev-db” ;
Current User=ramesh ;
Message=$ is USD
Double quotes will remove the special meaning of all characters except the
following:
$ Parameter Substitution.
` Backquotes
\$ Literal Dollar Sign.
\´ Literal Backquote.
\” Embedded Doublequote.
\\ Embedded Backslashes.
CONDITIONAL COMMANDS
There are total 5 conditional statements which can be used in bash programming
1. if statement
2. if-else statement
3. if..elif..else..fi statement (Else If ladder)
4. if..then..else..if..then..fi..fi..(Nested if)
5. switch statement
Their description with syntax is as follows
If statement
This block will process if specified condition is true.
Syntax:
if [ expression ]
then
statement
fi
if-else statement
If specified condition is not true in if part then else part will be execute.
Syntax
if [ expression ]
then
statement1
else
statement2
fi
if..then..else..if..then..fi..fi..(Nested if)
Nested if-else block can be used when, one condition is satisfies
then it again checks another condition. In the syntax, if expression1 is false
then it processes else part, and again expression2 will be check.
Syntax:
if [ expression1 ]
then
statement1
statement2
.
else
if [ expression2 ]
then
statement3
.
fi
fi
Switch Statement
The case statement works as a switch statement if specified value
match with the pattern then it will execute a block of that particular pattern
When a match is found all of the associated statements until the double
semicolon (;;) is executed. A case will be terminated when the last command
is executed. If there is no match, the exit status of the case is zero.
Syntax:
Case $var in
Pattern 1) Statement 1;;
Pattern n) Statement n;;
esac
Example 1 Example 2
#Initializing two variables #Initializing two variables
a=10 a=20
b=20 b=20
Example
CARS="bmw"
#Pass the variable in string
case "$CARS" in
#case 1
"mercedes") echo"Headquarters-Affalterbach,Germany" ;;
#case 2
"audi") echo "Headquarters - Ingolstadt, Germany" ;;
#case 3
"bmw") echo"Headquarters - Chennai, Tamil Nadu,India" ;;
esac
ITERATIVE COMMANDS
There are total 2 looping statements which can be used in
bash programming
1. while statement
2. for statement
While Statement
Here command is evaluated and based on the result loop will
executed, if command raise to false then loop will be terminated
Syntax
while
command
do
Statement to be executed
done
For Statement
The for loop operate on lists of items. It repeats a set of commands for
every item in a list. Here var is the name of a variable and word1 to wordN
are sequences of characters separated by spaces (words). Each time the for
loop executes, the value of the variable var is set to the next word in the list
of words, word1 to wordN.
Syntax
for var in word1 word2 ...wordn
do
Statement to be executed
done
Example 1 Example 1
#Start of for loop for a in 1 2 3 4 5 6 7 8 9 10
for a in 1 2 3 4 5 6 7 8 9 10 do
do # if a = 5 then continue the loop and
# if a is equal to 5 break the loop # don't move to line 8
if [ $a == 5 ] if [ $a == 5 ]
then then
break continue
fi fi
# Print the value echo "Iteration no $a"
echo "Iteration no $a" done
done
Example
a=0
# -lt is less than operator
Example
#!/bin/bash
#breaking a loop
num=1
while [ $num –lt 10 ]
do
if [ $num –eq 5 ]
then
break
fi
done
echo “Loop is complete”
Example
#!/bin/bash
# using continue command
for i in 1 2 3 4 5 6 7 8 9
do
if [ $i –eq 5 ]
then
echo “skipping number 5”
continue
fi
echo “I is equal to $i”done
EVALUATING EXPRESSIONS
The expr is a command line Unix utility which evaluates an
expression and outputs the corresponding value. expr evaluates integer or
string expressions, including pattern matching regular expressions. Most of
the challenge posed in writing expressions is preventing the invoking
command line shell from acting on characters intended for expr to process.
The operators available for integers: addition, subtraction, multiplication,
division and modulus for strings: find regular expression, find a set of
characters in a string; in some versions: find substring, length of string for
either: comparison (equal, not equal, less than, etc.)
Options:
bc [ -hlwsqv ] [long-options] [ file ... ]
h, {- -help } : Print the usage and exit
-i, {- -interactive } : Force interactive mode
-l, {- -mathlib } : Define the standard math library
-w, {- -warn } : Give warnings for extensions to POSIX bc
-s, {- -standard } : Process exactly the POSIX bc language
-q, {- -quiet } : Do not print the normal GNU bc welcome
-v, {- -version } : Print the version number and copyright and quit
Examples
BC
bc is an acronym for “basic calculator.” bc command is used for command line
calculator. It is similar to basic calculator by using which we can do basic
mathematical calculations.
Arithmetic operations are the most basic in any kind of programming language.
Linux or Unix operating system provides the bc command and expr command for doing
arithmetic calculations. We can use these commands in bash or shell script also for
evaluating arithmetic expressions.
Example:
Input : $ echo "12+5" | bc
Output : 17
STRINGS
Finding length of a Finding substring of a Matching number of
string string characters in two strings
x=nirmala
x=ncas sub=`expr substr $x 2 3` $ expr nirmala : nirma
len=`expr length $x` echo $sub
echo $len
GREP
The grep filter searches a file for a particular pattern of characters,
and displays all lines that contain that pattern. The pattern that is searched in
the file is referred to as the regular expression (grep stands for globally search
for regular expression and print out).
Syntax:
grep [options] pattern [files]
Options Description
-c : This prints only a count of the lines that match a pattern
-h : Display the matched lines, but do not display the filenames.
-i : Ignores, case for matching
-l : Displays list of a filenames only.
-n : Display the matched lines and their line numbers.
-v : This prints out all the lines that do not matches the pattern
-e exp : Specifies expression with this option. Can use multiple times.
-f file : Takes patterns from file, one per line.
-E : Treats pattern as an extended regular expression (ERE)
-w : Match whole word
-o : Print only the matched parts of a matching
line, with each such part on a separate output
line.
Example
Display the file names that matches the pattern : We can just display
the files that contains the given string/pattern.
$grep -l "unix" *
Or $grep -l "unix" [Link] [Link] [Link] [Link]
Checking for the whole words in a file : By default, grep matches the
given string/pattern even if it found as a substring in a file. The -w
option to grep makes it match only the whole words.
$ grep -w "unix" [Link]
ARRAYS
An array is a data structure consist multiple elements based on key
pair basis. Each array element is accessible via a key index number. An
Array is a data structure that stores a list (collection) of objects (elements)
that are accessible using zero-based index. In Linux shells, arrays are not
bound to a specific data type; there is no array of data type integer, and
array of data type float. An array can contain an integer value in one
element, and a string value in the element next to it.
Defining Array Values
The difference between an array variable and a scalar variable can
be explained as follows. Suppose we are trying to represent the names of
various students as a set of variables. Each of the individual variables is a
scalar. Here array_name is the name of the array, index is the index of the
item in the array that we want to set, and value is the value we want to set for
that item.
NAME[0]="Zara"
NAME[1]="Qadir"
NAME[2]="Mahnaz"
NAME[3]="Ayan"
NAME[4]="Daisy"
If we are using the bash shell, here is the syntax of array initialization –
Example
# !/bin/bash # !/bin/bash
# To declare static Array # To declare static Array
arr=(1 12 31 4 5) arr=(1 2 3 4 5)
i=0
# Loop upto size of array # loops iterate through a
# starting from index, # set of values until the
i=0 # list (arr) is exhausted
while [ $i -lt ${#arr[@]} ] for i in "${arr[@]}"
do do
# To print index, ith # access each element
# element # as $i
echo ${arr[$i]} echo $i
done
# Increment the i = i + 1
i=`expr $i + 1`
done
***************************