Introduction to Unix & Linux

Orange & Bronze Software Labs

Company Overview

Mission
Mission
Adopt
Adopt best
best practices
practices and
and share
share with
with
others.
others.

2/224
 Services

Software Development

Training

Advisory Services

3/224

Clients in 11 Countries

4/224
 Agenda
Overview
Overview
General
General utility
utility commands
commands
Files
Files and
and directories
directories
Handling
Handling ordinary
ordinary files
files and
and filters
filters
File
File attributes
attributes and
and permissions
permissions
User
User and
and Group
Group management
management
Text Editor: Vi/Vim
Bash
Bash Scripting
Scripting
Package Management
Managing Processes
Basic Networking

UNIX History
Overview

Developed primarily by Ken

Thompson & Dennis Ritchie at Bell
Labs, starting 1969.
· Intended to replace MIT's
MULTICS.
- MULTICS - Multiplexed
Information and Computer Services
- UNICS - Uniplexed Information and Computing Service, pronounced as
"eunuchs"
· Initially written in assembly, rewritten in 1972 in C, thus becoming the
first portable operating system.

6/224
UNIX Adoption
Overview

· Proliferation of "Unix-like" Systems: MacOS / OS X, BSD, AIX, HP-UX,

Solaris, Linux, etc.
· ARPANET - precursor of the internet
· Y2K bug - accelerated migration from mainframe to UNIX-like systems
· Unix-like systems are now the preferred OS for core database and
business application infrastructure, due to stability & performance
· 67% of web servers & >90% of smart phones run on Unix-like systems
(2015)

7/224

UNIX Characteristics
Overview

· Multitasking
· Multiuser
· Modular
· Scriptable
- UNIX programs
are meant to
work with one
another through
scripts
· Portable
- UNIX is the first portable operating system

8/224
The UNIX Philosophy
Overview

· Write programs that do one thing and do it well.

· Write programs to work together.
· Write programs to handle text streams, because that is a universal
interface.

9/224

GNU - GNU's Not Unix!

Overview

The GNU Project was founded by

Richard Stallman in 1983 while
working at MIT.
It's goal was to provide free,
community-developed software,
including an operating system, so
that people can maximize the use
of their hardware.
The only part they were unable to develop was the kernel.

10/224
Linux
Overview

Developed by Linus Torvalds starting in

1991, as a hobby project while a student
at the University of Helsinki.
"Hello everybody out there using Minix –
I’m doing a (free) operating system (just a
hobby, won’t be big and professional like
GNU)..."
He ended up building just a kernel, distributed with software from the
GNU Project.
*"GNU/Linux" is the more accurate name for Linux-based operating
systems.

11/224

Shell Usage & Navigation

The Bash Shell
Bash is the most widely-used Unix command line interpreter ("shell").
Open a Bash terminal
now. On most Linux
desktops, you do this by
launching an application
called "Terminal", which
you can find either in
Menu, or by pressing Alt-
F2 and searching for
"Terminal".

13/224

Display Text - echo

$ echo Hello World

14/224
Commands to navigate your way
· pwd
- (your) present working directory
· ls
- list directory (like the MS-DOS DIR)
· cd
- change directory

15/224

Present Working Directory - pwd

Most Bash shells would be configured to show the current directory,
username, and other information at the prompt, but not always. To
check your current directory, use pwd:
$ pwd

This should say that you are currently in your "home directory".

16/224
List Directory Contents - ls
Contents of present working directory (by default, in alphabetical order):
$ ls

Contents of pwd in reverse alphabetical order:

$ ls -r

Include hidden files:

$ ls -a

17/224

List Directory Contents - ls

Metadata of contents:
$ ls -l

Show metadata with file size in human-readable form:

$ ls -lh

Include metadata & include hidden files:

$ ls -al

What does this do?

$ ls -alrh

18/224
List Directory Contents - ls
List contents of a specified directory by including the directory path:
$ ls /

$ ls /usr

$ ls /usr/lib

Contents of a specified directory with metadata:

$ ls -l /

$ ls -l /usr

19/224

List Directory Contents - ls

Sorting

What is the behavior of the statements below?

$ ls --sort=size /var/log

$ ls -l --sort=size /var/log

$ ls -S /var/log

$ ls -lS /var/log

$ ls -lSrh /var/log

20/224
List Directory Contents - ls
Sorting

What is the behavior of the statements below?

$ ls --sort=time -l /var/log

$ ls -tl /var/log

$ ls -tlr /var/log

21/224

List Directory Contents - ls

Sorting

What is the behavior of the statements below?

$ ls --sort=extension /var/log

$ ls -X /var/log

22/224
Change Directory - cd
Use cd to change your working directory:
$ cd D​
ocuments
$ pwd

$ cd ..
$ pwd

$ cd /usr
$ pwd

$ cd
$ pwd

23/224

Common Unix/Linux Directory Structure

24/224
Paths
Absolute Path - Location of a file or directory starting from the root
directory. Starts with a /.
· /home/training/Documents
· /usr
· /usr/lib

Relative Path - Without the slash, denotes the location of a file or

directory relative to the present working directory.
· Documents
· ../../usr
· ../../usr/lib

25/224

Paths
Exercise:
Use ls to list the contents of the directories in the previous slides, using
absolute paths first, then using relative paths.

26/224
Paths
You can also represent the current working directory using '.':
$ ls .

$ ls ./D​
ocuments

27/224

Wildcards
Partial string matches.

$ echo *

$ echo D*s

$ echo D*p

$ echo D​
o*

$ echo D​
o*

$ echo d​
o* # Unix is case sensitive

$ echo *es

28/224
Wildcards
Partial string matches.

Application:
$ ls * # List contents of all directories in pwd

$ echo /var/log/*.log # List files with .log extension in /var/log

29/224

Wildcards
Partial string matches.

$ echo D*
$ echo D??????
$ echo D???????

$ echo *c
$ echo ????c
$ echo ?????c

What is the difference between '*' and '?'?

30/224
Wildcards
Partial string matches.

$ echo *s
$ echo *[e]s
$ echo *[eo]s
$ echo *[eod]s

What does '[]' do?

$ echo *[E]s
$ echo *[E​
e]s
$ echo *[eOd]s
$ echo [L-Q]*

What is being demonstrated by the examples above?

31/224

Wildcards
Partial string matches.

$ echo *[!e]s
$ echo *[!eo]s
$ echo *[!eod]s

What does the '!' do?

32/224
Alias for Home Directory - ~
The ~ is an alias for the home directory of the logged-in user:
$ ls ~

$ cd ~/D​
ocuments
$ pwd

$ cd ~
$ pwd

$ cd ~/..
$ pwd

33/224

Multiple Statements in One Line - ;

Some statements take a while to execute. If you need to execute several
statements, you may not want to wait for each one to finish before
typing in the next one.
In order to execute several statements in one go, use the semicolon - ; -
to separate each statement.
$ cd ~/D​
ocuments; pwd # Change directory to ~/Documents, then output working directory.

If an error happens in any of the statements, the others will still execute.
$ cd ~/D​
oc; pwd # ~/Doc does not exist

34/224
Tab Completion
Present the tab button can complete a partially typed-in command,
directory name or filename.
Try these partial commands and hit tab:
$ ls ~/D​
oc

$ ls ~/D​
esk

If there is more than one possibility, tab will present to you a list of
options. Hit the tab button after typing the partial command below:
$ pw

35/224

Command History
Press the up arrow key (↑). You will see the previous command you
typed.
Keep pressing to see the next older command. Use the down arrow key
(↓) to see newer commands.
Press enter to re-run the command, or you may edit the command first
before running it.
Use the history command to see a listing of previous commands:
$ history

36/224
User Manual - man
Use man to read the documentation of a command:
$ man pwd

Use the arrow keys or PgUp/PgDown to navigate, and 'q' to quit.

$ man ls

To search, start with a '/', and then your search string. For example, to
search for the string "file":
/file

Press 'N' to move forward to the next result, and 'Shift-N' to move
backward to a previous result.

37/224

In Unix & Linux forums, people asking lazy questions

often get the response, "RTFM".

38/224
Clearing the Screen - clear
$ clear

39/224

Files
"Everything is a File"
Unix handles all input/output resources the same way as files.
· Directories are files describing other files.
· A keyboard is a file being read by the OS.
· A monitor is a file being written by the OS.
· A network card is a file being written and read by the OS.
This allows a consistent set of tools and techniques for handling a wide
set of resources.

41/224

File Extensions Ignored

Some operating systems use the file extenstion (.txt, .pdf, .exe...) to
determine how a file is handled. Unix ignores the extension and
examines the content of the file.
Some specific Unix programs do choose to use file extensions to
determine how to handle certain files.

42/224
Case Sensitive
Some operating systems are case insensitive, so "[Link]", "[Link]" or
"[Link]" will all refer to the same file.
Unix is case sensitive, so for example you try to open "[Link]" but
instead you type in "[Link]", you will get the error:
[Link]: ERROR: cannot open '[Link]' (No such file or
directory)

43/224

Spaces in Names
File & directory names can have spaces, but you'll need to handle them
differently. For example, if you have a directory called "Wedding
Pictures" you can:
Enclose in quotes:
· "Wedding Pictures"
· 'Wedding Pictures'

Use backslash (\) to escape the space:

· Wedding\ Pictures

44/224
Hidden Files & Directories
Run ls in your home directory, then run it again with the "all" option -
ls -a. Notice that with the "all" option, you see more files and
directories, this time files and directories that start with a period (.).
Any file with a period prefix is automatically hidden from directory
listings.
Sensitive files and directories are usually hidden to prevent accidental
modification or deletion. Ex: configuration files, shared libraries, etc.

45/224

Basic File Handling Commands

Determine File Type - file
$ file D​
ocuments

$ file .profile

$ file /bin/ls

47/224

Detailed File Information - stat

size, permissions, last accessed, last modified, etc.
$ stat D​
ocuments

$ stat .profile

$ stat /bin/ls

48/224
Detailed File Information - stat
If you just want specific info, add the -c option and supply a valid
parameter (see man page):
Access rights: $ stat -c %A .profile

Size in bytes: $ stat -c %s .profile

Owner: $ stat -c %U .profile

Last access: $ stat -c %x .profile

Last change in content ("Modify"): $ stat -c %y .profile

Last change in content or meta data ("Change"): $ stat -c %z .profile

49/224

File Manipulation Commands

· cp
- copy a file/directory
· mv
- move (or rename) a file/directory
· mkdir
- make a directory
· rm
- delete a file/directory
· ln
- symbolic linking (soft or hard)

50/224
Copy - cp
To copy an entire directory, use the -r option:
$ cp -r /var/log .

Some files will not be copied because of permissions.

Examine the new directory created:
$ ls
$ stat log
$ ls -l log

51/224

Copy - cp
Copy [Link] to your current directory:
$ cp /var/log/[Link] .
$ ls

Compare the information of the original and copy:

$ stat /var/log/[Link]
$ stat [Link]

What are the differences?

52/224
Create Directory - mkdir
$ mkdir NewDirectory
$ ls

53/224

Move or Rename - mv
Moving and renaming files & directories use the same command - mv.
$ mv [Link] NewDirectory
$ ls # look for [Link]
$ ls NewDirectory

$ mv NewDirectory/[Link] NewDirectory/[Link]
$ ls NewDirectory

$ mv NewDirectory LogDir
$ ls
$ ls New Directory
$ ls LogDir

54/224
Deleting - rm
Deleting a file:
$ rm NewFile
$ stat NewFile
$ ls

Deleting directories:
$ rm LogDir

What happened? Why?

55/224

Deleting - rm
Correctly deleting directories:
$ rm -r LogDir
$ stat LogDir
$ ls

Note that if there are no errors in your statement, rm will delete your
files and directories without prompting. Be sure you are deleting the right
files and directories.

56/224
Links to Files & Directories - ln
Linking allows you to have more than one
path to a file or directory.
Sample Applications:
· Creating shortcuts.
· Organizing files in more than one
directory.
· Serving files in an application server
(e.g. web server) while at the same time
maintaining them in the user's within
the user's home directory.
· Access of a file between otherwise
incompatible filesystems.

57/224

Links to Files & Directories - ln

By default, ln creates a "hard link" - just another reference to the data.

$ ln /var/log/[Link] HardLinkToIoDeviceLog
$ ls -l #check that link was created

58/224
Links to Files & Directories - ln -s

To make a "soft link"

$ ln -s /var/log/[Link] SoftLinkToIoDeviceLog
$ ls -l #check that link was created

59/224

Hard vs. Soft Links

1. When you move the source/origin file...
· hard links retain its relationship
· soft links are orphaned

2. Hard links consume the same file size that of source file, soft links are
way smaller
3. Soft links are more visible/obvious that it is a symlink

60/224
Reading Text Files
· cat
- Output contents to console.
· less
- Scroll through file.
· head
- Output first lines to console.
· tail
- Output last lines to console.

61/224

Reading Text Files

cat:

$ cat .profile

$ cat
the quick brown fox jumped over the lazy dog

Do a Ctrl+D to escape or end the command

62/224
Reading Text Files
less:

$ less [Link]

· Use the arrow keys and (↑↓) 'PgUp'/'PgDown' to navigate.

· Use "/" to search - /keyboard - then use 'N' & 'Shift-N' to move
forward or backward to each result.
· Use 'q' to quit.

63/224

Reading Text Files

head:

First 10 lines (default): $ head /var/log/[Link]

First 20 lines: $ head -20 /var/log/[Link]

First 5 lines: $ head -5 /var/log/[Link]

tail:

Last 10 lines (default): $ tail /var/log/[Link]

Last 20 lines: $ tail -20 /var/log/[Link]

Last 5 lines: $ tail -5 /var/log/[Link]

64/224
Word Count - wc
$ wc [Link]

· 1st number: number of lines

· 2nd number: number of words
· 3rd number: number of characters

65/224

Create or Change File - touch

'touch' creates a new empty file:
$ touch NewFile
$ ls # Look for 'NewFile'
$ file NewFile
$ stat NewFile

On existing files, 'touch' updates Access/Modify/Change timestamps:

$ touch NewFile
$ stat NewFile

66/224
Check Differences - diff
Returns differences in content (not metadata).
$ diff /var/log/[Link] [Link]

There should be no difference if no change in your peripherals. Try

unplugging the plugging back your mouse, keyboard or monitor, then
run the command again.
Use the -r option to compare directories:
$ diff -r /var/log log

67/224

Standard I/O
With the notion that "everything is a file" in Unix, Command Outputs
are also files
Types of outputs
1. standard input (stdin); your keyboard inputs
2. standard output (stdout)
3. standard error (stderr)

68/224
Redirect stdout
To redirect Standard Outputs
$ ls -l /usr/bin > [Link]

Append output to an existing file

$ ls -l /usr/local >> [Link]

69/224

Redirect stderr
To redirect Standard Error
$ ls -l /bin/usr 2> [Link]

To redirect both stdout and stderr

$ ls -l /usr/bin > [Link] 2>&1

70/224
Redirect stdin
$ cat > lazy_dog.txt
the quick brown fox jumped over the lazy dog

How about trying...

$ cat < lazy_dog.txt

· The character < is used to redirect stdin

71/224

Pipelines - pipe operator |

The pipe operator | (vertical bar) is used to pipe the standard output of
a command to the standard input of the next command
$ command1 | command2

Example
$ ls -l /usr/bin | less

72/224
Silence is Golden - /dev/null
When you want to discard, mute, or do nothing on the output
$ ls -l /usr/bin > /dev/null
$ ls -l /bin/usr 2> /dev/null

73/224

Searching for Files & Directories - find

Commonly-used options:
-o : the OR operation to string multiple search criteria together
-name : find files by name
-iname : find files by name, ignoring case
-type f : find only files
-type d : find only directories
-size : find by size
-mtime / -mmin : find by modified time

74/224
Searching for Files & Directories - find
What are each of the statements below searching for?
$ find -name [Link]

$ find -name [Link]

$ find -iname [Link]

$ find -iname [Link] -type d

$ find -iname [Link] -type f

75/224

Searching for Files & Directories - find

By default, find will search just within the present working directory. To
specify different directories, place the path/s after the find command:
$ find log -name [Link]

$ find D​
ocuments -name [Link]

$ find / -name [Link] # Search whole filesystem

$ find /var/log -name [Link] # Search /var/log

$ find /var/log ~ -name [Link] # Search /var/log and home directory

76/224
Searching for Files & Directories - find
What are each of the statements below searching for?
$ find -size +1k

$ find -size -1k

Units for size:

b or no suffix : blocks (512-bytes)
c : bytes
w : two-byte words
k : kilobytes
M : megabytes
G : gigabytes

77/224

Searching for Files & Directories - find

Find by time in days:
$ find -mtime -1 # files modified less than 1 day ago

$ find -mtime +1 # files modified more than 1 day ago

$ find -mtime 1 # files modified exactly 1 day ago

-mtime : modification time in days

-atime : access time in days
-ctime : change time in days

78/224
Searching for Files & Directories - find
Find by time in minutes:
$ find -mmin -60 # files modified less than 1 hour ago

$ find -mmin +60 # files modified more than 1 hour ago

$ find -mmin 60 # files modified exactly 1 hour ago

-mmin : modification time in minutes

-amin : access time in minutes
-cmin : change time in minutes

79/224

Searching for Files & Directories - find

Find by time range:
$ find -mtime +1 -mtime -20 # files modified more than 1 day but less that 20 days

$ find -mmin +10 -mmin -180 # files modified more than 10 min but less than 3 hours

80/224
Searching for Files & Directories - find
Wildcards

When using wildcards, place the expression in quotes (single or double),

otherwise the wildcard will evaluate before the find command, and if
there is a match in the working directory, the argument will be the
file/directory name that matched, not the original wildcard expression:
$ find -name *log

$ find -name "*log"

What's the difference in the output of the two statements above? Why?

81/224

Searching for Files & Directories - find

More Wildcard Examples

Describe the behavior of each statement below:

$ find -name "[Xx]org.*"

$ find -name "[a-e]*.gz"

$ find -name "*[0-9]*.log"

$ find -name "[Dd]??????"

82/224
Searching for Files & Directories - find
Combining Multiple Criteria

By default, multiple search criteria will be interpreted as logical "AND":

$ find -name "*.log" -type d

To combine criteria via logical "OR", use the -o or -or option:

$ find -name "*.log" -or -type d

AND will take precedence over OR, unless you use parenthesis:
$ find -iname "dpkg*" -o -name "*log" -type d

$ find \( -iname "dpkg*" -o -name "*log" \) -type d # note space before & after parentheses

83/224

Searching for Files & Directories - find

Negation - !

Describe the behavior of the statements below:

$ find ! -name "*log"

$ find log -name "*log" ! -name "*[0-9]*"

$ find log ! -name "*log" -name "*[0-9]*"

$ find log -name "*log" ! -name "*[0-9]*" -name "Xorg*"

84/224
Regular Expression Parser - grep
syntax:
grep [options] regex [file...]

ex:
$ ls /usr/bin | grep zip

Note: the "|" symbol is to "pipe" the output of a command to the next one

85/224

File attributes and permissions

Basic User Commands
· id
· su
· sudo
· passwd

87/224

User Identity - id
id

$ id
uid=1000(armand) gid=1000(armand) groups=1000(armand),4(adm),24(cdrom),27(sudo)

· When user accounts are created, the user is assigned a user ID (uid)
which is mapped to a username
· The user is also assigned its own group ID (gid). This is different from
the uid.

88/224
User Identity files
· /etc/passwd
- defines the username, uid, gid
· /etc/group
- defines the groups and members for each
· /etc/shadow
- pseudo-copy of /etc/passwd but has other pertinent details

89/224

Substitute User - su
su - substitute user

$ su
Password:
#

· used to start a shell as another user

· by default, this assumes that you want to log-in as the superuser
· the -l [user] option lets you specify the desired user

90/224
Execute a command as another user (or root)
1. Execute as root using su -c

$ su -c 'ls -l /root'

· will ask for the password of the root user

2. Execute as "dummy" user using su - [user] -c

$ su - dummy -c 'ls -l /path/to/folder'

· will ask for the password of the substitute user

91/224

Execute a command with root privileges

sudo

$ sudo ls -l /root

· Using sudo uses root privileges for the provided command

· Will ask for your password
· Active user must be a member of the sudo group

92/224
Changing password - passwd
Syntax
passwd [user]

· by default, it will let you change the active session user

· it will try to enforce the use of "strong" passwords

93/224

File Permission 101

94/224
File Permission 101
Attribute File Type

- Regular File

d Directory

l Symbolic Link

c Character Special File

b Block Special File

95/224

File Permission by-the-numbers

96/224
File Permission by-the-numbers

97/224

File Permission Management Commands

· chmod
· chown
· chgrp
· umask

98/224
Changing File Permissions
chmod - change file mod bits

Syntax
chmod [options] [mode] [file]

Examples
$ chmod 600 [Link] # change a file to read and write by user only
$ chmod +x [Link] # adds executable permission to all
$ chmod -R -w /home/user/some_folder/ # removes write permissions recursively in location

99/224

Changing File Permissions

Symbol Target

u user

g group

o others

a (or blank) all

$ chmod u+x [Link] # change a file be executable by user

$ chmod g-rw [Link] # removes read and execute to the group
$ chmod u+x,o-xw some_file # adds executable to user and
# removes execute and write permissions for others

100/224
Changing File User/Group Owners
chown - change user/group ownership of a file/directory

Syntax
chown [options] [owner]:[group] [file]

Example
$ chown armand:armand [Link] # changes a file to be owned
# by user "armand" and group "armand"

$ chown -R www-data:armand newfolder/ # changes a file recursively to be owned

# by user "www-data" and group "armand"

101/224

Changing File Group Owners

chgrp - change group ownership of a file/directory

Syntax
$ chgrp [options] [group] [file]

Example
$ chgrp armand [Link] # changes a file to be owned by group "armand"

102/224
Setting Default Permissions - umask
umask - uses octal notation to represent a mask of bits

$ umask
0000

103/224

Setting Default Permissions - umask

Note: The typical umask is 0002

104/224
 User and Group Management

User Management Commands

· useradd and adduser
· groupdd and addgroup
· deluser and delgroup
· usermod

106/224
Creating a User
useradd

· Low-level command to create a user account

· Automatically creates a corresponding group
· Will need to do a passwd [username] after using this
# useradd [username]

107/224

Creating a User
adduser

· Higher-level command in user account creation (only in Debian)

· Recommended to use than useradd
· Will ask other pertinent information via guided prompt
# adduser [username]

108/224
Creating a Group
groupadd and addgroup

· Low and high-level command respectively to create a new group

# groupadd [groupname]

# addgroup [groupname]

109/224

Deleting a User/Group
deluser - deletes a user

# deluser [username]

delgroup - deletes a unix group

# delgroup [groupname]

Notes:
· Be mindful of files dependent on the permissions of the user/group to
be deleted
· Some user/group are being used by the system or other applications

110/224
Modifying an existing User
usermod - change details for a user

Examples
# usermod -aG sudo user # adds user to the sudo group
# usermod -md /home/newuserhome user # changes the home directory location of user

Caution:
· not recommended to modify a user's password, the passwd
command is there for you

111/224

Day 1 Exercise
Day 1 Exercise
1. from the command lspci, find out the following
· anything about "Ethernet"
· anything about "VGA"

2. save whatever relevant output you got for each category in their own
respective file
· name the file for each output however you please
· save it in a "lspci-output" folder inside your home directory

3. make the "lspci-output" folder and its contents be read-only to

"dummy1" user
· you may need to create a "dummy1" user account
· anyone else (others) should not have rights to access it
· at your discretion in what kind of permission do you wish to implement it

113/224

Vim - the Text Editor

Text Editors
Numerous text editors exist in Unix, both graphical and command line.
Example Graphical Editors:
· Simple: GEdit
· Word Processor: LibreOffice Writer
Example Commandline Editors: Pico, Nano, Emacs, Vi/Vim
Vi/Vim is the most popular commandline editor, and will be available on
most systems by default.

115/224

Vi/Vim
Vi
· Written by Bill Joy in 1976.
· Vi is short for "visual".

Vim
· Written by Bram Moolenaar in 1991.
· "Vi Improved"

116/224
Vi/Vim
Launching & Quitting

Start a Vim session:

$ vim NewFile # If you save, vim will write to 'NewFile'.

Quit:
:q

Check if a new file was created:

$ ls

117/224

Vi/Vim
Writing to Buffer

Press the 'i' key. This will start the "insert" mode, which allows you to
write to the Vi/Vim buffer (memory).
Write something:
Hello Vim!

Press 'Esc' to leave insert mode.

118/224
Vi/Vim
Writing to Buffer

Try to quit:
:q

Force quit:
:q!

Check if a new file was created:

$ ls

119/224

Vi/Vim
Save & Exit

Save and exit:

:x

Check what happened:

$ cat NewFile

120/224
Vi/Vim
Edit & Save

Launch a 2nd terminal so that we can check on the file while we edit it.
Launch Vim in one of the terminals:
$ vim NewFile

Edit the line of text:

· Move the cursor where you want to start editing:
- On Vim, you can use the arrow keys (← →) to navigate.
- On Vi, you press 'l' to move right and 'h' to move left.
· Press 'i', then start modifying the text.
Hello Vim! Where have you been all my life?

121/224

Vi/Vim
Edit & Save

In the other terminal, check if NewFile was modified:

$ cat NewFile

Hit 'Esc' to leave insert mode.

Call the write command:
:w

Now go to the other terminal and check what happened to the file:
$ cat NewFile

122/224
Vi/Vim Exercise
Edit a file using Vi/Vim
· download the file by doing a wget [Link]
- rename the file (may look like 2ptwfJP) to [yournickname].txt
- provide your answers beside the respective field
- replace blank fields with respective answer by inserting it
- no using of gedit

123/224

BASH Scripting
Main Steps in Making Scripts
1. Write a Script
2. Make the Script Executable
3. Put the Script somewhere the shell can find it

125/224

Write a Script
Start by creating the file
$ vim hello_world

Note: Make sure you know which directory you are before doing this.

126/224
Write a Script
Sample Script file format
#!/bin/bash
# This is our first script
# The script should always start with
# #!/bin/bash or intended shell
echo 'Hello World'

127/224

Comments - #
Anything after a '#' is a comment. This is useful when scripting
commands together.
$ # This is a comment. It does nothing.

$ ls # Adding a comment after a command does not affect it.

128/224
Make the Script Executable
$ ls -l
$ chmod +x hello_world

When you do another ls -l

$ -rwxrwxrwx 1 user user 0 Apr 17 15:00 hello_world

129/224

Execute the Script

$ ./hello_world

OR
$ /path/to/the/file/hello_world

130/224
Adding a little Data
#!/bin/bash
title='Hello'
echo $title World
echo $title Armand Shifu

131/224

Rules in declaring variables

1. Variable names may consist of ...
· alphanumeric characters
· underscore character

2. The first character of a variable name must either be ...

· a letter, or
· an underscore

3. Spaces and punctuation are not allowed

· no space between the variable name, equal sign, and value

Bonus: PLEASE be consistent

132/224
Declaring variables
a=z # Assign the string " z " to variable a.
b='a string' # Embedded spaces must be within quotes.
c="a string and $b" # Other expansions such as variables can be
# expanded into the assignment.
d=$(ls -l [Link]) # Results of a command.
e=$((5 \* 7)) # Arithmetic expansion.
f='\t\ta string\n' # Escape sequences such as tabs and newlines.

133/224

Avoiding ambiguity in variable names

Example
$ filename='myfile'
$ touch $filename
$ mv $filename $filename1

Must be...
$ mv $filename ${filename}1

134/224
Taking Variables a notch higher
#!/bin/bash
TITLE="System Information Report for $HOSTNAME"
CURRENT_TIME=$(date +"%x %r %Z")
TIME_STAMP="Generated $CURRENT_TIME, by $USER"

cat << _EOF_

<HTML>
<HEAD>
<TITLE>$TITLE</TITLE>
</HEAD>
<BODY>
<H1>$TITLE</H1>
<P>$TIME_STAMP</P>
</BODY>
</HTML>
_EOF_

135/224

Environment Variables
· Environment Variables are a set of dynamic named values that can
affect the way running processes will behave.
· To list all environment variables, use the env command
· They are typically located in the following config files:
- /etc/profile
- /etc/environment
- ~/.bash_profile
- ~/.profile
- ~/.bashrc

136/224
Environment Variables
Example
$ echo $PATH # PATH is an environment variable
/usr/local/bin:/usr/bin:/bin

Define an Environment Variable SCRIPT_HOME

$ SCRIPT_HOME=/home/user/bin
$ echo $SCRIPT_HOME
/home/user/bin

· initialize/declare Environment variables the same way we define

ordinary variables in the terminal or a script
· add/define Environment variables by editing ENV config file/s

137/224

Environment Variables
Define an Environment Variable in ~/.bashrc
$ vim .bashrc # edit the .bashrc file

Add the new Environment Variable

SCRIPT_HOME=/home/user/bin
export SCRIPT_HOME

138/224
Environment Variables
Test by reloading .bashrc or re-login the terminal session
$ echo $SCRIPT_HOME

$ source .bashrc
$ echo $SCRIPT_HOME
/home/user/bin

139/224

Passing arguments to the script

Make a script with a title my_name_is
#!/bin/bash
my_name=$1
echo Hi, my name is $my_name

Then run the script

$ ./my_name_is slim shady
$ ./my_name_is slim\ shady

140/224
Functions
#!/bin/bash
# Portion for functions
my_name=$1
rap_song() {
echo 'Hi, my name is'
}

# Start of Script
rap_song
echo what
rap_song
echo who
rap_song
echo $my_name

141/224

Functions

#!/bin/bash

function rap_song() {
echo 'Hi, my name is'
}

142/224
Functions - Arguments
· $#
- number of arguments
· $*
- all arguments
· $@
- all arguments, starting from first
· $1, $2, ... $n
- first, second, and nth argument

#!/bin/bash

function rap_song() {
echo "Hi, my name is $1"
}

rap_song "what"
rap_song "who"
rap_song "$1" 143/224

Conditional Statements
Syntax
if *commands*; then
*commands*
\[elif *commands*; then
*commands*...\]
\[else
*commands*\]
fi

$ x=5
$ if [ $x = 5 ]; then echo "equals 5"; else echo "not equal to 5"; fi

144/224
Conditional Statements - String Comparison
· [ -z STRING ]
- checks if STRING is empty
· [ -n STRING ]
- checks if STRING is not empty
· [ STRING1 == STRING2 ]
- checks if STRING1 and STRING2 are equal
· [ STRING1 != STRING2 ]
- checks if STRING1 and STRING2 are not equal

145/224

Conditional Statements - Numeric

Comparison
· [ NUM1 -eq NUM2 ]
- checks if NUM1 and NUM2 are equal
· [ NUM1 -ne NUM2 ]
- checks if NUM1 and NUM2 are not equal
· [ NUM1 -lt NUM2 ]
- checks if NUM1 is less than NUM2
· [ NUM1 -le NUM2 ]
- checks if NUM1 is less than or equal to NUM2

146/224
Conditional Statements - Numeric
Comparison
· [ NUM1 -gt NUM2 ]
- checks if NUM1 is greater than NUM2
· [ NUM1 -ge NUM2 ]
- checks if NUM1 is greater than or equal to NUM2

147/224

Conditional Statements - NOT, AND, OR, and

REGEX
· [ ! EXPRESSION ]
- negates the result of the EXPRESSION
· [ [EXPR1] && [EXPR2] ]
- result of both EXPR1 and EXPR2 should be true
· [ [EXPR1] || [EXPR2] ]
- either or both EXPR1 and EXPR2 should be true
· [ STRING =~ REGEX ]
- STRING1 should satify the REGEX expression

148/224
Conditional Statements - File
· [ -e FILE ]
- checks if the FILE exists
· [ -f FILE ]
- checks if the FILE is a file
· [ -d FILE ]
- checks if the FILE is a directory
· [ -x FILE ]
- checks if the FILE is executable

149/224

Conditional Statements - File

· [ -h FILE ]
- checks if the FILE is a symbolic link
· [ -r FILE ]
- checks if the FILE is readable
· [ -w FILE ]
- checks if the FILE is writable
· [ -s FILE ]
- checks if the FILE size is greater than 0 bytes

150/224
Conditional Statements - Using Case
case "$1" in
hello)
echo 'You said hello'
;;
hi)
echo 'You said hi'
;;
esac

151/224

Loops - For
Syntax
for var in **EXPR**; do
echo $var
done

Example
for word in `cat /usr/share/dict/words`; do
echo $word
done

for i in {1..2}; do
echo $i
done

152/224
Loops - While
Syntax
while **EXPR**; do
echo "I'm alive!!!"
done

Example
while true; do
echo "I'm alive!!!"
done

153/224

Loops - Reading Text File

< [Link] | while read line; do
echo $line
done

154/224
Brace Expansion
· {A, B}1
- results to A1 B1
· {A..E}
- results to A B C D E
· {Z..U}
- results to Z Y X V U
· {:..?}
- results to : ; < = > ?
· {1..10..2}
- results to 1 3 5 7 9

155/224

Arrays
Declaration
myarray=('quick' 'brown' 'fox' 'jumps')

myarray[1]='quick'
myarray[2]='brown'
myarray[3]='fox'
myarray[4]='jumps'

156/224
Arrays - Operations
myarray=("the" "${myarray[@]}")

· adds an element to the beginning of an array

myarray=("${myarray[@]}" "over")

· adds an element to the end of an arary

myarray+=("the")

· also adds an element to the end of an array

157/224

Arrays - Operations
myarray=("${myarray1[@]}" "${myarray2[@]}")

· combines both arrays

unset myarray[1]

· removes an element from the array

158/224
Arrays - Working with Arrays
${myarray[1]}

· gets the element at index 1

${myarray[@]} or ${myarray}

· returns all elements (space separated)

${#myarray[@]} or ${#myarray}

· returns the number of elements in the array

${#myarray[1]}

· returns the length of the element in index 1

159/224

Waiting for user inputs

echo -n "Are you sure you are okay?"

read input

echo $input

160/224
Exit Status
· a value of 0 indicates success
· a value of 1 or 2 indicates fail or error
· exit status can range from 0 to 255
Try the following commands
$ ls -d /usr/bin
$ echo $?
$ ls -d /bin/usr
$ echo $?

161/224

Miscellaneous - Special Variables

$$

· returns the PID of the current shell

$!

· returns the PID of the the last background task

162/224
Miscellaneous - Subshells
$ pwd
/home/training

$ (cd Documents; pwd)

/home/training/Documents

$ pwd
/home/training

163/224

Process Management
Processes 101
Boot process variants
1. init.d
· old/robust one
· runs init scripts that starts the system processes
· system processes to run is based on the runlevel

2. systemd
· "next generation"
· service level dependencies are defined to make boot process faster
· all services/process will start as control groups

165/224

Processes 101 - init.d

166/224
Init runlevels
init Description

0 shutdown or halt

1 /bin/sh with root privileges

2 ... with network

3,4,5 multi-user mode; going towards normal boot

6 restart

167/224

Processes 101 - Systemd

168/224
Viewing Processes
· ps
- Report a snapshot of current processes
· top
- Display tasks
· jobs
- List active Jobs

169/224

Viewing processes with ps

ps

$ ps

· by default shows processes associated with the current terminal

session
· PID represents the process id
· TTY represents teletype; controlling terminal of the process

170/224
Viewing processes with ps
ps x (Note that there is no dash)

$ ps x

· will show all processes regardless of what terminal it is controlled

· will show the status of the process
· may show too many output/list of running process; best to use with
grep

171/224

Viewing processes with ps

ps uax (Note that there is no dash)

$ ps uax

· will show processes belonging to every user

· will show other pertinent info per process
- USER - user owning the process
- %CPU - cpu usage in percentage
- %MEM - memory usage in percentage
- VSZ - virtual memory size
- RSS - resident set size; amount of physical RAM in kbytes
- START - time when the process started

172/224
Status codes for process
State Meaning

R Running

S Sleeping

D Uninterruptible Sleep; waiting for i/o

T Stopped

Z Zombie Process

< High-priority Process

N Low priority Process

· refer to man ps for the other supplemental status codes

173/224

Viewing Dynamically with top

top

$ top

· displays a continuous updating display of the system processes listed

in order of process activity
· used to see the "top" processes of the system
· also displays a "system summary"
· similar to using the MS Windows' Task Manager

174/224
Viewing Dynamically with top

· To quit, press Q

175/224

Showing jobs of a terminal with jobs

jobs

$ jobs

· shows the jobs running in the current terminal

· managing foreground and background processes

176/224
Managing jobs
· Ctrl+C
- interrupts or exits a process
· Ctrl+Z
- stop or pauses a process
· bg or &
- Places a job in the background
· fg
- Places a Job in the foreground

177/224

Interrupting a program
Do a Ctrl+C
Example
$ xlogo # then do a Ctrl+C
$

· it interrupts a program
· asks the program to terminate on the spot
· many (but not all) command-line programs can be interrupted by
using this technique

178/224
Stopping (or pausing) a program
Do a Ctrl+Z
Example
$ xlogo # then do a Ctrl+Z
[1] Stopped xlogo
$

· it stops a foreground process/program

· often used to allow a foreground process to be moved to the
background

179/224

Putting a Process in the Background

bg - puts a job in the background of the current terminal

$ bg %1 # the %1 is to select the previous command

[1]+ xlogo &
$

& - puts a process/program in the background of the current terminal

$ xlogo &
[1] #####
$

180/224
Putting a Process in the Background
· A process in the background is immune from keyboard input
· A process/job in the background will only exist while the current
terminal session is alive
· A background process/job will still output to the active terminal unless
redirected elsewhere before putting in the background

181/224

Returning a Process to the Foreground

fg - returns a job in the foreground

$ jobs
[1]+ Running xlogo &
$ fg %1
xlogo

· the command fg is followed by a percent sign and job number (or

jobspec).

182/224
Automating Jobs
· Users manage and create scheduled jobs or periodic execution of
scripts/commands via crontab
- Can create, edit, install and remove cron jobs
- Schedule to as accurate by the second
- Run commands/scripts by the system or by a specific user account
· A cron daemon/application is responsible for the execution of
scheduled scripts or commands
· The cron service periodically checks /etc/crontab and
/etc/cron.*/ for scheduled tasks/jobs

183/224

Automating Jobs via crontab

crontab

$ crontab -l # list scheduled jobs of the active user session

$ crontab -e # create/edit scheduled jobs on the active user session

· first-time access of the user crontab will prompt the default text
editor to use
· will generate a crontab file for the active user session

184/224
Automating Jobs via crontab
crontab job schedule format

m h dom mon dow command

* * * * * [command to be executed]

| | | | |
| | | | ----- Day of week (0 - 7) (Sunday=0 or 7)
| | | ------- Month (1 - 12)
| | --------- Day of month (1 - 31)
| ----------- Hour (0 - 23)
------------- Minute (0 - 59)

185/224

Automating Jobs via crontab

Examples
* * * * * /path/to/script # runs the script every 1 minute
5 0 * * * /path/to/script # runs the script 5 mins after midnight everyday
0 22 * * 1-5 /path/to/command # every 10 pm on weekdays
*/5 * * * /path/to/script # every 5 mins everyday
0 4 * * 0,3 /path/to/command # every 4 am on Sunday and Wednesday
45 11 1 * * /path/to/command # at 11:45 am on the first of every month
0 18 5 4-5 * /path/to/script # on the 5th of April and May, at 6 pm

· asterisk (*) specifies all possible values

· comma (,) specifies list of values
· dash (-) specifies a range of values
· separator (/) specifies a step-value

186/224
Handling cronjob outputs
MAILTO="armand@[Link]"
0 10 * * * /path/to/[Link] > /dev/null 2>&1
5 0 * * * /path/to/command | mail -s "command results" armand@[Link]
*/15 13-18/2 * * * /path/to/[Link] > /var/log/`date +%F`-[Link] 2>&1

· by default, cron jobs will send the output of the executed

script/command to your "local email"
· you can either change the recipient email address via the MAILTO
variable
- or pipe the output to mail command if installed
· you can redirect the stdout (or stderr) to a file or to /dev/null

187/224

Signals
· mechanism in communicating with programs/processes
· programs "listen" for signals and act upon receiving one
· the keyboard commands Ctrl+C and Ctrl+Z also sends signals
- Ctrl+C sends a INT (interrupt) signal
- Ctrl+Z sends a TSTP (terminal stop)

188/224
Common Signals
Number Name Meaning

1 HUP Hang-up

2 INT Interrupt; terminates a program on-point, Ctrl+C

3 QUIT Quit

9 KILL tells terminal to "kill" the program by force

15 TERM Terminate "gently"; the default signal sent by kill

18 CONT Continue; restore a STOP signal

19 STOP Stop; process stops without terminating

20 TSTP Terminal Stop; Ctrl+Z

189/224

Process Management commands

· kill -[signal] [pid]
- Send a signal to a process
- Do a kill -l for a complete list of signals
· killall -[signal] [name]
- Kill process/es by name
· shutdown
- shuts down or reboot the system

190/224
Other Process-Related commands
1. pstree
· Outputs a process list arranged in a tree pattern
· Useful to identify the parent-child relationship between processes

2. tload
· display a graph in the terminal of system load vs. time

3. vmstat
· Displays a snapshot of system resource usage

4. df
· Displays a report on file system storage usage

5. free
· Displays current physical and virtual memory usage

191/224

Package Management
About Package Systems
· Linux applications are distributed in a form of Package Files
- a compressed collection of files that compromise the software package
- may contain supporting apps for the target software
- created by Package Managers
· Package files are hosted in Repositories (or Mirrors)
- may be hosted by the software vendor or third-parties
- contains thousands of packages, typically grouped by distro and OS
architecture
- may contain various versions of a package/software

193/224

Major Packaging System Families

1. Debian Style - .deb
· Debian
· Ubuntu

2. RedHat Style - .rpm

· Oracle RHEL
· CentOS
· Fedora
· openSUSE

194/224
Package Tools
Distributions Low-level Tools High-level Tools

Debian (.deb) dpkg apt-get, aptitude, apt

RedHat (.rpm) rpm yum, dnf

195/224

Dependencies
· Programs typically rely on other software/program in order to work
- Common functions such as i/o control and hardware routines
- Will need to install shared libraries to satisfy the dependencies of
programs to be installed
· Modern or the High-level Package management tools takes care of
including these dependencies when installing an application
- Makes a list of package dependencies of intended software
- Fetches the package software and package dependencies from the
Repository (online)
- Install all the packages into the OS

196/224
Configuring Target Package Repositories
in Debian

· typically configured by default

· List of mirrors to be accessed are configured at
/etc/apt/[Link]
- or make an individual entry per mirror site at
/etc/apt/[Link].d/

in RedHat (Fedora)

· the configuration for the package repositories may vary per distro
· typically configured at /etc/[Link]
- or make an individual entry per mirror site at /etc/[Link].d/

197/224

Searching for a Package from the Repository

in Debian

# apt-get update
# apt-cache search [search string] # searches for a package matching the string
# apt-cache show [package name] # display details of identified package

in RedHat

# yum search [search string] # searches for a package matching the string
# yum info [package name] # display details of identified package

Note: may have to do a apt-get clean in Debian distros to flush old

cache

198/224
Installing a Package from the Repository
in Debian

# apt-get update # optional if you've recently done this

# apt-get install [package name]

in RedHat

# yum install [package name]

199/224

Installing a Package from a local file

in Debian

# dpkg -i [path to package file]

in RedHat

# rpm -i [path to package file]

Note: If there are unmet dependencies, the installation will stop/fail

200/224
Updating Packages from the Repository
in Debian

# apt-get update # optional if you've recently done this

# apt-get upgrade [package name]

in RedHat

# yum update

201/224

Updating Packages from a local file

in Debian

# dpkg -i [path to package file]

in RedHat

# rpm -U [path to package file]

· dpkg doesn't have a separate option for update unlike rpm

202/224
List/Query about a (installed) Package
in Debian

# dpkg --list # list packages installed

# dpkg --search [path to file] # searches which package originated the file/command
# dpkg --status [package name] # check if the package is installed

in RedHat

# rpm -qa # list packages installed

# rpm -qf [path to file] # searches which package originated the file/command
# rpm -q [package name] # check if the package is installed

203/224

Removing an Installed Package

in Debian

# apt-get remove [package name] # when installed from repos

# dpkg -r [package name] # when installed from either repo or file

in RedHat

# yum remove [package name] # when installed from repos

# rpm -evv [package name] # when installed from a package file

204/224
 Basic Networking

Basic Network Monitoring

ping

$ ping [Link]

traceroute

$ traceroute [Link]

mtr

$ mtr [Link]

206/224
Network Settings and Statistics - netstat
netstat

$ netstat

· used for examining network settings and stats

· wide array of uses via different options
· mostly used for diagnosing...
- network interfaces
- IP routes

207/224

Network Settings and Statistics - netstat

netstat -ie

$ netstat -ie

· display details for "detected" network interfaces

- eth# - typically refers to ethernet LAN
- lo - loopback interface; localhost
- wlan# - wireless LAN (wifi) interface
- other network interfaces (i.e. vlan & bridge)

208/224
Network Settings and Statistics - netstat
netstat -r

$ netstat -r

· displays the IP routing table

Note: requires some general knowledge on IP routing table
management to understand

209/224

Network Configuration files

· Network Interfaces
- Debian/Ubuntu
- /etc/network/interfaces
- Debian/Ubuntu (using NetworkManager)
- /etc/NetworkManager
- Redhat/Fedora
- /etc/sysconfig/network-scripts/ifcfg-[interface]
- /etc/sysconfig/network

210/224
Network Configuration files
· Machine Hostname
- /etc/hosts - can be modified to override DNS routing table
- /etc/hostname - contains the value for hostname

· Nameserver resolver (DNS)

- /etc/[Link] - contains value for IPs of nameserver

211/224

Start/Stop/Restart the Network Connections

for Debian/Ubuntu

# service network-manager [start | stop | restart]

for Debian/Ubuntu (using NetworkManager)

# service network-manager [start | stop | restart]

for RedHat/Fedora

# service network [start | stop | restart]

212/224
Network Management commands
· ifconfig
- similar to netstat -ie and MS-DOS ipconfig

· ip [options] [object]
- a replacement for ifconfig
· ifup [interface name]
- enables a given network interface
· ifdown [interface name]
- disables a given network interface

213/224

Remote Shell Access via SSH

ssh - OpenSSH Secure Shell Client

$ ssh [username@][target IP/domain address][:port]

· secure communication that enables terminal control to a remote host

· also used as a medium for file/data transfers (e.g. Git, scp)
· majority of UNIX-based operating systems implement a OpenSSH
server and client application
· target remote host should have openssh-server package/service
installed
· typically listens on port 22

214/224
File Transfers Protocol
ftp - FTP terminal client

$ ftp [target IP/domain address]

sftp - Secure FTP terminal client

$ sftp [target IP/domain address]

· Implementation of the File Transfer Protocol client access

· Provides a command prompt FTP terminal access to the target host
- Uses typical FTP syntax commands

215/224

Typical FTP commands

Command Meaning

ls list directory on remote host

lcd changes the working directory on the local system; the default directory is from where the ftp
command was executed

get tells remote host to transfer the target file to the working local directory

put transfers the local target file to the remote host

bye Log-off from the remote host and ends the ftp session

Note: you can refer to man ftp for more commands

216/224
Download files over the web
wget - non-interactive HTTP GET downloader

$ wget [target URL of file]

· useful for downloading content from both publicly-accessible web and

FTP sites
· wide array of options to let you download files in the background or
recursively
· refer to man wget

217/224

Secure Copy via SSH - scp

scp - Secure Copy (like cp)

$ scp [source host]:[file] [destination host]:[file]

· uses SSH protocol to transfer files remotely

· must have openssh-server running on target remote host
· follow the syntax behavior of cp but allows you to define remote
source/destination

218/224
Secure Copy via SSH - scp
· Transfer a file to a remote host
$ scp /path/to/local/file user@[Link]:/destination/path/

· Download a file from a remote host

$ scp user@[Link]:/path/to/remote/file /destination/path/

· Download files/folders recursively from a remote host directory

$ scp -R user@[Link]:/path/to/remote/folder/ /destination/path/

Note: Will ask for password/credentials for the remote host

219/224

Day 2 Exercise
Day 2 Exercise
· install the apache2 application
- once it finished the installation, test it by using a browser and access
[Link]

· make a simple [Link] in your home directory

- save it in a folder apache-html
- make a soft-link of that folder inside /var/www/
- even just a basic "hello world" HTML will do; bonus if you can add some
finesse
· make a copy of the /etc/apache2/sites-avaiable/000-
[Link]
- save it in the same location, name it as [Link]

221/224

Day 2 Exercise
· make some minor changes in [Link]
- ServerAdmin email
- the DocumentRoot; should match the new soft-link you made in no.2
- the filename output for ErrorLog and CustomLog
· replace the [Link] in /etc/apache2/sites-enabled
- make a new soft-link of the [Link] inside the sites-
enabled folder

· do a sudo service apache2 restart

- if it successfully restarted, test it again with a browser
via[Link]

222/224
State of the UNIX address

<Thank You!>
training-sales@[Link]