10.

1 Introduction
A large number of the files in a typical file system are text files . Text files only contain text, no
formatting features that you might see in a word processing file.

Because there are so many of these files on a typical Linux system, a significant number of
commands exist to help users manipulate text files. There are commands to both view and
modify these files in various ways.

Additionally, there are features available for the shell to control the output of commands, so
instead of having the output placed in the terminal window, the output can be redirected into
another file or another command. These redirection features provide users with a much more
flexible and powerful environment to work within.

​
​

10.1.1 Viewing Files in the Terminal

The cat command, short for concatenate , is a simple but useful command whose functions
include creating and displaying text files, as well as combining copies of text files. One of the
 most popular uses of cat is to display the content of text files. To display a file in the standard
output using the cat command, type the command followed by the filename:

sysadmin@localhost:~$ cd Documents
sysadmin@localhost:~/Documents$ cat [Link]
Food is good.

Although the terminal is the default output of this command, the cat command can also be used
for redirecting file content to other files or input for another command by using redirection
characters.

​
​

10.1.2 Viewing Files Using a Pager

While viewing small files with the cat command poses no problems, it is not an ideal choice for
large files. The cat command doesn't provide any easy ways to pause and restart the display, so
the entire file contents are dumped to the screen.

For larger files, use a pager command to view the contents. Pager commands display one page
of data at a time, allowing you to move forward and backward in the file by using movement
keys.

There are two commonly used pager commands:

●​ The less command provides a very advanced paging capability. It is usually the default
pager used by commands like the man command.
●​ The more command has been around since the early days of UNIX. While it has fewer
features than the less command, however, the less command isn't included with all
Linux distributions. The more command is always available.

The more and less commands allow users to move around the document using keystroke
commands. Because developers based the less command on the functionality of the more
command, all of the keystroke commands available in the more command also work in the less
command.

The focus of our content is on the more advanced less command. The more command is still
useful to remember for times when the less command isn't available. Remember that most of
the keystroke commands provided work for both commands.

​
​

[Link] Pager Movement Commands

 To view a file with the less command, pass the file name as an argument:

sysadmin@localhost:~/Documents$ less words

There are many movement commands for the less command, each with multiple possible keys
or key combinations. While this may seem intimidating, it is not necessary to memorize all of
these movement commands. When viewing a file with the less command, use the H key or
Shift+H to display a help screen:
SUMMARY OF LESS COMMANDS

Commands marked with * may be preceded by a number, N.

Notes in parentheses indicate the behavior if N is given.
A key preceded by a caret indicates the Ctrl key; thus ^K is ctrl-K.

h H Display this help.

q :q Q :Q ZZ Exit.
------------------------------------------------------------------------
MOVING

e ^E j ^N CR * Forward one line (or N lines).

y ^Y k ^K ^P * Backward one line (or N lines).
f ^F ^V SPACE * Forward one window (or N lines).
b ^B ESC-v * Backward one window (or N lines).
z * Forward one window (and set window to N).
w * Backward one window (and set window to N).
ESC-SPACE * Forward one window, but don't stop at end-of-file.
d ^D * Forward one half-window (and set half-window to N).
u ^U * Backward one half-window (and set half-window to N).
ESC-) RightArrow * Left one half screen width (or N positions).
ESC-( LeftArrow * Right one half screen width (or N positions).
HELP -- Press RETURN for more, or q when done

The first group of movement commands to focus on are the ones that are most commonly used.
To make it even more convenient, the keys that are identical in more and less are summarized
below in order to demonstrate how to move in more and less at the same time:

‌⁠​
Key Movement

Spacebar Window forward

 B Window backward

Enter Line forward

Q Exit

H Help

When using less as a pager, the easiest way to advance forward a page is to press the
Spacebar.

​
​

[Link] Pager Searching Commands

There are two ways to search in the less command: searching forward or backward from your
current position.

To start a search to look forward from your current position, use the slash / key. Then, type the
text or pattern to match and press the Enter key.
Abdul
Abdul's
Abe
/frog

If a match can be found, then the cursor moves in the document to the match. For example, in
the following graphic the expression "frog" was searched for in the words file:

bullfrog
bullfrog's
bullfrogs
bullheaded
bullhorn
bullhorn's

Notice that "frog" didn't have to be a word by itself. Also notice that while the less command
moved to the first match from the current position, all matches were highlighted.
 If no matches forward from your current position can be found, then the last line of the screen will
report Pattern not found:

None
Pattern not found (press RETURN)

To search backward from your current position, press the question mark ? key, then type the text
or pattern to match and press the Enter key. The cursor moves backward to the first match it can
find or reports that the pattern cannot be found.

If more than one match can be found by a search, then use the n key to move the next match
and use the Shift+N key combination to go to a previous match.

The search terms actually use patterns called regular expressions . More details regarding
regular expressions are provided later in this chapter.

​
​

10.1.3 Head and Tail

The head and tail commands are used to display only the first few or last few lines of a file,
respectively (or, when used with a pipe, the output of a previous command). By default, the head
and tail commands display ten lines of the file that is provided as an argument.

For example, the following command displays the first ten lines of the /etc/[Link] file:

sysadmin@localhost:~/Documents$ cd
sysadmin@localhost:~$ head /etc/[Link]
#
# /etc/[Link] - Configuration file for setting system variables
# See /etc/sysctl.d/ for additional system variables
# See [Link] (5) for information.
#

#[Link] = [Link]

# Uncomment the following to stop low-level messages on console

#[Link] = 3 4 1 3

Passing a number as an option will cause both the head and tail commands to output the
specified number of lines, instead of the standard ten. For example to display the last five lines of
the /etc/[Link] file use the -5 option:

sysadmin@localhost:~$ tail -5 /etc/[Link]

# Protects against creating or following links under certain conditions
# Debian kernels have both set to 1 (restricted)
# See [Link]
#fs.protected_hardlinks=0
#fs.protected_symlinks=0

The -n option can also be used to indicate how many lines to output. Pass a number as an
argument to the option:
sysadmin@localhost:~$ head -n 3 /etc/[Link]
#
# /etc/[Link] - Configuration file for setting system variables
# See /etc/sysctl.d/ for additional system variables

Negative Value Option

Traditionally in UNIX, the number of lines to output would be specified as an option with either
command, so -3 meant to show three lines. For the tail command, either -3 or -n -3 still
means show three lines.

However, the GNU version of the head command recognizes -n -3 as show

all but the last three lines , and yet the head command still recognizes the option -3 as show the
first three lines.

Positive Value Option

The GNU version of the tail command allows for a variation of how to specify the number of
lines to be printed. If the -n option is used with a number prefixed by the plus sign, then the tail
command recognizes this to mean to display the contents starting at the specified line and
continuing all the way to the end.

For example, the following displays the contents of the /etc/passwd from line 25 to the end of
the file:
sysadmin@localhost:~$ nl /etc/passwd | tail -n +25
25 sshd:x:103:65534::/var/run/sshd:/usr/sbin/nologin
26 operator:x:1000:37::/root:/bin/sh
27 sysadmin:x:1001:1001:System Administrator,,,,:/home/sysadmin:/bin/bash

Consider This

Live file changes can be viewed by using the -f option to the tail command—useful when you
want to see changes to a file as they are happening.

A good example of this would be when viewing log files as a system administrator. Log files can
be used to troubleshoot problems and administrators often view them "interactively" with the
tail command while performing commands in a separate window.
 For example, if you were to log in as the root user, you could troubleshoot issues with the email
server by viewing live changes to the /var/log/[Link] log file.

​
​

10.2 Command Line Pipes

The pipe | character can be used to send the output of one command to another. Typically,
when a command has output or generates an error, the output is displayed to the screen;
however, this does not have to be the case. Instead of being printed to the screen, the output of
one command becomes input for the next command. This tool can be powerful, especially when
looking for specific data; piping is often used to refine the results of an initial command.

In previous examples the head and tail commands were given files as arguments to operate
on. However, the pipe character allows you to utilize these commands not only on files, but on
the output of other commands. This can be useful when listing a large directory, for example the
/etc directory:

sysadmin@localhost:~$ ls /etc
X11 gss [Link] rpc
[Link] [Link] modprobe.d [Link]
alternatives hostname modules rsyslog.d
apparmor hosts modules-load.d securetty
apparmor.d [Link] motd security
apt [Link] mtab selinux
[Link] init.d nanorc services
bind initramfs-tools netplan shadow
[Link] inputrc network shadow-
binfmt.d [Link].d networks shells
ca-certificates iproute2 newt skel
[Link] issue [Link] ssh
calendar [Link] opt ssl
console-setup kernel os-release subgid
cron.d [Link] [Link] subgid-
[Link] [Link] pam.d subuid
[Link] [Link].d passwd subuid-
[Link] ldap passwd- sudoers
[Link] legal perl sudoers.d
crontab [Link] pinforc [Link]
dbus-1 [Link] ppp sysctl.d
[Link] [Link] profile systemd
debian_version localtime profile.d terminfo
default logcheck protocols timezone
[Link] [Link] python3 tmpfiles.d
depmod.d [Link] python3.6 [Link]
dhcp logrotate.d rc0.d udev
dpkg lsb-release rc1.d ufw
environment machine-id rc2.d update-motd.d
fstab magic rc3.d [Link]
[Link] [Link] rc4.d vim
groff mailcap rc5.d vtrgb
group [Link] rc6.d wgetrc
group- [Link] rcS.d xdg
gshadow mc [Link]
gshadow- [Link] rmt

The previous command lists a large number of files. If you execute this in our terminal, the output
is cut off and can only be viewed if scrolling up. To more easily view the beginning of the output,
pipe it to the head command. The following example displays only the first ten lines:

sysadmin@localhost:~$ ls /etc | head

X11
[Link]
alternatives
apparmor
apparmor.d
apt
[Link]
bind
[Link]
binfmt.d

The full output of the ls command is passed to the head command by the shell instead of being
printed to the screen. The head command takes this output from the ls command as input data,
and the output of head is then printed to the screen.

Multiple pipes can be used consecutively to link multiple commands together. If three commands
are piped together, the output of the first command is passed to the second command. Then, the
output of the second command is passed to the third command. The output of the third command
would then be printed to the screen.

It is important to carefully choose the order in which commands are piped, as each command
only sees input from the previous command. The examples below illustrate this using the nl
command, which adds line numbers to the output. In the first example, the nl command is used
to number the lines of the output of the preceding ls command:

sysadmin@localhost:~$ ls /etc/ssh | nl
1 moduli
2 ssh_config
3 ssh_host_ecdsa_key
4 ssh_host_ecdsa_key.pub
5 ssh_host_ed25519_key
6 ssh_host_ed25519_key.pub
7 ssh_host_rsa_key
8 ssh_host_rsa_key.pub
 9 ssh_import_id
10 sshd_config

In the next example, note that the ls command is executed first and its output is sent to the nl
command, numbering all of the lines from the output of the ls command. Then the tail
command is executed, displaying the last five lines from the output of the nl command:

sysadmin@localhost:~$ ls /etc/ssh | nl | tail -5

6 ssh_host_ed25519_key.pub
7 ssh_host_rsa_key
8 ssh_host_rsa_key.pub
9 ssh_import_id
10 sshd_config

Compare the output above with the next example:

sysadmin@localhost:~$ ls /etc/ssh | tail -5 | nl
1 ssh_host_ed25519_key.pub
2 ssh_host_rsa_key
3 ssh_host_rsa_key.pub
4 ssh_import_id
5 sshd_config

Notice how the line numbers are different. Why is this?

In the second example, the output of the ls command is first sent to the tail command which
only grabs the last five lines of the output. Then the tail command sends those five lines to the
nl command, which numbers them 1-5.

Pipes can be powerful, but it is necessary to consider how commands are piped to ensure that
the desired output is displayed.

​
​

10.3 Input/Output Redirection

Input/Output (I/O) redirection allows for command line
information to be passed to different streams. Before discussing redirection, it is important to
understand the standard streams .

​ STDIN Standard input ​

, or STDIN , is information entered normally by the user via the keyboard. When a
 command prompts the shell for data, the shell provides the user with the ability to type
commands that, in turn, are sent to the command as STDIN.
​ STDOUT Standard output ​
, or STDOUT , is the normal output of commands. When a command functions correctly
(without errors) the output it produces is called STDOUT. By default, STDOUT is
displayed in the terminal window where the command is executing. STDOUT is also
known as stream or channel #1.
​ STDERR Standard error ​
, or STDERR , is error messages generated by commands. By default, STDERR is
displayed in the terminal window where the command is executing. STDERR is also
known as stream or channel #2.​
‌⁠​

I/O redirection allows the user to redirect STDIN so that data comes from a file and
STDOUT/STDERR so that output goes to a file. Redirection is achieved by using the arrow < >
characters.

​
​

10.3.1 STDOUT
STDOUT can be directed to files. To begin, observe the output of the following echo command
which displays to the screen:
sysadmin@localhost:~$ echo "Line 1"
Line 1

Using the > character, the output can be redirected to a file instead:

sysadmin@localhost:~$ echo "Line 1" > [Link]

This command displays no output because STDOUT was sent to the file [Link] instead
of the screen. You can see the new file with the output of the ls command.

sysadmin@localhost:~$ ls
Desktop Downloads Pictures Templates [Link]
Documents Music Public Videos

The file contains the output of the echo command, which can be viewed with the cat command:

sysadmin@localhost:~$ cat [Link]

Line 1
 It is important to realize that the single arrow overwrites any contents of an existing file:
sysadmin@localhost:~$ cat [Link]
Line 1
sysadmin@localhost:~$ echo "New line 1" > [Link]
sysadmin@localhost:~$ cat [Link]
New line 1

The original contents of the file are gone, replaced with the output of the new echo command.

It is also possible to preserve the contents of an existing file by appending to it. Use two arrow >>
characters to append to a file instead of overwriting it:
sysadmin@localhost:~$ cat [Link]
New line 1
sysadmin@localhost:~$ echo "Another line" >> [Link]
sysadmin@localhost:~$ cat [Link]
New line 1
Another line

Instead of being overwritten, the output of the echo command is added to the bottom of the file.

​
​

10.3.2 STDERR
STDERR can be redirected similarly to STDOUT. When using the arrow character to redirect,
stream #1 (STDOUT) is assumed unless another stream is specified. Thus, stream #2 must be
specified when redirecting STDERR by placing the number 2 preceding the arrow > character.

To demonstrate redirecting STDERR, first observe the following command which produces an
error because the specified directory does not exist:
sysadmin@localhost:~$ ls /fake
ls: cannot access /fake: No such file or directory

Note that there is nothing in the example above that implies that the output is STDERR. The
output is clearly an error message, but how could you tell that it is being sent to STDERR? One
easy way to determine this is to redirect STDOUT:
sysadmin@localhost:~$ ls /fake > [Link]
ls: cannot access /fake: No such file or directory

In the example above, STDOUT was redirected to the [Link] file. So, the output that is
displayed can't be STDOUT because it would have been placed in the [Link] file instead
 of the terminal. Because all command output goes either to STDOUT or STDERR, the output
displayed above must be STDERR.

The STDERR output of a command can be sent to a file:

sysadmin@localhost:~$ ls /fake 2> [Link]

In the example, the 2> indicates that all error messages should be sent to the file [Link],
which can be confirmed using the cat command:

sysadmin@localhost:~$ cat [Link]

ls: cannot access /fake: No such file or directory

​
​

10.3.3 Redirecting Multiple Streams

It is possible to direct both the STDOUT and STDERR of a command at the same time. The
following command produces both STDOUT and STDERR because one of the specified
directories exists and the other does not:
sysadmin@localhost:~$ ls /fake /etc/ppp
ls: cannot access /fake: No such file or directory
/etc/ppp:
ip-down.d ip-up.d

If only the STDOUT is sent to a file, STDERR is still printed to the screen:
sysadmin@localhost:~$ ls /fake /etc/ppp > [Link]
ls: cannot access /fake: No such file or directory
sysadmin@localhost:~$ cat [Link]
/etc/ppp:
ip-down.d
ip-up.d

If only the STDERR is sent to a file, STDOUT is still printed to the screen:
sysadmin@localhost:~$ ls /fake /etc/ppp 2> [Link]
/etc/ppp:
ip-down.d
ip-up.d
sysadmin@localhost:~$ cat [Link]
ls: cannot access /fake: No such file or directory
 Both STDOUT and STDERR can be sent to a file by using the ampersand & character in front of
the arrow > character. The &> character set means both 1> and 2>:

sysadmin@localhost:~$ ls /fake /etc/ppp &> [Link]

sysadmin@localhost:~$ cat [Link]
ls: cannot access /fake: No such file or directory
/etc/ppp:
ip-down.d
ip-up.d

Note that when you use &>, the output appears in the file with all of the STDERR messages at
the top and all of the STDOUT messages below all STDERR messages:
sysadmin@localhost:~$ ls /fake /etc/ppp /junk /etc/sound &> [Link]
sysadmin@localhost:~$ cat [Link]
ls: cannot access '/fake': No such file or directory
ls: cannot access '/junk': No such file or directory
ls: cannot access '/etc/sound': No such file or directory
/etc/ppp:
ip-down.d
ip-up.d

If you don't want STDERR and STDOUT to both go to the same file, they can be redirected to
different files by using both > and 2>. For example, to direct STDOUT to [Link] and
STDERR to [Link] execute the following:

sysadmin@localhost:~$ ls /fake /etc/ppp > [Link] 2> [Link]

sysadmin@localhost:~$ cat [Link]
ls: cannot access /fake: No such file or directory
sysadmin@localhost:~$ cat [Link]
/etc/ppp:
ip-down.d
ip-up.d

The order in which the streams are specified does not matter.

​
​

10.3.4 STDIN
The concept of redirecting STDIN is a difficult one because it is more difficult to understand why
you would want to redirect STDIN. With STDOUT and STDERR, their purpose is straightforward;
sometimes it is helpful to store the output into a file for future use.
 Most Linux users end up redirecting STDOUT routinely, STDERR on occasion, and STDIN very
rarely.

There are very few commands that require you to redirect STDIN because with most commands
if you want to read data from a file into a command, you can specify the filename as an argument
to the command.

For some commands, if you don't specify a filename as an argument, they revert to using STDIN
to get data. For example, consider the following cat command:

sysadmin@localhost:~$ cat
hello
hello‌⁠
​
how are you?
how are you?
goodbye
goodbye

Note

If you do attempt the cat command without arguments, kill the process and return to the prompt
by using Ctrl+C.

In the preceding example, the cat command isn't provided a filename as an argument. So, it
asks for the data to display on the screen from STDIN. The user types hello, and then the cat
command displays hello on the screen. While this is mildly entertaining, it isn’t particularly
useful.

However, if the output of the cat command were redirected to a file, then this method could be
used either to add text to an existing file or to place text into a new file.

The first command in the example below redirects the output of the cat command to a newly
created file called [Link]. This action is followed up by providing the cat command with the
[Link] file as an argument to display the redirected text in STDOUT.

sysadmin@localhost:~$ cat > [Link]

Hello
How are you?
Goodbye
sysadmin@localhost:~$ cat [Link]
Hello
How are you?
Goodbye

While the previous example demonstrates another advantage of redirecting STDOUT, it doesn't
address why or how STDIN can be directed. To understand this, consider a new command called
tr. This command takes a set of characters and translates them into another set of characters.
 For example, to capitalize a line of text use the tr command as follows:

sysadmin@localhost:~$ tr 'a-z' 'A-Z'

watch how this works
WATCH HOW THIS WORKS

The tr command took the STDIN from the keyboard and converted all lower-case letters before
sending STDOUT to the screen.

It would seem that a better use of the tr command would be to perform translation on a file, not
keyboard input. However, the tr command does not support file name arguments:

sysadmin@localhost:~$ cat [Link]

/etc/ppp:
ip-down.d
ip-up.d
sysadmin@localhost:~$ tr 'a-z' 'A-Z' [Link]
tr: extra operand `[Link]'
Try `tr --help' for more information

It is possible, however, to tell the shell to get STDIN from a file instead of from the keyboard by
using the < character:

sysadmin@localhost:~$ tr 'a-z' 'A-Z' < [Link]

/ETC/PPP:
IP-DOWN.D
IP-UP.D

Most commands do accept file names as arguments, so this use case is relatively rare. However,
for those that do not, this method could be used to have the shell read from the file instead of
relying on the command to have this ability.

One last note to save the resulting output, redirect it into another file:
sysadmin@localhost:~$ tr 'a-z' 'A-Z' < [Link] > [Link]
sysadmin@localhost:~$ cat [Link]
/ETC/PPP:
IP-DOWN.D
IP-UP.D

​
​

10.4 Sorting Files or Input

 The sort command can be used to rearrange the lines of files or input in either dictionary or
numeric order. The following example creates a small file, using the head command to grab the
first 5 lines of the /etc/passwd file and send the output to a file called mypasswd.

sysadmin@localhost:~$ head -5 /etc/passwd > mypasswd

sysadmin@localhost:~$ cat mypasswd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync

Now we will sort the mypasswd file:

sysadmin@localhost:~$ sort mypasswd

bin:x:2:2:bin:/bin:/usr/sbin/nologin
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
root:x:0:0:root:/root:/bin/bash
sync:x:4:65534:sync:/bin:/bin/sync
sys:x:3:3:sys:/dev:/usr/sbin/nologin

Upon close examination of the output in the preceding example, the sort command has
arranged the lines of the file in alphabetical order. Compare this output to the output of the
previous cat command.

​
​

10.4.1 Fields and Sort Options

The sort command can rearrange the output based on the contents of one or more fields. Fields
are determined by a field delimiter contained on each line. In computing, a delimiter is a
character that separates a string of text or data; it defaults to whitespace, like spaces or tabs.

The following command can be used to sort the third field of the mypasswd file numerically.
Three options are used to achieve this sort:

O
o Function
 -t The -t option specifies the field delimiter . If the file or input is separated by
delimiter other than whitespace, for example a comma or colon, the -t option
will allow for another field separator to be specified as an argument.

The mypasswd file used in the previous example uses a colon : character as
delimiter to separate the fields, so the following example uses the -t: option

-k The -k option specifies the field number . To specify which field to sort by, us
the -k option with an argument to indicate the field number, starting with 1 fo
the first field.

The following example uses the -k3 option to sort by the third field.

-n This option specifies the sort type .

The third field in the mypasswd file contains numbers, so the -n option is use
to perform a numeric sort.

sysadmin@localhost:~$ sort -t: -n -k3 mypasswd

root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync

Another commonly used option to the sort command is the -r option, which is used to perform
a reverse sort. The following shows the same command as the previous example, with the
addition of the -r option, making the higher numbers in the third field appear at the top of the
output:
sysadmin@localhost:~$ sort -t: -n -r -k3 mypasswd
sync:x:4:65534:sync:/bin:/bin/sync
sys:x:3:3:sys:/dev:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
root:x:0:0:root:/root:/bin/bash
 Lastly, you may want to perform more complex sorts, such as sorting by a primary field and then
by a secondary field. For example, consider the following comma-separated value file, a file
where the comma character is the field delimiter:

Follow Along

Use the following command to switch to the Documents directory:

sysadmin@localhost:~$ cd ~/Documents

sysadmin@localhost:~/Documents$ cat [Link]

1970,Unix,Richie
1987,Minix,Tanenbaum
1970,Unix,Thompson
1991,Linux,Torvalds

To sort first by the operating system (field #2) and then year (field #1) and then by last name
(field #3), use the following command:
sysadmin@localhost:~/Documents$ sort -t, -k2 -k1n -k3 [Link]
1991,Linux,Torvalds
1987,Minix,Tanenbaum
1970,Unix,Richie
1970,Unix,Thompson

The following table breaks down the options used in the previous example:

Optio Function

-t, Specifies the comma character as the field delimiter

-k2 Sort by field #2

-k1n Numerically sort by field #1

-k3 Sort by field #3

Use the following command to return to the home directory:

sysadmin@localhost:~/Documents$ cd
sysadmin@localhost:~$

​
​

10.5 Viewing File Statistics

The wc command provides the number of lines, words and bytes (1 byte = 1 character in a text
file) for a file, and a total line count if more than one file is specified. By default, the wc command
allows for up to three statistics to be printed for each file provided, as well as the total of these
statistics if more than one filename is provided:
sysadmin@localhost:~$ wc /etc/passwd /etc/passwd-
35 56 1710 /etc/passwd
34 55 1665 /etc/passwd-
69 111 3375 total

The output of the previous example has four columns:

1.​ Number of lines

2.​ Number of words
3.​ Number of bytes
4.​ File name

It is also possible to view only specific statistics, by using the -l option to show just the number
of lines, the -w option to show just the number of words, the -c option to show just the number of
bytes, or any combination of these options.

The wc command can be useful for counting the number of lines output by some other command
through a pipe. For example, if you wanted to know the total number of files in the /etc
directory, pipe the output of ls to wc and count only the number of lines:

sysadmin@localhost:~$ ls /etc/ | wc -l
142

​
​

10.6 Filter File Sections

The cut command can extract columns of text from a file or standard input. It’s primarily used for
working with delimited database files. Again, delimited files are files that contain columns
separated by a delimiter. These files are very common on Linux systems.
 By default, the cut command expects its input to be separated by the tab character, but the -d
option can specify alternative delimiters such as the colon or comma.

The -f option can specify which fields to display, either as a hyphenated range or a
comma-separated list.

In the following example, the first, fifth, sixth and seventh fields from the mypasswd database file
are displayed:
sysadmin@localhost:~$ cut -d: -f1,5-7 mypasswd
root:root:/root:/bin/bash
daemon:daemon:/usr/sbin:/usr/sbin/nologin
bin:bin:/bin:/usr/sbin/nologin
sys:sys:/dev:/usr/sbin/nologin
sync:sync:/bin:/bin/sync

The cut command is also able to extract columns of text based upon character position with the
-c option—useful when working with fixed-width database files or command outputs.

For example, the fields of the ls -l command are always in the same character positions. The
following will display just the file type (character 1), permissions (characters 2-10), a space
(character 11), and filename (characters 50+):
sysadmin@localhost:~$ ls -l | cut -c1-11,50-
total 44
drwxr-xr-x Desktop
drwxr-xr-x Documents
drwxr-xr-x Downloads
drwxr-xr-x Music
drwxr-xr-x Pictures
drwxr-xr-x Public
drwxr-xr-x Templates
drwxr-xr-x Videos
-rw-rw-r-- [Link]
-rw-rw-r-- [Link]
-rw-rw-r-- mypasswd
-rw-rw-r-- [Link]

​
​

10.7 Filter File Contents

The grep command can be used to filter lines in a file or the output of another command that
matches a specified pattern. That pattern can be as simple as the exact text that you want to
match or it can be much more advanced through the use of regular expressions.
 For example, to find all the users who can log in to the system with the BASH shell, the grep
command can be used to filter the lines from the /etc/passwd file for the lines containing the
pattern bash:

sysadmin@localhost:~$ grep bash /etc/passwd

root:x:0:0:root:/root:/bin/bash
sysadmin:x:1001:1001:System Administrator,,,,:/home/sysadmin:/bin/bash

To make it easier to see what exactly is matched, use the --color option. This option will
highlight the matched items in red:
sysadmin@localhost:~$ grep --color bash /etc/passwd
root:x:0:0:root:/root:/bin/bash
sysadmin:x:1001:1001:System Administrator,,,,:/home/sysadmin:/bin/bash

Note

On our virtual machines, the grep command is aliased to include the --color option automatically.

In some cases, it may not be important to find the specific lines that match the pattern, but rather
how many lines match the pattern. The -c option provides a count of how many lines match:

sysadmin@localhost:~$ grep -c bash /etc/passwd

2

When viewing the output from the grep command, it can be hard to determine the original line
numbers. This information can be useful when going back into the file (perhaps to edit the file) to
quickly find one of the matched lines.

The -n option to the grep command will display original line numbers. To display all lines and
their line numbers in the /etc/passwd file which contain the pattern bash:

sysadmin@localhost:~$ grep -n bash /etc/passwd

1:root:x:0:0:root:/root:/bin/bash
27:sysadmin:x:1001:1001:System Administrator,,,,:/home/sysadmin:/bin/bash

The -v option inverts the match, outputting all lines that do not contain the pattern. To display all
lines not containing nologin in the /etc/passwd file:

sysadmin@localhost:~$ grep -v nologin /etc/passwd

root:x:0:0:root:/root:/bin/bash
sync:x:4:65534:sync:/bin:/bin/sync
operator:x:1000:37::/root:/bin/sh
sysadmin:x:1001:1001:System Administrator,,,,:/home/sysadmin:/bin/bash
 The -i option ignores the case (capitalization) distinctions. The following searches for the pattern
the in [Link], allowing each character to be uppercase or lowercase:

sysadmin@localhost:~$ cd Documents
sysadmin@localhost:~/Documents$ grep -i the [Link]
There are three bathrooms.
**Beware** of the ghost in the bedroom.
The kitchen is open for entertaining.
**Caution** the spirits don't like guests.

The -w option only returns lines which contain matches that form whole words. To be a word, the
character string must be preceded and followed by a non-word character. Word characters
include letters, digits, and the underscore character.

The following examples search for the are pattern in the [Link] file. The first command
searches with no options, while the second command includes the -w option. Compare the
outputs:
sysadmin@localhost:~/Documents$ grep are [Link]
There are three bathrooms.
**Beware** of the ghost in the bedroom.
sysadmin@localhost:~/Documents$ grep -w are [Link]
There are three bathrooms.

​
​

10.8 Basic Regular Expressions

Regular expressions , also referred to as regex , are a collection of
normal and special characters that are used to find simple or complex patterns, respectively,
in files. These characters are characters that are used to perform a particular matching function
in a search.

Normal characters are alphanumeric characters which match themselves. For example, an a
would match an a. Special characters have special meanings when used within patterns by
commands like the grep command. They behave in a more complex manner and do not match
themselves.

There are both Basic Regular Expressions (available to a wide variety of Linux commands) and
Extended Regular Expressions (available to more advanced Linux commands). Basic Regular
Expressions include the following:
 Char
er Matches

. Any single character

[ ] A list or range of characters to match one character

If the first character within the brackets is the caret ^, it means any
character not in the list

* The previous character repeated zero or more times

^ If the first character in the pattern, the pattern must be at the beginning of
the line to match, otherwise just a literal ^ character

$ If the last character in the pattern, the pattern must be at the end of the lin
to match, otherwise just a literal $ character

The grep command is just one of the many commands that support regular expressions. Some
other commands include the more and less commands.

While some of the regular expressions are unnecessarily quoted with single quotes, it is good
practice to use single quotes around regular expressions to prevent the shell from trying to
interpret special meaning from them.

​
​

10.8.1 The Period . Character

One of the most useful expressions is the period . character. It matches any character except for
the new line character. Consider the unfiltered contents of the ~/Documents/[Link] file:

sysadmin@localhost:~/Documents$ cat [Link]

red
reef
rot
reeed
rd
rod
roof
reed
root
reel
read

The pattern r..f would find any line that contained the letter r followed by exactly two
characters and then the letter f:

sysadmin@localhost:~/Documents$ grep 'r..f' [Link]

reef
roof

The line does not have to be an exact match, it simply must contain the pattern, as seen here
when r..t is searched for in the /etc/passwd file:

sysadmin@localhost:~/Documents$ grep 'r..t' /etc/passwd

root:x:0:0:root:/root:/bin/bash
operator:x:1000:37::/root:

The period character can be used any number of times. To find all words that have at least four
characters, the following pattern can be used:
sysadmin@localhost:~/Documents$ grep '....' [Link]
reef
reeed
roof
reed
root
reel
read

​
​

10.8.2 The Bracket [ ] Characters

When using the . character, any possible character could match it. In some cases, you want to
specify exactly which characters you want to match, such as a lowercase alphabet character or a
number character.
 The square brackets [ ] match a single character from the list or range of possible characters
contained within the brackets. For example, given the [Link] file:

sysadmin@localhost:~/Documents$ cat [Link]

Hello my name is Joe.
I am 37 years old.
3121991
My favorite food is avocados.
I have 2 dogs.
123456789101112

To find all the lines in [Link] which have a number in them, use the pattern
[0123456789] or [0-9]:

sysadmin@localhost:~/Documents$ grep '[0-9]' [Link]

I am 37 years old.
3121991
I have 2 dogs.
123456789101112

Note that each possible character can be listed out [abcd] or provided as a range [a-d], as
long as the range is in the correct order. For example, [d-a] wouldn't work because it isn't a
valid range:
sysadmin@localhost:~/Documents$ grep '[d-a]' [Link]
grep: Invalid range end

The range is specified by a standard called the ASCII table. This table is a collection of all
printable characters in a specific order. You can see the ASCII table with the ascii command. A
small sample:
041 33 21 ! 141 97 61 a
042 34 22 “ 142 98 62 b
043 35 23 # 143 99 63 c
044 36 24 $ 144 100 64 d
045 37 25 % 145 101 65 e
046 38 26 & 146 102 66 f

The ASCII value of the letter a is 97 while the value of d is 100. Since 97 is smaller than 100,
the range a-d (97-100) is a valid range.

What about exempting characters?, For instance, to match a character that can be anything
except an x, y or z? It would be inefficient to provide a set with all of the characters except x, y
or z.
 To match a character that is not one of the listed characters, start the set with a ^ symbol. To
find all the lines which contain any non-numeric characters, insert a ^ as the first character inside
the brackets. This character negates the characters listed:
sysadmin@localhost:~/Documents$ grep '[^0-9]' [Link]
Hello my name is Joe.
I am 37 years old.
My favorite food is avocados.
I have 2 dogs.

Consider This

Do not mistake [^0-9] to match lines which do not contain numbers . It actually matches lines
which contain non-numbers . Look at the original file to see the difference. The third and sixth
lines only contain numbers; they do not contain non-numbers, so those lines do not match.

​
​

10.8.3 The Asterisk * Character

The asterisk * character is used to match zero or more occurrences of a character or pattern
preceding it. For example, e* would match zero or more occurrences of the letter e:

sysadmin@localhost:~/Documents$ cat [Link]

red
reef
rot
reeed
rd
rod
roof
reed
root
reel
read
sysadmin@localhost:~/Documents$ grep 're*d' [Link]
red
reeed
rd
reed

It is also possible to match zero or more occurrences of a list of characters by utilizing the square
brackets. The pattern [oe]* used in the following example matches zero or more occurrences of
the o character or the e character:
sysadmin@localhost:~/Documents$ grep 'r[oe]*d' [Link]
red
reeed
rd
rod
reed

When used with only one other character, * isn't very helpful. Any of the following patterns would
match every string or line in the file: '.*' 'e*' 'b*' 'z*' because the asterisk * character
can match zero occurrences of a pattern.
sysadmin@localhost:~/Documents$ grep 'z*' [Link]
red
reef
rot
reeed
rd
rod
roof
reed
root
reel
read

sysadmin@localhost:~/Documents$ grep 'e*' [Link]

red
reef
rot
reeed
rd
rod
roof
reed
root
reel
read

To make the asterisk character useful, it is necessary to create a pattern which includes more
than just the one character preceding it. For example, the results above can be refined by adding
another e to make the pattern ee* effectively matching every line which contains at least one e.

sysadmin@localhost:~/Documents$ grep 'ee*' [Link]

red
reef
reeed
reed
reel
read

​
​

10.8.4 Anchor Characters

When performing a pattern match, the match could occur anywhere on the line.
Anchor characters are one of the ways regular expressions can be used to narrow down search
results. They specify whether the match occurs at the beginning of the line or the end of the line.

For example, the pattern root appears many times in the /etc/passwd file:

sysadmin@localhost:~/Documents$ grep 'root' /etc/passwd

root:x:0:0:root:/root:/bin/bash
operator:x:1000:37::/root:

The caret (circumflex) ^ character is used to ensure that a pattern appears at the beginning of
the line. For example, to find all lines in /etc/passwd that start with root use the pattern
^root. Note that ^ must be the first character in the pattern to be effective:

sysadmin@localhost:~/Documents$ grep '^root' /etc/passwd

root:x:0:0:root:/root:/bin/bash

The second anchor character $ can be used to ensure a pattern appears at the end of the line,
thereby effectively reducing the search results. To find the lines that end with an r in the
[Link] file, use the pattern r$:

sysadmin@localhost:~/Documents$ cat [Link]

A is for Animal
B is for Bear
C is for Cat
D is for Dog
E is for Elephant
F is for Flower
sysadmin@localhost:~/Documents$ grep 'r$' [Link]
B is for Bear
F is for Flower

Again, the position of this character is important. The $ must be the last character in the pattern
to be effective as an anchor.

​
​
 10.8.5 The Backslash \ Character
In some cases, you may want to match a character that happens to be a special regular
expression character. For example, consider the following:

sysadmin@localhost:~/Documents$ cat [Link]

Thanks for purchasing your new home!!

**Warning** it may be haunted.

There are three bathrooms.

**Beware** of the ghost in the bedroom.

The kitchen is open for entertaining.

**Caution** the spirits don't like guests.

Good luck!!!
sysadmin@localhost:~/Documents$ grep 're*' [Link]
Thanks for purchasing your new home!!
**Warning** it may be haunted.
There are three bathrooms.
**Beware** of the ghost in the bedroom.
The kitchen is open for entertaining.
**Caution** the spirits don't like guests.

In the output of the grep command above, the search for re* matched every line which
contained an r followed by zero or more of the letter e. To look for an actual asterisk * character,
place a backslash \ character before the asterisk * character:

sysadmin@localhost:~/Documents$ grep 're\*' [Link]

**Beware** of the ghost in the bedroom.

​
​

10.8.6 Extended Regular Expressions

The use of extended regular expressions often requires a special option be provided to the
command to recognize them. Historically, there is a command called egrep, which is similar to
grep, but can understand extended regular expressions. Now, the egrep command is
deprecated in favor of using grep with the -E option.
 The following regular expressions are considered extended:

Char
er Meaning

? Matches previous character zero or one time, so it is an optional characte

+ Matches previous character repeated one or more times

| Alternation or like a logical "or" operator

To match colo followed by zero or one u character followed by an r character:

sysadmin@localhost:~/Documents$ grep -E 'colou?r' [Link]

American English: Do you consider gray to be a color or a shade?
British English: Do you consider grey to be a colour or a shade?

To match one or more e characters:

sysadmin@localhost:~/Documents$ grep -E 'e+' [Link]

red
reef
reeed
reed
reel
read

To match either gray or grey:

sysadmin@localhost:~/Documents$ grep -E 'gray|grey' [Link]

American English: Do you consider gray to be a color or a shade?
British English: Do you consider grey to be a colour or a shade?

​
​