Unix Coding
Used for Printing Statement:
echo "Hello World!!"
cat is a standard Unix utility that reads files sequentially, writing them to
standard output.
cat [Link]
To print complete data of file [Link].
awk '{}' [Link]
To print the first column data of file [Link].
awk '{print $1}' [Link]
To print the second column data of file [Link].
awk '{print $2}' [Link]
To print complete data of file [Link].
awk '{print $0}' [Link]
To print the more than one column data of file [Link].
awk '{print $1, $2}' [Link]
awk '{print $4, $2}' [Link]
To print the last column data of file [Link].
awk '{print $NF}' [Link]
To print the search a word data of file [Link].
awk '/Paul/{print $0}' [Link]
To print the line no.s of all data in a file [Link].
awk '{print NR,$0}' [Link]
To print the line no. of search a word data of file [Link].
awk '/Paul/{print NR,$0}' [Link]
To print only a given line no. of data in a file [Link].
awk 'NR==6{print $0}' [Link]
To print a row or only a given line no. of data in a file [Link] at start
of each line.
awk 'NR==6{print NR,$0}' [Link]
To print range of line of data in a file [Link].
awk 'NR==3, NR==6{print NR,$0}' [Link]
To print line no. of empty lines in a file [Link].
awk 'NF==0{print NR}' [Link]
Search Multiple Words: (Case Sensitive, like uppar lower cases gives diff
result)
Awk '/Rahul|Yash/{print $0}' [Link]
Awk '/rahul|yash/{print $0}' [Link]
Ignore Case while searching:
Awk 'BEGIN{IGNORECASE=1}/raju/{print $0}' [Link]
Awk 'BEGIN{IGNORECASE=1}/Raju/{print $0}' [Link]
Awk 'BEGIN{IGNORECASE=1}/RAJU/{print $0}' [Link]
How to check if a given character is present in column or not:
Awk '$2 ~ /a/ {print $0}' [Link]
Print a values of file if contents are separated with delimiter:
Awk -F, '{print $1}' [Link]
How to work with csv files:
Print a values of file of 2nd column:
Awk -F, '{print $2 }' [Link]
Print a data of employees whose salary is >50k:
Awk -F, '$NF>50000{print $0 }' [Link]
What if file is having a multiple Delimiter (like , - : #):
Awk -F[,:-] '{print $2}' [Link]
Awk -F[,:] '{print $4}' [Link]
To get list of files:
ls -lt
To print the list file details of last column ignoring first row bcozz it shows
total.
ls -lt | awk 'NR>1{print $NF}'
or
ls -1
How to replace a word.
Awk '{gsub(“Raju”,”RAJA”)}' [Link]
Awk '{gsub(“Raju”,”RAJA”); {print $0}}' [Link]
How to find length of a line.
Awk '{print length($2)}' [Link]
Awk '{print $2,length($2)}' [Link]
Index position of a word in given line.
Awk '/Paul/{print NR, index($0,”Paul”)}' [Link]
Print values in upper and lower case.
Awk '{print tolower($2)}' [Link]
Awk '{print toupper($2)}' [Link]
Print a values:
awk -F, 'BEGIN{print "Start"} {print $0} END{print "End"}'
Input:
1234,Ishita,MCA,Alld
1235,Kituu,Btech,Alld
1236,Rahul,MA,Alld
Output:
Start
1234,Ishita,MCA,Alld
1235,Kituu,Btech,Alld
1236,Rahul,MA,Alld
End
How to find total/sum of salary:
awk -F, 'BEGIN{s=0} {s+=$5} END{print "Sum of Salary:",s}'
awk -F, 'BEGIN{s=0} {s=s+$NF} END{print "Sum of Salary:",s}'
awk -F, '{s+=$NF} END{print "Sum of Salary:",s}'
Input:
Id,Name,Subject,City,Salary
1234,Ishita,MCA,Alld,25000
1235,Kituu,Btech,Alld,35000
1236,Rahul,MA,Alld,55000
Output:
Sum of Salary: 115000
How to find Avg of salary:
If we want to ignore header line:
awk -F, 'BEGIN {c=0; s=0} NR>1{c+=1; s+=$5;} END{print "Average Salary:",(s/c)}'
If there is a space in your file in the last line:
awk -F, 'BEGIN {c=0; s=0} NR>1{if($NF>0){c+=1; s+=$5;}} END{print "Average
Salary:",(s/c)}'
How to print number of lines in your file or data:
awk -F, '{} END{print "Number of Lines:",NR }'
If we want to ignore header line:
awk -F, 'BEGIN{c=0} NR>1{c++;} END{print "Number of Lines:",c}'
If there is a space in your file in the last line:
awk -F, 'BEGIN{c=0} NR>1{if($NF>0){c++;}} END{print "Number of Lines:",c}'
How to get the length of all lines:
awk -F, '{print length($0)}'
How to get the length of longest line:
awk -F, 'NR>1{if(length($0)>max){max=length($0)}} END{print "Length of
longest line:"max}'
Print high if salary is >30000 else Low:
awk -F, 'NR>1{if($5>30000){
print $0,"High";
}else{
print $0,"Low";
}}'
Or
awk -F, 'NR>1{if($NF>30000){
$6="High";
}else{
$6="Low";
} print $0}'
Print total salary of MCA Students:
awk -F, 'BEGIN{s=0} NR>1{if($3=="MCA"){s+=$NF;}} END{print "Total Salary of
MCA Students:"s}'
awk -F, 'BEGIN{s=0} NR>1{if($3=="MCA"){s+=$5;}} END{print "Total Salary of
MCA Students:"s}'
Using Awk patterns in a files:
Create awk file:
vi [Link]
then Enter, awk file will open
if($NF>30000)
$6="High";
else
$6="Low";
print $0
then save this file with shift+colon+w+q
then goes to bash area and write
to read
cat [Link]
to run
awk -f [Link]
or
awk -f [Link] [Link]
We can also add BEGIN and END statements inside awk file
BEGIN{
Print(“Start”)
}
{
if($NF>30000)
$6="High";
else
$6="Low";
print $0
}
END{
Print(“END”)
}
Create awk file:
vi awk_demo.awk
then Enter, awk file will open
#!/bin/awk -f
#comment
Begin{
Print “Hello World!!”
}
Try to give permission to file, like give all permission
Chmod 777 awk_demo.awk
To print the script
./awk_demo.awk
It will show
Hello World!!
Using Arrays:
awk 'BEGIN{
name="Ishita";
age=24;
Company="TCS";
arr[1]=10;
arr[2]=11;
arr[3]=12;
arr[4]=15;
}{
} END{
print name,age,company,arr[1],arr[2]
}'
Using Arrays with key value pairs:
awk 'BEGIN{
name="Ishita";
age=24;
Company="TCS";
arr[1]=10;
arr[2]=11;
arr[3]=12;
arr[4]=15;
marks["C"]=89;
marks["Java"]=90;
marks["C++"]=86;
}{
} END{
print name,age,company,arr[1],arr[2],marks["Java"]
}'
Using Arrays with For Loops:
awk 'BEGIN{
name="Ishita";
age=24;
Company="TCS";
arr[1]=10;
arr[2]=11;
arr[3]=12;
arr[4]=15;
marks["C"]=89;
marks["Java"]=90;
marks["C++"]=86;
for(i in arr){
print arr[i]}
}{
} END{
print name,age,company,arr[1],arr[2],marks["Java"]
}'
12
11
10
15
Ishita 24 10 11 90
Alert: But we have to give some values as input during gdb bash input or give
some file name.
awk 'BEGIN{
name="Ishita";
age=24;
Company="TCS";
arr[1]=10;
arr[2]=11;
arr[3]=12;
arr[4]=15;
marks["C"]=89;
marks["Java"]=90;
marks["C++"]=86;
for(i in arr){
print "Array "i,arr[i]}
for(subj in marks){
print "Marks in " subj, marks[subj]
}
}{
} END{
print name,age,company,arr[1],arr[2],marks["Java"]
}'
Array 3 12
Array 2 11
Array 1 10
Array 4 15
Marks in Java 90
Marks in C++ 86
Marks in C 89
Ishita 24 10 11 90
We can also use For, While loop here.
Examples:
Input Data:
customer_id,first_name,last_name,date_of_opening,Account_Type,Balance
180607, Rahul, Varma,12.12.22,Savings,30000
180608, Sita, Raman,12.12.22,Joint,50000
190607, Sheetal, Patil,20.07.22,Current,40000
190617, Pooja, Patil,04.08.22,Salary,70000
200607, Rahul, Sharma,20.08.22,SAVINGS,20000
210617, Pooja, Srikari,20.08.22,SALARY,25000
210607, Pooja, Rewa,20.08.22,joint,35000
210527, Dan, Stewart,04.08.22,current,35000
280707, Sia, R,21.09.23,Savings,30000
220888, Sonali, G,1.05.23,Savings,15000
User Story 1:
UNIX command to display the customers having balance greater than or
equal to 30000.
Sol1:
awk 'BEGIN {FS=","}{
if($5>=30000){
print $1","$2","$3","$4","$5
}
}
END{}'
Or
awk -F, '{if($6>=30000){
print $0
}}'
Output:
customer_id,first_name,last_name,date_of_opening,Account_Type,Balance
180607, Rahul, Varma,12.12.22,Savings,30000
180608, Sita, Raman,12.12.22,Joint,50000
190607, Sheetal, Patil,20.07.22,Current,40000
190617, Pooja, Patil,04.08.22,Salary,70000
210607, Pooja, Rewa,20.08.22,joint,35000
210527, Dan, Stewart,04.08.22,current,35000
280707, Sia, R,21.09.23,Savings,30000
Or
awk -F, 'NR>1{if($6>=30000){
print $0
}}'
180607, Rahul, Varma,12.12.22,Savings,30000
180608, Sita, Raman,12.12.22,Joint,50000
190607, Sheetal, Patil,20.07.22,Current,40000
190617, Pooja, Patil,04.08.22,Salary,70000
210607, Pooja, Rewa,20.08.22,joint,35000
210527, Dan, Stewart,04.08.22,current,35000
280707, Sia, R,21.09.23,Savings,30000
User Story2: UNIX command to display the total balance of customers having
Joint account.
Sol2:
awk 'BEGIN {FS=",";s=0}{
if(tolower($4) == "joint"){
s+=$5;
}
}
END{print s}'
Output:
85000
Or
awk -F, 'BEGIN{sum=0;} {if(tolower($5)=="joint"){sum+=$6;}} END{print "Total
balance:"sum}'
Total balance:85000
User Story3: UNIX command to search for a customers having account
number 222635 and 180607.
Sol3:
awk 'BEGIN {FS=","}{
if($1==222635 || $1==180607){
print $1","$2","$3","$4","$5
}
}
END{}'
or
awk -F, '{if($1==222635 || $1==180607){print $0}}'
Output:
180607, Rahul Varma,12.12.22,Savings,30000
User Story4: UNIX command to display the average balance of all customers
except those customers having Joint account.
Sol4:
awk 'BEGIN {FS=","; c=0; s=0}{
if((tolower($4)!="joint")){
s+=$5;
c+=1;
}
}
END{print (s/c)}'
Output:
33125
awk -F, 'BEGIN{sum=0;c=0} NR>1{if(tolower($5)!="joint"){sum+=$6; c+=1;}}
END{print "AVG balance:",(sum/c)}'
AVG balance: 33125
Examples:
Input Data:
customer_id,first_name,last_name,date_of_opening,Account_Type,Balance
180607, Rahul, Varma,12.12.22,Savings,30000
180608, Sita, Raman,12.12.22,Joint,50000
190607, Sheetal, Patil,20.07.22,Current,40000
190617, Pooja, Patil,04.08.22,Salary,70000
200607, Rahul, Sharma,20.08.22,SAVINGS,20000
210617, Pooja, Srikari,20.08.22,SALARY,25000
210607, Pooja, Rewa,20.08.22,joint,35000
210527, Dan, Stewart,04.08.22,current,35000
280707, Sia, R,21.09.23,Savings,30000
220888, Sonali, G,1.05.23,Savings,15000
# 1. Sort by Customer ID (ascending order)
echo "Sort by Customer ID (ascending):"
awk -F, 'BEGIN { OFS="," } NR>1 { print $1,$2,$3,$4,$5,$6 } END{}' | sort -t, -
k1,1n
echo "--------------------------------"
Sort by Customer ID (ascending):
180607, Rahul, Varma,12.12.22,Savings,30000
180608, Sita, Raman,12.12.22,Joint,50000
190607, Sheetal, Patil,20.07.22,Current,40000
190617, Pooja, Patil,04.08.22,Salary,70000
200607, Rahul, Sharma,20.08.22,SAVINGS,20000
210527, Dan, Stewart,04.08.22,current,35000
210607, Pooja, Rewa,20.08.22,joint,35000
210617, Pooja, Srikari,20.08.22,SALARY,25000
220888, Sonali, G,1.05.23,Savings,15000
280707, Sia, R,21.09.23,Savings,30000
--------------------------------
# 2. Sort by First Name (descending order)
echo "Sort by First Name (descending):"
awk -F, 'BEGIN { OFS="," } NR>1 { print $1,$2,$3,$4,$5,$6 } END{}'| sort -t, -
k2,2r
echo "--------------------------------"
Sort by First Name (descending):
220888, Sonali, G,1.05.23,Savings,15000
180608, Sita, Raman,12.12.22,Joint,50000
280707, Sia, R,21.09.23,Savings,30000
190607, Sheetal, Patil,20.07.22,Current,40000
180607, Rahul, Varma,12.12.22,Savings,30000
200607, Rahul, Sharma,20.08.22,SAVINGS,20000
190617, Pooja, Patil,04.08.22,Salary,70000
210607, Pooja, Rewa,20.08.22,joint,35000
210617, Pooja, Srikari,20.08.22,SALARY,25000
210527, Dan, Stewart,04.08.22,current,35000
--------------------------------
# 3. Sort by Last Name (ascending order)
echo "Sort by Last Name (ascending):"
awk -F, 'BEGIN { OFS="," } NR>1 { print $1,$2,$3,$4,$5,$6 } END{}'| sort -t, -k3,3
echo "--------------------------------"
Sort by Last Name (ascending):
220888, Sonali, G,1.05.23,Savings,15000
190607, Sheetal, Patil,20.07.22,Current,40000
190617, Pooja, Patil,04.08.22,Salary,70000
280707, Sia, R,21.09.23,Savings,30000
180608, Sita, Raman,12.12.22,Joint,50000
210607, Pooja, Rewa,20.08.22,joint,35000
200607, Rahul, Sharma,20.08.22,SAVINGS,20000
210617, Pooja, Srikari,20.08.22,SALARY,25000
210527, Dan, Stewart,04.08.22,current,35000
180607, Rahul, Varma,12.12.22,Savings,30000
--------------------------------
# 4. Reverse the data (excluding header)
echo "Reversed Data (excluding header):"
awk -F, 'BEGIN{} NR>1{ print $0 } END{}'| tac
echo "--------------------------------"
Reversed Data (excluding header):
220888, Sonali, G,1.05.23,Savings,15000
280707, Sia, R,21.09.23,Savings,30000
210527, Dan, Stewart,04.08.22,current,35000
210607, Pooja, Rewa,20.08.22,joint,35000
210617, Pooja, Srikari,20.08.22,SALARY,25000
200607, Rahul, Sharma,20.08.22,SAVINGS,20000
190617, Pooja, Patil,04.08.22,Salary,70000
190607, Sheetal, Patil,20.07.22,Current,40000
180608, Sita, Raman,12.12.22,Joint,50000
180607, Rahul, Varma,12.12.22,Savings,30000
--------------------------------
# 5. Sort by Balance (ascending order)
echo "Sort by Balance (ascending):"
awk -F, 'BEGIN{} NR>1{ print $1","$2","$3","$4","$5","$6 } END{}'| sort -t, -
k6,6n
echo "--------------------------------"
Sort by Balance (ascending):
220888, Sonali, G,1.05.23,Savings,15000
200607, Rahul, Sharma,20.08.22,SAVINGS,20000
210617, Pooja, Srikari,20.08.22,SALARY,25000
180607, Rahul, Varma,12.12.22,Savings,30000
280707, Sia, R,21.09.23,Savings,30000
210527, Dan, Stewart,04.08.22,current,35000
210607, Pooja, Rewa,20.08.22,joint,35000
190607, Sheetal, Patil,20.07.22,Current,40000
180608, Sita, Raman,12.12.22,Joint,50000
190617, Pooja, Patil,04.08.22,Salary,70000
--------------------------------
# 6. Sort by Balance (descending order)
echo "Sort by Balance (descending):"
awk -F, 'BEGIN{} NR>1{ print $1","$2","$3","$4","$5","$6 } END{}'| sort -t, -
k6,6nr
echo "--------------------------------"
Sort by Balance (descending):
190617, Pooja, Patil,04.08.22,Salary,70000
180608, Sita, Raman,12.12.22,Joint,50000
190607, Sheetal, Patil,20.07.22,Current,40000
210527, Dan, Stewart,04.08.22,current,35000
210607, Pooja, Rewa,20.08.22,joint,35000
180607, Rahul, Varma,12.12.22,Savings,30000
280707, Sia, R,21.09.23,Savings,30000
210617, Pooja, Srikari,20.08.22,SALARY,25000
200607, Rahul, Sharma,20.08.22,SAVINGS,20000
220888, Sonali, G,1.05.23,Savings,15000
--------------------------------
Explanation
• AWK Syntax:
Each AWK command uses -F, to set the comma as the field separator. The BEGIN
block can be used for initializing (like setting the output field separator, OFS), the
NR>1 block skips the header, and the END block is empty in these examples.
• Sorting with sort:
o sort -t, -k1,1n sorts numerically on field 1 (Customer ID).
o sort -t, -k2,2r sorts on field 2 (First Name) in reverse order.
o sort -t, -k3,3 sorts on field 3 (Last Name) in ascending order.
o sort -t, -k6,6n and sort -t, -k6,6nr sort numerically on field 6
(Balance) in ascending and descending order, respectively.
• Reversing Data:
The tac command (which is like cat but in reverse) reverses the order of lines
coming from AWK (after skipping the header).
• Pipes:
Each AWK command is piped (|) to a sort or tac command, demonstrating how to
combine multiple commands in UNIX.