PYTHON FOR ENGINEERS
OEEC6205
4th Semester, B. Tech(CSE)
IGIT, Sarang
SYLLABUS
Module IV
Files, Types of Files, Creating and Reading Text Data, File
Methods to Read and Write Data, Reading and Writing
Binary Files, The Pickle Module, Reading and
Writing CSV Files.
Python os and [Link] Modules.
Regular Expression Operations, Using Special Characters,
Regular Expression Methods, Named Groups in Python
Regular Expressions, Regular Expression with glob
Module.
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
2
CSEA, IGIT, Sarang
FILE HANDLINGS
Python provides in-built functions for
creating, writing, and reading files.
There are two types of files that can be
handled in Python, text files and binary files
(written in binary language, 0s, and 1s).
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
3
CSEA, IGIT, Sarang
FILE HANDLINGS
TEXT FILES:
o Each line of text is terminated with a special character
called EOL (End of Line), which is the new line
character (‘\n’) in Python by default.
o In the case of CSV (Comma Separated Values)
Files, the EOF is a comma (,) by default.
BINARY FILES:
o There is no terminator for a line, and the data is stored
after converting it into machine-understandable
binary language, i.e., 0 and 1 format.
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
4
CSEA, IGIT, Sarang
open( ) FUNCTION
It is used to open the file either for reading or for
writing purpose.
It returns a file object and takes two arguments, the
file name and the access mode.
SYNTAX:
File_object = open("File_Name", "Access_Mode")
Example:
fobj = open('[Link]', 'w')
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
5
CSEA, IGIT, Sarang
Sl. No. ACCESS MODE DESCRIPTION
1. Read Only ( 'r' ) Open the text file for reading.
Read and Write
2. Open the text file for reading and writing.
( 'r+' )
3. Write Only ( 'w' ) Open the text file for writing.
Write and Read Open the text file for writing and reading.
4.
( 'w+' )
Append Only Open the file for appending data.
5.
( 'a' )
Append and Open the file for appending and reading data.
6.
Read ('a+')
Read Only in Binary
7. Open the file for reading in binary format.
Format ( 'rb' )
Write Only in Binary Open the file for writing in binary format.
8.
Format ( 'wb' )
Append Only in Binary
9. Open
PYTHON FOR the Byfile
ENGINEERS for appending
Mr. Bapuji Rao, in binary format.
Format ( 'ab' ) CSEA, IGIT, Sarang
6
close( ) METHOD
It closes the file.
Syntax:
file_object.close( )
Example:
fobj = open('[Link]', 'w')
[Link]( )
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
7
CSEA, IGIT, Sarang
OPENING A FILE USING with CLAUSE
The other way of opening a file is done using with
clause.
It automatically closes the file as soon as the control
goes outside of the with clause.
SYNTAX:
with open("File_Name", "Access_Mode") as File_Object:
# statement(s)
Example:
with open('[Link]', 'w') as fobj:
# statement(s)
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
8
CSEA, IGIT, Sarang
write( ) METHOD
It takes a string as an argument and writes it to the
text file.
It returns the number of characters being written in
the file.
Add a newline character ('\n') at the end of the string
to mark the end of line.
SYNTAX:
file_object.write(string)
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
9
CSEA, IGIT, Sarang
write( ) METHOD
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
10
CSEA, IGIT, Sarang
write( ) METHOD
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
11
CSEA, IGIT, Sarang
writelines( ) METHOD
It writes multiple strings in a file.
It passes an iterable object like lists, tuple, etc.
containing strings.
It does not return the number of characters written in
the file.
Each sentence should end with the new line ('\n')
character.
SYNTAX:
file_object.write(Paragraph/ Iterable)
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
12
CSEA, IGIT, Sarang
writelines( ) METHOD
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
13
CSEA, IGIT, Sarang
writelines( ) METHOD
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
14
CSEA, IGIT, Sarang
read( ) METHOD
It reads certain number of characters from file’s
current reading position.
SYNTAX:
variable_name = file_object.read(number_of_characters)
file_object.read( ) or file_object.read(any_negative_number)
reads the contents of the file and assigned to a variable.
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
15
CSEA, IGIT, Sarang
read( ) METHOD
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
16
CSEA, IGIT, Sarang
read( ) METHOD
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
17
CSEA, IGIT, Sarang
readline([n] ) METHOD
It reads one complete line from a file where each line
terminates with a newline ('\n') character.
It can also be used to read a specified number (n) of
bytes of data from a file but maximum up to the
newline character ('\n').
If no argument or a negative number is specified, it
reads a complete line and returns the string.
Syntax:
variable_name = file_object.readline( )
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
18
CSEA, IGIT, Sarang
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
19
CSEA, IGIT, Sarang
readlines( ) METHOD
It reads all the lines from the file and returns a list object.
Syntax:
list_name = file_object.readlines( )
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
20
CSEA, IGIT, Sarang
readlines( ) METHOD
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
21
CSEA, IGIT, Sarang
BINARY FILE HANDLING USING pickle MODULE
The open( ) function opens a file in text format by default.
To open files in binary format add 'b' to mode parameter.
Hence "rb" mode opens file in binary format for reading and
"wb" mode opens file in binary format for writing.
The binary file stores only objects such as list, tuple,
dictionary, etc.
The pickle module is used to store objects and read objects
in/from the file.
The dump( ) and load( ) methods used for storing objects
and reading objects in/from the file.
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
22
CSEA, IGIT, Sarang
dump( ) METHOD
It is used to dump (pickling) data in a binary file.
SYNTAX:
import pickle
[Link](data_object, file_object)
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
23
CSEA, IGIT, Sarang
load( ) METHOD
It is used to load (unpickling) data from a binary file.
SYNTAX:
import pickle
data_object = [Link](file_object)
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
24
CSEA, IGIT, Sarang
READING CSV FILES
The methods loadtxt( ) and genfromtxt( ) are used to
load data from CSV (Comma Separated Values) file
into the numpy array.
Each row in the CSV file must have the same number of
values in order to load data from a CSV file into numpy
array.
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
25
CSEA, IGIT, Sarang
loadtxt( ) METHOD
SYNTAX:
array_name = [Link] (file_name,
skiprows = number_of_rows,
delimiter = 'delimeter_character',
unpack = False, dtype = data_type)
skiprows: It skips number of rows data not to be loaded into the
array.
delimiter: It specifies whether the values are separated by
comma, semicolon, tab or space, or any other character. The
default value for delimiter is space.
dtype: It specifies the data type of the array. By default, dtype is
float.
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
26
CSEA, IGIT, Sarang
EXAMPLE
[Link]
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
27
CSEA, IGIT, Sarang
genfromtxt( ) METHOD
SYNTAX:
array_name = [Link] (file_name,
skip_header = number_of_rows,
delimiter = 'delimeter_character',
filling_values = value, dtype = data_type)
skip_header: It skips number of rows data not to be loaded into the
array.
delimiter: It specifies whether the values are separated by comma,
semicolon, tab or space, or any other character. The default value for
delimiter is space.
dtype: It specifies the data type of the array. By default, dtype is float.
If the data file contains non-numeric data and missing data, then it is
automatically converted into ‘nan’ data type.
filling_values: This attribute is used to replace the non-numeric data
and the missing data with the specified value.
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
28
CSEA, IGIT, Sarang
EXAMPLE
[Link]
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
29
CSEA, IGIT, Sarang
EXAMPLE
Missing value and non-numeric value replaces with -1
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
30
CSEA, IGIT, Sarang
WRITING CSV FILES
savetxt( ) METHOD
It is used to save a numpy array as a CSV/Text file
on the disk.
SYNTAX:
[Link](file_name, array_name, delimiter
= 'delimiter_character', fmt = '%format_code',
newline = '\n', header = ' ', footer = ' ', comments =
'# ', encoding = None)
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
31
CSEA, IGIT, Sarang
WRITING CSV FILES
array_name: 1-D or 2-D numpy array.
delimiter: String or character to be used as element separator
(Optional).
fmt: It is used to store the numpy array data in the file with the
specified format.
The available formats are '%d' (to save as integer), '%f' (to save
as float), and '%s' (to save as string).
The default format code is '%f'.
If a single formatter is specified then it will be applied to all the
elements of the array.
In case of 2-D arrays, a list of specifiers can be used i.e. different
for each column. (Optional)
newline: String or character to be used as line separator (Optional)
header: String to be written at the beginning of the txt file.
footer: String to be written at the end of the txt file.
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
32
CSEA, IGIT, Sarang
EXAMPLES
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
33
CSEA, IGIT, Sarang
EXAMPLES
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
34
CSEA, IGIT, Sarang
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
35
CSEA, IGIT, Sarang
LOAD CSV FILES WITH THE PYTHON STANDARD LIBRARY
reader( ) METHOD
The Python API provides the module CSV and the
reader( ) method is used to load CSV files.
Once loaded, you can convert the CSV data to a NumPy
array.
SYNTAX:
import csv
data_name = [Link](csv_file_name, delimiter = ',',
quoting = csv.QUOTE_NONE)
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
36
CSEA, IGIT, Sarang
EXAMPLE
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
37
CSEA, IGIT, Sarang
Writing Data in CSV and Text Format on the Disk
to_csv( ) METHOD
It writes the dataFrame data in a CSV(.csv) format on the disk.
SYNTAX:
dataFrame_data.to_csv('File_Name.csv', index=False)
index = False parameter ignores of writing the index values.
The default value for 'index' parameter is True.
to_csv( ) METHOD
It writes the dataFrame data in a Text (.txt) format on the disk.
SYNTAX:
dataFrame_data.to_csv('File_Name.txt', sep='\t', index=False)
index = False parameter ignores of writing the index values.
The default value for 'index' parameter is True.
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
38
CSEA, IGIT, Sarang
EXAMPLE
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
39
CSEA, IGIT, Sarang
os MODULE
The OS module provides methods for creating and
removing a directory (folder), renaming a
directory, retrieving directory contents, and
identifying the current directory, etc.
The various methods are:
o getcwd( )
o mkdir( )
o chdir( )
o listdir( )
o rmdir( )
o makedirs( )
o remove( )
o rename( ) PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
40
CSEA, IGIT, Sarang
getcwd( ) METHOD
It returns the current working directory.
SYNTAX:
import os
Variable_Name = [Link]( )
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
41
CSEA, IGIT, Sarang
mkdir( ) METHOD
It creates a new directory.
SYNTAX:
import os
[Link]("Directory_Name")
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
42
CSEA, IGIT, Sarang
chdir( ) METHOD
It changes to another directory.
SYNTAX:
import os
[Link]("Directory_Name")
# To get back to the parent directory
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
43
CSEA, IGIT, Sarang
listdir( ) METHOD
It returns the list of all files and directories in the specified
directory.
SYNTAX:
import os
Variable_Name = [Link]("Directory_Name")
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
44
CSEA, IGIT, Sarang
rmdir( ) METHOD
It removes the specified directory either with an
absolute or relative path.
It removes only the empty directory.
SYNTAX:
import os
Variable_Name = [Link]("Directory_Name")
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
45
CSEA, IGIT, Sarang
rmdir( ) METHOD
It removes the specified directory either with an absolute or
relative path.
It removes only the empty directory.
SYNTAX:
import os
Variable_Name = [Link]("Directory_Name")
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
46
CSEA, IGIT, Sarang
rename( ) METHOD
It renames an old file name with a new file name.
SYNTAX:
import os
[Link]("Old_File_Name", "New_File_Name")
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
47
CSEA, IGIT, Sarang
rename( ) METHOD
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
48
CSEA, IGIT, Sarang
remove( ) METHOD
It removes a file from a directory.
SYNTAX:
import os
[Link]("File_Name")
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
49
CSEA, IGIT, Sarang
[Link] MODULE
join( ) method to join a sub-directory or file with the
parent directory for creation of a path.
SYNTAX:
import os
[Link](Parent_Directory, Sub_Directory/File_Name)
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
50
CSEA, IGIT, Sarang
makedirs( ) METHOD
It combines the existing director with the parent directory.
If the directory does not exist, then it creates newly with that
name.
SYNTAX:
Path_Name = [Link](Parent_Directory, Child_Directory)
[Link](Path_Name)
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
51
CSEA, IGIT, Sarang
makedirs( ) METHOD
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
52
CSEA, IGIT, Sarang
REGULAR EXPRESSION OPERATIONS
A regular expression is a special sequence of characters that
helps you match or find other strings or sets of strings.
The re module has various methods for regular expression
operations.
The raw string (i.e. string literal) is prefixed with the
alphabet 'r' i.e. r'STRING'.
The various methods are: search() [It has four methods:
start(), end(), span(), group()], match(), findall(),
compile(), split(), sub(), subn(), escape().
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
53
CSEA, IGIT, Sarang
search( ) METHOD
It searches for first occurrence of pattern with in a string.
It searches for a match anywhere in the string.
It returns a search object on success, none on failure.
SYNTAX:
import re
[Link](Pattern, String, flags=0)
• Pattern: Regular expression to be matched.
• String: The Pattern to be searched from anywhere in the
String.
• flags: It is an optional. It specifies different flags using
bitwise OR ( | ). These are called modifiers.
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
54
CSEA, IGIT, Sarang
start( ), end( ), span( ) METHODS
start( ): It returns the starting index of the first occurrence of
the pattern in the string.
SYNTAX:
Variable = Search_Object.start( )
end( ): It returns the ending index of the first occurrence of
the pattern in the string.
SYNTAX:
Variable = Search_Object.end( )
span( ): It returns the starting and ending index as a tuple of
the first occurrence of the pattern in the string.
SYNTAX:
Tuple_Variable = Search_Object.span( )
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
55
CSEA, IGIT, Sarang
EXAMPLE-1
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
56
CSEA, IGIT, Sarang
EXAMPLE-2
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
57
CSEA, IGIT, Sarang
findall( ) METHOD
It finds all the patterns present in the string.
SYNTAX-1:
import re
List_Variable = [Link](Pattern, String)
SYNTAX-2:
import re
Compile_Object = [Link](Pattern)
List_Variable = Compile_Object.findall(String)
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
58
CSEA, IGIT, Sarang
EXAMPLE-1
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
59
CSEA, IGIT, Sarang
EXAMPLE-2
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
60
CSEA, IGIT, Sarang
EXAMPLE-3
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
61
CSEA, IGIT, Sarang
EXAMPLE-3
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
62
CSEA, IGIT, Sarang
EXAMPLE-4
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
63
CSEA, IGIT, Sarang
EXAMPLE-4
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
64
CSEA, IGIT, Sarang
group( ) METHOD
group( ) or group(0): It prints the full match.
group(1): It prints the first match.
group(2): It prints the second match.
SYNTAX:
import re
Search_Object = [Link](Pattern, String)
Variable = Search_Object.group( )
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
65
CSEA, IGIT, Sarang
EXAMPLE-1
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
66
CSEA, IGIT, Sarang
EXAMPLE-2
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
67
CSEA, IGIT, Sarang
EXAMPLE-3
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
68
CSEA, IGIT, Sarang
EXAMPLE-3
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
69
CSEA, IGIT, Sarang
match( ) METHOD
It checks for a match only at the beginning of the string.
It returns a match object on success, none on failure.
SYNTAX:
import re
[Link](Pattern, String, flags=0)
• Pattern: Regular expression to be matched.
• String: The Pattern to be searched from anywhere in the
String.
• flags: It is an optional. It specifies different flags using
bitwise OR ( | ). These are called modifiers.
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
70
CSEA, IGIT, Sarang
EXAMPLE
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
71
CSEA, IGIT, Sarang
EXAMPLE
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
72
CSEA, IGIT, Sarang
sub( ) [substitute] METHOD
It detects a pattern in the given string and replaced by the
replacement_string in the string.
The count argument for number of times of detected pattern to
be replaced by the replacement_string.
By default it replaces all the detected pattern with the
replacement_string.
It returns the changed string as result.
SYNTAX:
import re
Variable = [Link] (Pattern, Replacement_String, String,
count=0, flags=0)
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
73
CSEA, IGIT, Sarang
EXAMPLE-1
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
74
CSEA, IGIT, Sarang
EXAMPLE-1
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
75
CSEA, IGIT, Sarang
EXAMPLE-2
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
76
CSEA, IGIT, Sarang
EXAMPLE-2
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
77
CSEA, IGIT, Sarang
subn( ) METHOD
It detects a Pattern in the given string and replaced by the
Replacement_String in the string.
The count argument for number of times of detected Pattern to
be replaced by the Replacement_String.
By default it replaces all the detected pattern with the
Replacement_String.
It returns the tuple having the changed string and the total
number of replacements as result.
SYNTAX:
import re
Tuple_Variable = [Link] (Pattern, Replacement_String,
String, count=0, flags=0)
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
78
CSEA, IGIT, Sarang
EXAMPLE
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
79
CSEA, IGIT, Sarang
EXAMPLE
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
80
CSEA, IGIT, Sarang
split( ) METHOD
It ignores the detected patterns and splits the non-
detected patterns from the string.
SYNTAX:
import re
List_Variable = [Link] (Pattern, String, maxsplit=0,
flags=0)
maxsplit=0: By default it splits all. Otherwise it splits
the non-zero value times.
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
81
CSEA, IGIT, Sarang
EXAMPLE-1
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
82
CSEA, IGIT, Sarang
EXAMPLE-1
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
83
CSEA, IGIT, Sarang
EXAMPLE-2
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
84
CSEA, IGIT, Sarang
EXAMPLE-2
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
85
CSEA, IGIT, Sarang
escape( ) METHOD
It ignores escape sequence characters (\t, \n, \", \', \b)
and replaces with \spaces.
The remaining characters adds a \(backslash) character
at the beginning with all the return strings.
SYNTAX:
import re
List_Variable = [Link](String)
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
86
CSEA, IGIT, Sarang
EXAMPLE
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
87
CSEA, IGIT, Sarang
EXAMPLE
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
88
CSEA, IGIT, Sarang
glob MODULE
The glob module is used to retrieve files/pathnames matching a
specified pattern.
The pattern may be wildcards (*, ?, [ranges]) or exact string
search to make path retrieval more simple and convenient.
glob Module has two methods:
• glob( ) Method
• iglob( ) Method
SYNTAX
import glob
(i) List_Name = [Link](Pathname, recursive=False)
(ii) for File_Name in [Link](Pathname, recursive=False):
print(File_Name)
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
89
CSEA, IGIT, Sarang
EXAMPLE
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
90
CSEA, IGIT, Sarang
EXAMPLE
PYTHON FOR ENGINEERS By Mr. Bapuji Rao,
91
CSEA, IGIT, Sarang