Unit-4: Python Interaction with Text and CSV
UNIT-4 Python Interaction with text and SCV file
File handling (text and CSV file) using CSV module:
What is CSV File?
A CSV file is a simple type of plain text file which uses a specific structure to
arrange tabular data. The standard format of a CSV file is defined by rows and columns
data where a newline terminates each row to begin the next row, and each column is
separated by a comma within the row.
Data in the form of tables is also called CSV (comma separated values) - literally
"comma-separated values." This is a text format intended for the presentation of
tabular data. Each line of the file is one line of the table. The values of individual
columns are separated by a separator symbol - a comma (,), a semicolon (;) or another
symbol.
Consider the following table,
You can represent this table in csv as below.
BY: Heta S. Desai Shri [Link] College of CS & BM Page 1
Unit-4: Python Interaction with Text and CSV
As you can see each row is a new line, and each column is separated with a comma. This
is an example of how a CSV file looks like.
Python CSV Module:
Python provides a CSV module to handle CSV files. To read/write data, you need to loop
through rows of the CSV. You need to use the split method to get data from specified
columns.
The csv module gives the Python programmer the ability to parse CSV (Comma
Separated Values) files. A CSV file is a human readable text file where each line has a
number of fields, separated by commas or some other delimiter. You can think of each
line as a row and each field as a column. The CSV format has no standard, but they are
similar enough that the csv module will be able to read the vast majority of CSV files.
You can also write CSV files using the csv module.
File Modes:
Read a CSV Files:
Step-1: To read data from CSV files, you must use the reader function to generate a
reader object.
The reader function is developed to take each row of the file and make a list of all
columns. Then, you have to choose the column you want the variable data for.
Open() function:
The open() function opens a file, and returns it as a file object.
Syntax:
BY: Heta S. Desai Shri [Link] College of CS & BM Page 2
Unit-4: Python Interaction with Text and CSV
Open(“File name”, “mode”)
Parameter Values:
Reader() function:
The reader() is used to read the file, which returns an iterable reader object. The reader
object is then iterated using a for loop to print the contents of each row.
Syntax:
reader(“File name”)
Example: (Read_csv.py)
#import necessary modules
import csv
file=open('C:\sqlite\[Link]','r')
BY: Heta S. Desai Shri [Link] College of CS & BM Page 3
Unit-4: Python Interaction with Text and CSV
data = [Link](file)
next(data)
for row in data:
print(row)
[Link]()
Step-2: When you execute the program above, the output will be:
Write Into CSV File:
BY: Heta S. Desai Shri [Link] College of CS & BM Page 4
Unit-4: Python Interaction with Text and CSV
[Link] class is used to insert data to the CSV file. This class returns a writer object
which is responsible for converting the user’s data into a delimited string.
A csvfile object should be opened with newline='' otherwise newline characters inside
the quoted fields will not be interpreted correctly.
To write to an existing file, you must add a parameter to the open() function:
"a" - Append - will append to the end of the file
"w" - Write - will overwrite any existing content
[Link] class provides two methods for writing to CSV.
They are:
1) writerow() and
2) writerows().
1) Writerow():
This method writes a single row at a time. Field row can be written using
this method.
Syntax:
Writerow(fields)
2) Writerows():
This method is used to write multiple rows at a time. This can be used to
write rows list.
Syntax:
Writerows(rows)
Example: (csv_write.py)
BY: Heta S. Desai Shri [Link] College of CS & BM Page 5
Unit-4: Python Interaction with Text and CSV
Python Dictionary:
Dictionary in Python is an ordered collection of data values, used to store data
values like a map, which, unlike other Data Types that hold only a single value as an
element, Dictionary holds key:value pair.
Key-value is provided in the dictionary to make it more optimized. Dictionary holds a
pair of values, one being the Key and the other corresponding pair element being its
Key:value. Values in a dictionary can be of any datatype and can be duplicated, whereas
keys can’t be repeated.
Write into CSV file using [Link] class:
This class returns a writer object which maps dictionaries onto output rows.
Syntax:
[Link](csvfile, fieldnames)
BY: Heta S. Desai Shri [Link] College of CS & BM Page 6
Unit-4: Python Interaction with Text and CSV
Here,
• csvfile: A file object with write() method.
• fieldnames: A sequence of keys that identify the order in which
values in the dictionary should be passed.
[Link] provides two methods for writing to CSV.
They are:
1) writeheader():
writeheader() method simply writes the first row of your csv file using the
pre-specified fieldnames.
Syntax:
Writeheader()
2) writerows():
writerows method simply writes all the rows but in each row, it writes only
the values(not keys).
Syntax:
writerows(mydict)
Example: (Write_dict.py)
BY: Heta S. Desai Shri [Link] College of CS & BM Page 7
Unit-4: Python Interaction with Text and CSV
Example: (Write_dict1.py)
Header name (Column name ) taken from Dictionary Key.
BY: Heta S. Desai Shri [Link] College of CS & BM Page 8
Unit-4: Python Interaction with Text and CSV
Reading from CSV file using [Link] class:
Create an object that operates like a regular reader but maps the information in each
row to a dict whose keys are given by the optional fieldnames parameter.
Syntax:
[Link](csvfile, fieldnames)
Example: (Read_dict.py)
BY: Heta S. Desai Shri [Link] College of CS & BM Page 9
Unit-4: Python Interaction with Text and CSV
Append data into csv file.
Example: (append_dict.py)
BY: Heta S. Desai Shri [Link] College of CS & BM Page 10
Unit-4: Python Interaction with Text and CSV
How to Install pandas, Numpy and matplotlib packages in python?
Step-1: First, make sure pip has been installed on your OS. If it is not installed,
then refer following command.
You can download file using [Link].
Step-2: Run pip install command to install related packages
pip install numpy
pip install pandas
pip install matplotlib
BY: Heta S. Desai Shri [Link] College of CS & BM Page 11
Unit-4: Python Interaction with Text and CSV
BY: Heta S. Desai Shri [Link] College of CS & BM Page 12
Unit-4: Python Interaction with Text and CSV
Numpy:
NumPy is a Python library used for working with arrays. In Python we have
lists that serve the purpose of arrays, but they are slow to process. NumPy aims to
provide an array object that is up to 50x faster than traditional Python lists. The
array object in NumPy is called ndarray, it provides a lot of supporting functions
that make working with ndarray very easy.
NumPy is used to work with arrays. The array object in NumPy is called ndarray.
We can create a NumPy ndarray object by using the array() function.
Importing numpy:
Once NumPy is installed, import it in your applications by adding the import
keyword:
Import numpy
Example: ([Link])
import numpy # import numpy library
a=[Link]([10,20,30,40]) # creating array under numpy using array()
print(a)
BY: Heta S. Desai Shri [Link] College of CS & BM Page 13
Unit-4: Python Interaction with Text and CSV
NumPy as np
NumPy is usually imported under the np alias.
Create an alias with the as keyword while importing:
Syntax:
import numpy as np
Now the NumPy package can be referred to as np instead of numpy.
Example:
import numpy as np
arr = [Link]([1, 2, 3, 4, 5])
print(arr)
BY: Heta S. Desai Shri [Link] College of CS & BM Page 14
Unit-4: Python Interaction with Text and CSV
Creating 2-D array in NumPy:
Example: ([Link])
import numpy as np
arr = [Link]([[1, 2, 3], [4, 5, 6]])
print(arr)
BY: Heta S. Desai Shri [Link] College of CS & BM Page 15
Unit-4: Python Interaction with Text and CSV
Check the number of dimension:
NumPy Arrays provides the ndim attribute that returns an integer that tells us
how many dimensions the array have.
Example: ([Link])
import numpy as np
a = [Link](42)
b = [Link]([1, 2, 3, 4, 5])
c = [Link]([[1, 2, 3], [4, 5, 6]])
print([Link])
print([Link])
print([Link])
BY: Heta S. Desai Shri [Link] College of CS & BM Page 16
Unit-4: Python Interaction with Text and CSV
Pandas in Python
Pandas is a Python library used for working with data sets.
It has functions for analyzing, cleaning, exploring, and manipulating data.
pandas is a Python package providing fast, flexible, and expressive data structures
designed to make working with “relational” or “labeled” data both easy and intuitive.
It aims to be the fundamental high-level building block for doing practical, real-world
data analysis in Python.
What Can Pandas Do?
Pandas gives you answers about the data. Like:
1) Is there a correlation between two or more columns?
2) What is average value?
3) Max value?
BY: Heta S. Desai Shri [Link] College of CS & BM Page 17
Unit-4: Python Interaction with Text and CSV
4) Min value?
Import Pandas
Once Pandas is installed, import it in your applications by adding the import keyword:
Syntax:
Import pandas
Pandas as pd
Pandas is usually imported under the pd alias.
Create an alias with the as keyword while importing:
import pandas as pd
Now the Pandas package can be referred to as pd instead of pandas.
DataFrames:
Pandas DataFrame is two-dimensional size tabular data structure with labeled axes
(rows and columns).
A Data frame is a two-dimensional data structure, i.e., data is aligned in a tabular
fashion in rows and columns.
Pandas DataFrame consists of three principal components, the data, rows, and columns.
BY: Heta S. Desai Shri [Link] College of CS & BM Page 18
Unit-4: Python Interaction with Text and CSV
Creating a dataframe
Example: ([Link])
# import pandas as pd
import pandas as pd
# list of strings
lst = ['Nidhi', 'Ankita', 'Minruhi', 'Khushi','Nisha']
# Calling DataFrame constructor on list
df = [Link](lst)
print(df)
BY: Heta S. Desai Shri [Link] College of CS & BM Page 19
Unit-4: Python Interaction with Text and CSV
Pandas Read CSV File
A simple way to store big data sets is to use CSV files (comma separated files).
CSV files contains plain text and is a well know format that can be read by everyone
including Pandas.
Load the CSV into DataFrame:
Example: ([Link])
import pandas as pd
df = pd.read_csv('c:\sqlite\[Link]')
print(df)
BY: Heta S. Desai Shri [Link] College of CS & BM Page 20
Unit-4: Python Interaction with Text and CSV
Here you can see that some of the records from top and some of the records from
bottom will be printed in dataframe.
If yoy want to print all the records from CSV file then use to_string().
use to_string() to print the entire DataFrame.
Syntax:
DataFrame.to_string()
BY: Heta S. Desai Shri [Link] College of CS & BM Page 21
Unit-4: Python Interaction with Text and CSV
Write a Dataframe into CSV File:
Example: ([Link])
import pandas as pd
# list of name, degree, score
nme = ["Kinjal", "Bhumi", "Archana", "Pinky"]
deg = ["MBA", "BCA", "[Link]", "MBA"]
scr = [90, 40, 80, 98]
BY: Heta S. Desai Shri [Link] College of CS & BM Page 22
Unit-4: Python Interaction with Text and CSV
# dictionary of lists
dict = {'name': nme, 'degree': deg, 'score': scr}
df = [Link](dict)
# saving the dataframe
df.to_csv('[Link]')
Write DataFrame into CSV file without Header and Index:
Example: ([Link])
import pandas as pd
BY: Heta S. Desai Shri [Link] College of CS & BM Page 23
Unit-4: Python Interaction with Text and CSV
# list of name, degree, score
nme = ["Kinjal", "Bhumi", "Archana", "Pinky"]
deg = ["MBA", "BCA", "[Link]", "MBA"]
scr = [90, 40, 80, 98]
# dictionary of lists
dict = {'name': nme, 'degree': deg, 'score': scr}
df = [Link](dict)
# saving the dataframe
df.to_csv('[Link]',header=False, index=False)
BY: Heta S. Desai Shri [Link] College of CS & BM Page 24
Unit-4: Python Interaction with Text and CSV
How to change row Index number as per Choice:
For this purpose Use range function
Syntax:
range(start_value,End_value)
Example: ([Link])
import pandas as pd
# list of name, degree, score
nme = ["Kinjal", "Bhumi", "Archana", "Pinky"]
BY: Heta S. Desai Shri [Link] College of CS & BM Page 25
Unit-4: Python Interaction with Text and CSV
deg = ["MBA", "BCA", "[Link]", "MBA"]
scr = [90, 40, 80, 98]
# dictionary of lists
dict = {'name': nme, 'degree': deg, 'score': scr}
df = [Link](dict,index=range(1,5))
# saving the dataframe
df.to_csv('[Link]')
Example,
BY: Heta S. Desai Shri [Link] College of CS & BM Page 26
Unit-4: Python Interaction with Text and CSV
import pandas as pd
# list of name, degree, score
nme = ["Kinjal", "Bhumi", "Archana", "Pinky"]
deg = ["MBA", "BCA", "[Link]", "MBA"]
scr = [90, 40, 80, 98]
# dictionary of lists
dict = {'name': nme, 'degree': deg, 'score': scr}
#df = [Link](dict,index=range(1,5))
# You can provide index value like this also
df=[Link](dict,index=['s101','s102','s103','s104'])
# saving the dataframe
df.to_csv('[Link]')
BY: Heta S. Desai Shri [Link] College of CS & BM Page 27
Unit-4: Python Interaction with Text and CSV
How to Append a CSV file Using DataFrame?
Example: ([Link])
BY: Heta S. Desai Shri [Link] College of CS & BM Page 28
Unit-4: Python Interaction with Text and CSV
How to write data into Excel file using DataFrame?
Note:
Before Working With Excel install following packages:
• pip install openpyxl
• pip install xlrd==1.2.0
Example: ([Link])
How to write data into Specific Excel Sheet in an Excel using DataFrame?
Example: ([Link])
BY: Heta S. Desai Shri [Link] College of CS & BM Page 29
Unit-4: Python Interaction with Text and CSV
How to read data from Excel file using DataFrame?
Example: ([Link])
BY: Heta S. Desai Shri [Link] College of CS & BM Page 30
Unit-4: Python Interaction with Text and CSV
DataFrame Functions:
1) [Link] () :
The head() function is used to get the first n rows. This function returns the first n
rows for the object based on position.
It is beneficial when we have massive datasets, and it is not possible to see the
entire dataset at once.
Syntax:
[Link](n)
Here, n is the number of rows to be selected. Here default value is
5.
BY: Heta S. Desai Shri [Link] College of CS & BM Page 31
Unit-4: Python Interaction with Text and CSV
Example: ([Link])
import pandas as pd
df=pd.read_csv('[Link]')
print("\n",[Link]())
print("=====================================")
print("\n",[Link](2))
print("=====================================")
Note:
We can use Negative Number with [Link]() function. we
want to see all the rows except for the last n rows, we can pass the negative
value as a parameter to [Link]().
Example:
BY: Heta S. Desai Shri [Link] College of CS & BM Page 32
Unit-4: Python Interaction with Text and CSV
if we want to display all the rows except the bottom two rows, we
can use [Link](-2) function.
2) [Link] () :
We can use the [Link]() function to display the last n rows of the
DataFrame. Like the head function, this function is used when we want to
view a smaller section of the entire DataFrame.
It takes input as the number of rows to be displayed from the bottom. The
default value is 5.
Syntax:
[Link](n)
Here, n is number to be printed from the bottom.
Example: ([Link])
import pandas as pd
df=pd.read_csv('[Link]')
print("\n",[Link]())
print("=====================================")
print([Link](2))
BY: Heta S. Desai Shri [Link] College of CS & BM Page 33
Unit-4: Python Interaction with Text and CSV
Note:
We can use negative number with [Link]() function. When we
want to see our entire dataset except for the first few rows, we can use
[Link]() function and pass the negative value as a parameter to it.
Example:
if we display all the rows except the top 2 rows using [Link](2).
3) [Link]():
The describe() method is used for calculating some statistical data like
percentile, mean and std of the numerical values of the Series or
DataFrame.
Syntax:
[Link](percentiles=None)
Parameter:
BY: Heta S. Desai Shri [Link] College of CS & BM Page 34
Unit-4: Python Interaction with Text and CSV
• Percentiles: The percentiles to include in the output. All should fall
between 0 and 1. The default is [.25, .5, .75], which returns the 25th,
50th, and 75th percentiles.
How to calculate Central tendency-
1) Mean
2) Median
3) Mode
4) Variance using describe()
Example: ([Link])
BY: Heta S. Desai Shri [Link] College of CS & BM Page 35
Unit-4: Python Interaction with Text and CSV
BY: Heta S. Desai Shri [Link] College of CS & BM Page 36
Unit-4: Python Interaction with Text and CSV
4) loc():
loc is label-based, which means that you have to specify rows and
columns based on their row and column labels.
Syntax:
loc[row_label, column_label]
5) iloc():
iloc is integer position-based, so you have to specify rows and
columns by their integer position values (0-based integer position).
Syntax:
iloc[row_position, column_position]
Example: ([Link])
BY: Heta S. Desai Shri [Link] College of CS & BM Page 37
Unit-4: Python Interaction with Text and CSV
BY: Heta S. Desai Shri [Link] College of CS & BM Page 38
Unit-4: Python Interaction with Text and CSV
BY: Heta S. Desai Shri [Link] College of CS & BM Page 39
Unit-4: Python Interaction with Text and CSV
6) To_numpy:
Pandas DataFrame is two-dimensional size-mutable, potentially
heterogeneous tabular data structure with labeled axes (rows and
columns). This data structure can be converted to NumPy ndarray with the
help of Dataframe.to_numpy() method.
Syntax:
Dataframe.to_numpy(Data_set)
Example: ([Link])
BY: Heta S. Desai Shri [Link] College of CS & BM Page 40
Unit-4: Python Interaction with Text and CSV
7) [Link]():
This function will display the value of the dataframe.
Syntax:
[Link]()
BY: Heta S. Desai Shri [Link] College of CS & BM Page 41
Unit-4: Python Interaction with Text and CSV
BY: Heta S. Desai Shri [Link] College of CS & BM Page 42