Data Handling using Pandas –I
Pandas:
• It is a package useful for data analysis and manipulation.
• Pandas provide an easy way to create, manipulate and wrangle the data.
• Pandas provide powerful and easy-to-use data structures, as well as the means to quickly
perform operations on these structures.
Data scientists use Pandas for its following advantages:
• Easily handles missing data.
• It uses Series for one-dimensional data structure and DataFrame for multi-dimensional
data structure.
• It provides an efficient way to slice the data.
• It provides a flexible way to merge, concatenate or reshape the data.
DATA STRUCTURE IN PANDAS
A data structure is a way to arrange the data in such a way that so it can be accessed
quickly and we can perform various operation on this data like- retrieval, deletion,
modification etc.
Pandas deals with 3 data structure
1. Series
2. Data Frame
3. Panel
Series
Series-Series is a one-dimensional array like structure with homogeneous data, which can
be used to handle and manipulate data.
It has two parts
1. Data part (An array of actual data)
2. Associated index with data (associated array of indexes or data labels) e.g.-
✓ We can say that Series is a labeled one-dimensional array which can hold any type of
data.
✓ Data of Series is always mutable, means it can be changed.
✓ But the size of Data of Series is always immutable, means it cannot be changed.
✓ Series may be considered as a Data Structure with two arrays out which one array
works as Index (Labels) and the second array works as original Data.
✓ Row Labels in Series are called Index.
Syntax to create a Series:
<Series Object>= [Link] (data, index=idx(optional))
Creating a series from Scalar value
To create a series from scalar value, an index must be provided. The scalar value will be
repeated as per the length of index.
Creating a series from a Dictionary
head (): It is used to access the first 5 rows of a series.
Note :To access first 3 rows we can call series_name.head(3)
tail(): It is used to access the last 5 rows of a series.
DATAFRAME
DATAFRAME-It is a two-dimensional object that is useful in representing data in the form
of rows and columns. It is similar to a spreadsheet or an SQL table. This is the most
commonly used pandas object. Once we store the data into the Dataframe, we can perform
various operations that are useful in analyzing and understanding the data.
PROPERTIES OF DATAFRAME
DATAFEAME
1. A Dataframe has axes (indices)-
➢ Row index (axis=0)
➢ Column index (axes=1)
2. It is similar to a spreadsheet , whose row index is called index and column index is
called column name.
3. A Dataframe contains Heterogeneous data.
4. A Dataframe Size is Mutable.
5. A Dataframe Data is Mutable.
A data frame can be created using any of the following
1. Series
2. Lists
3. Dictionary
4. A numpy 2D array
Cleaning Data
Data cleaning means fixing bad data in your data set.
Bad data could be:
● Empty cells
● Data in wrong format
● Wrong data
● Duplicates
[Link] for upcoming programs and examples
Cleaning Empty Cells
Replace Using Mean, Median, or Mode
A common way to replace empty cells, is to calculate the mean, median or mode value of the column.
Pandas uses the mean() median() and mode() methods to calculate the respective values for a specified
column:
Removing Duplicates
Duplicate rows are rows that have been registered more than one [Link] discover duplicates, we can use the
duplicated() [Link] duplicated() method returns a Boolean values for each row:Returns True for every row that is a
duplicate, otherwise False
print([Link]())
Removing Duplicates
To remove duplicates, use the drop_duplicates() method.
Example
Remove all duplicates:
df.drop_duplicates(inplace = True)