lOMoARcPSD|44921472
Software tools of EDA explanation juky 26 , 2025 class
computer science and engineering (Anna University)
Scan to open on Studocu
Studocu is not sponsored or endorsed by any college or university
Downloaded by GNANAMURTHY S (gnanamurthyspec@[Link])
lOMoARcPSD|44921472
Title: Basic Notes on Exploratory Data Analysis (EDA) Using Python Libraries
Software Tools Available for EDA
Exploratory Data Analysis (EDA) is the process of summarizing the main characteristics of data, often using
visual methods. Various software tools and libraries in Python make EDA easy and efficient. Below are some
of the most commonly used tools with examples and simple explanations.
1. NumPy
NumPy is a powerful library used for numerical computing in Python. It provides efficient handling of large
multi-dimensional arrays and matrices along with a collection of mathematical functions to operate on
these arrays.
Basic Usage:
import numpy as np # Imports the NumPy library and assigns it the alias np
Example Operations:
x = [Link](0.0, 50.0, 1.0) # Creates an array with values from 0 to 49 with
step 1
[Link]('[Link]', x, delimiter=',') # Saves the array to a text file with
comma delimiter
z = [Link]('[Link]', unpack=True)
# Loads data from the text file into an array
my_array2 = [Link]('[Link]', skip_header=1, filling_values=-999) #
Loads data, skips header, fills missing values
Inspecting Arrays:
print([Link]) # Prints number of dimensions of the array
print([Link]) # Prints total number of elements in the array
print([Link]) # Prints memory layout information of the array
print([Link]) # Prints number of bytes per element
print([Link]) # Prints total memory used by the array in bytes
Downloaded by GNANAMURTHY S (gnanamurthyspec@[Link])
lOMoARcPSD|44921472
2. Pandas
Pandas is a high-level data manipulation tool built on the NumPy package. It offers data structures like
Series (1D) and DataFrames (2D) for efficient data handling.
Setup:
import pandas as pd # Imports the pandas library as pd
import numpy as np # Imports NumPy again as a dependency for pandas
print("Pandas Version:", pd.__version__) # Prints the version of pandas
installed
pd.set_option('display.max_columns', 500) # Sets display to show up to 500
columns
pd.set_option('display.max_rows', 500) # Sets display to show up to 500 rows
Creating Data Structures:
series = [Link]([2, 3, 7, 11]) # Creates a one-dimensional labeled array
(Series)
Creating a DataFrame:
series_df = [Link]({
'A': range(1, 5), # Column A with values 1 to 4
'B': [Link]('20190526'), # Same timestamp for all rows
'C': [Link](5, index=list(range(4)), dtype='float64'), # Column with same
float value (5.0)
'D': [Link]([3] * 4, dtype='int64'), # Column with repeated integer value 3
'E': [Link](["Depression", "Social Anxiety", "Bipolar Disorder",
"Eating Disorder"]), # Categorical text column
'F': 'Mental health', # Same text for all rows
'G': 'is challenging' # Same text for all rows
})
print(series_df) # Displays the DataFrame
Loading External Data:
columns = ['age', 'workclass', 'fnlwgt', 'education', 'education_num',
'marital_status',
'occupation', 'relationship', 'ethnicity', 'gender',
'capital_gain', 'capital_loss', 'hours_per_week',
'country_of_origin', 'income'] # Column names
Downloaded by GNANAMURTHY S (gnanamurthyspec@[Link])
lOMoARcPSD|44921472
df = pd.read_csv('[Link]
adult/[Link]', names=columns) # Loads dataset
[Link](10) # Displays the first 10 rows
Exploring Data:
[Link]() # Shows data types, non-null values, and memory usage
Selecting Data from DataFrame:
[Link][10] # Selects the 11th row
[Link][0:10] # Selects the first 10 rows
[Link][10:15] # Selects rows 11 to 15
[Link][-2:] # Selects the last 2 rows
[Link][::2, 3:5].head() # Selects every other row for columns 3 and 4, and
displays the first 5 rows
Combining Pandas and NumPy:
[Link](24) # Sets random seed to get same random values every time
dFrame = [Link]({'F': [Link](1, 10, 10)}) # Creates DataFrame with
linearly spaced values from 1 to 10
# Concatenates the df DataFrame with random values for 5 columns named E to A
# Resulting DataFrame has original df data plus these new columns
dFrame = [Link]([df, [Link]([Link](10, 5),
columns=list('EDCBA'))], axis=1)
[Link][0, 2] = [Link] # Sets the value at first row and third column to
NaN (missing)
Styling Data:
def colorNegativeValueToRed(value):
if value < 0:
return 'color: red' # Red for negative
Downloaded by GNANAMURTHY S (gnanamurthyspec@[Link])
lOMoARcPSD|44921472
elif value > 0:
return 'color: black' # Black for positive
else:
return 'color: green' # Green for zero
styled_df = [Link](colorNegativeValueToRed,
subset=['A','B','C','D','E']) # Apply color rule to columns
styled_df # Display styled DataFrame
Highlighting Min/Max Values:
def highlightMax(s):
isMax = s == [Link]() # True for maximum value(s)
return ['background-color: orange' if v else '' for v in isMax] # Orange
for max
def highlightMin(s):
isMin = s == [Link]() # True for minimum value(s)
return ['background-color: green' if v else '' for v in isMin] # Green for
min
styled =
[Link](highlightMax).apply(highlightMin).highlight_null(null_color='red')
# Apply style
styled # Show styled DataFrame
3. SciPy
SciPy is a library used for scientific and technical computing. It extends NumPy and provides modules for
optimization, integration, interpolation, eigenvalue problems, algebraic equations, and other tasks.
Start With:
from scipy import stats # Imports the stats module from SciPy for statistical
analysis
Useful for statistical analysis, e.g., mean, median, t-tests, etc.
4. Matplotlib
Matplotlib is a comprehensive library for creating static, animated, and interactive visualizations in Python.
Downloaded by GNANAMURTHY S (gnanamurthyspec@[Link])
lOMoARcPSD|44921472
Plot Type Code Example Description
Line Plot [Link](x, y) Line graph showing trend/relationship
Bar Chart [Link](x, y) Vertical bars for comparing quantities
Histogram [Link](data) Distribution of a single variable
Scatter [Link](x, y) Relationship between two variables
Simple Example:
import [Link] as plt # Imports the Matplotlib plotting module
[Link]([1, 2, 3, 4], [1, 4, 9, 16])
# Plots x=[1,2,3,4] and y=[1,4,9,16] as a line graph
[Link]("Simple Line Plot") # Adds title to the plot
[Link]("X-axis") # Labels x-axis
[Link]("Y-axis") # Labels y-axis
[Link]() # Displays the plot
These software tools (NumPy, Pandas, SciPy, and Matplotlib) provide essential functionality for carrying out
data preprocessing, analysis, and visualization effectively in Python.
Downloaded by GNANAMURTHY S (gnanamurthyspec@[Link])