Module 6 – Data Visualization and Analysis
using R language
Data Handling and Manipulation with Pandas
Pandas is a foundational library for managing structured data in Python. It provides powerful data
structures—Series (a one-dimensional labeled array) and Data Frame (a two-dimensional table)—to
store and operate on data efficiently.
• Data Import Example:
Reading data from files is simple.
import pandas as pd
df = pd.read_csv('[Link]') # Reads a CSV into a DataFrame
DataFrames make data processing intuitive: columns are accessed by name, and rows by label or
position.
• Filtering and Cleaning:
Pandas offers concise syntax for filtering and cleaning data.
filtered = df[df['Marks'] > 50] # Filters rows where "Marks" > 50
[Link](0, inplace=True) # Replaces missing values with 0
This facilitates removal of outliers, filling missing data, and selecting subsets relevant for analysis.
• Grouping and Aggregation:
For analytics, grouping and aggregation summarize patterns.
summary = [Link]('Class')['Marks'].mean() # Average marks by class
Analysts can easily compute sums, means, counts, and other metrics across categories.
Array Mathematics with NumPy
NumPy deals with homogeneous numerical data, providing high-performance arrays and a vast set of
mathematical operations.
• Array Creation and Operations:
import numpy as np
arr = [Link]([60, 75, 55, 89])
mean = [Link](arr) # Calculates mean efficiently
sqrt_arr = [Link](arr) # Takes element-wise square root
NumPy is optimized for computations such as linear algebra, statistics, and signal processing.
When to Use Pandas, NumPy, or SciPy
Choosing the right tool depends on the data and task:
• Pandas: Structured, tabular data manipulation, combining, reshaping, and analysis.
• NumPy: Fast, vectorized computations on large arrays—best for homogeneous numerical data.
• SciPy: Advanced scientific computations—statistics, optimization, and integration—building on
NumPy’s arrays. For example, hypothesis testing or curve fitting requires SciPy.
Scientific Calculations with SciPy
SciPy extends NumPy with functions for statistics and other scientific computation.
• Statistical Example:
from scipy import stats
scores = [80, 75, 90, 65]
result = stats.ttest_1samp(scores, popmean=70)
This tests whether the average score is statistically different from 70, illustrating scientific analysis using
real datasets.
Visualizing Data with Matplotlib
Data visualization is critical for exploration and communication. Matplotlib provides a toolkit for crafting
a wide variety of visualizations.
Line Plots
• Purpose & Implementation:
Line plots display trends over continuous or sequential data.
import [Link] as plt
years = [2021, 2022, 2023]
sales = [15000, 17500, 19000]
[Link](years, sales, linestyle='--', color='green')
[Link]("Year")
[Link]("Sales")
[Link]("Annual Sales Trend")
[Link]()
The line plot is effective for showing changes over time or ordered categories.
Scatter Plots vs Line Plots
• Comparison Table:
Line Plot Scatter Plot
Shows Trends, continuous progression Relationships, spread among data points
Data Sequential/time series Two quantitative variables
Usage Forecasting, progress tracking Detecting correlations/clusters
Bar Plots
• Implementation:
categories = ['Apples', 'Bananas', 'Cherries']
values = [12, 30, 22]
[Link](categories, values)
[Link]('Fruits')
[Link]('Quantity')
[Link]('Fruit Sales')
[Link]()
Bar plots are ideal for discrete, categorical data comparisons.
• Line Plot vs Bar Plot:
Line plots emphasize trends; bar plots compare categorical magnitudes. For sales by month, use a
line plot to show seasonality; for sales by product, use a bar plot to compare products.
Histograms and Bar Charts
• Histogram Example:
ages = [22, 23, 23, 25, 30, 35, 36, 38, 42, 47]
[Link](ages, bins=5)
[Link]('Age Group')
[Link]('Frequency')
[Link]('Age Distribution')
[Link]()
Histograms visualize distributions for continuous data, dividing values into ranges (bins).
• Difference Table:
Feature Histogram Bar Chart
Data Continuous Categorical
Bars Touching Separated
X-Axis Binned ranges Categories
Purpose Frequency Magnitude per category
• When to Use Histogram:
Use histograms for analyzing data spreads (e.g., salary distribution), and bar charts for categorical
counts (e.g., gender counts).
Studying Real-World Data with Histograms
Histograms reveal patterns such as skewness, modality, and outliers, allowing analysts to understand
distributions and detect irregularities in attributes like transaction amounts or exam scores.
Pie Charts and Their Relevance
• Pie Chart Example:
activities = ['Reading', 'Gaming', 'Sports']
time_spent = [3, 5, 2]
[Link](time_spent, labels=activities, autopct='%1.1f%%')
[Link]('Daily Activities')
[Link]()
• Relevance:
Pie charts display proportions of a whole, making it easy to see dominant segments—such as
market share or budget allocations.
Box Plots: Data Spread and Outliers
• Understanding Box Plots:
[Link]([70, 75, 80, 95, 100, 110])
[Link]('Test Scores Distribution')
[Link]()
The box illustrates the interquartile range (middle 50%), the center line is the median, and the
whiskers extend to non-outlier minimum and maximum values. Outliers are plotted as individual
points.
• Interpretation:
Box plots summarize data’s variability, highlight skewness, and signal presence of outliers which
may require further investigation.
Visualizing Distribution with Violin Plots
• Violin Plot Example:
import seaborn as sns
scores = [55, 60, 65, 80, 85, 86, 90]
[Link](data=scores)
[Link]('Scores Spread')
[Link]()
Violin plots combine box plots with kernel density estimation, showing shape and modality of data
distribution.
• Box Plot vs Violin Plot Table:
Feature Box Plot Violin Plot
Shape Rectangular, simple Smoothed density curve
Outliers Dots beyond whiskers Implied by densities
Detail Summary statistics Full distribution
Usage Quick summary In-depth distribution
Violin plots provide richer insight into distribution shape, especially for multi-modal data.
Customizing and Interpreting Plots with Matplotlib
• Customizing Style/Color:
[Link](x, y, color='red', linestyle='-.')
Customization helps differentiate series and emphasize patterns.
• Role of Labels and Titles:
Labels (xlabel, ylabel) clarify axes, while title provides context for the plot, enhancing
interpretability.
• Displaying Plots:
[Link]() must be called to render figures interactively.
Advanced Comparisons and Interpretation
• Grouped Bar Chart:
Comparing multiple categories side-by-side, such as sales of three products across two quarters,
enhances comparative analysis.
o Implementation:
import numpy as np
products = ['A', 'B', 'C']
Q1 = [20, 35, 30]
Q2 = [25, 32, 34]
x = [Link](len(products))
[Link](x-0.2, Q1, width=0.4, label='Q1')
[Link](x+0.2, Q2, width=0.4, label='Q2')
[Link](x, products)
[Link]()
[Link]()
• Interpretation of Box Plot Whiskers:
Whiskers indicate data range within 1.5*IQR, helping spot extremes and variability.