Course Contents

 Need for Exploratory Data Analysis

 Key factors of data visualization

 Exploring the Visual Data Spectrum: charting Primitives (Data Points,

Line Charts, Bar Charts, Pie Charts, Area Charts)

 Exploring advanced Visualizations (Candlestick Charts, Bubble Charts,

Surface Charts, Map Charts, Infographics).
 What is EDA
 Exploratory data analysis (EDA) involves using graphics and visualizations
to explore and analyze a data set. The goal is to explore, investigate and
learn, as opposed to confirming statistical hypotheses.
 The process of using numerical summaries and visualizations to explore
your data and to identify potential relationships between variables is called
exploratory data analysis, or EDA.
Why Exploratory Data Analysis is Important?

1. Understanding the data

2. Identifying patterns and relationships

3. Data quality check

4. Informing feature engineering and selection

5. Guiding model selection and improving accuracy

6. Hypothesis generation and testing

Types of Exploratory Data Analysis
 Exploratory Data Analysis Tools
 Python Libraries  Integrated Development Environments (IDEs)
 Pandas  Jupyter Notebook
 Matplotlib  Rstudio
 Seaborn
 Data Visualization Tools
 SciPy
 Tableau
 Plotly
 Power BI
 R Libraries
 Ggplot2  Statistical Analysis Tools
 dplyr  SPSS
 tidyr  SAS
 shiny  Data Cleaning Tools
 Plotly  OpenRefine
 SQL Databases
Data Visualization
Definition:
 Data Visualization is the graphical representation of information and data
using visual elements like charts, graphs, and maps.
Key Features:
 Uses visual elements like bar charts, line graphs, pie charts, heatmaps,
etc.
 Helps both technical and non-technical audiences grasp data quickly
 Enhances storytelling with data
Benefits:
 Clarity: Makes large datasets more accessible
 Speed: Enables faster decision-making
 Insight: Reveals hidden trends and relationships
 Engagement: Increases audience interest and comprehension
Need of Data Visualization
Simplifies Complex Data

Improves Understanding & Insight

Speeds Up Decision Making

Identifies Trends Over Time

Facilitates Better Communication

Enhances Data Exploration

Reduces Misinterpretation

Supports Big Data Analytics

Data visualization tools
 Tableau
 Power BI
 Matplotlib
 Plotly
 Bokeh
 Seaborn
Challenges in Data Visualization for BI

Choosing the right visualization

representation

Too much data

Unorganized data

Keeping it simple

Making it easy to use

 Future Trends in Data Visualization

Augmented Reality (AR) and Virtual Reality (VR) Visualization

Natural Language Processing (NLP) Integration

Real-time Data Visualization

Artificial Intelligence (AI) and Machine Learning (ML) Integration

Data Storytelling

Interactive and Dynamic Visualizations

QUESTIONS:
Explain key factors of data visualization.

Explain need of data visualization.

Define Exploratory Data Analysis. Why is it important in the data

analysis process?

What are the steps involved in performing Exploratory Data Analysis

(EDA)?

Explain essential components of effective data visualization and

their roles.
 Course Contents
 Need for Exploratory Data Analysis

 Key factors of data visualization

 Exploring the Visual Data Spectrum: charting Primitives (Data Points,

Line Charts, Bar Charts, Pie Charts, Area Charts)

 Exploring advanced Visualizations (Candlestick Charts, Bubble Charts,

Surface Charts, Map Charts, Infographics).
 Data Points
 Definition: Individual values or observations, typically represented as dots, bars, or slices in a chart.
 Example: In a temperature log, 75°F on Monday is a data point.
 Use: Basis of all chart types; they hold the values you visualize.
Data Point
(Day) (Temperature °F)
(●)
Monday 72°F ●

Tuesday 75°F ●

Wednesday 78°F ●

Thursday 77°F ●

Friday 74°F ●
 Line Chart
 A line chart is a form of graphical representation of data in the form of points that are joined
continuously with the help of a line. The line can either be straight or curved depending on the data
being researched.
 Its best use case is to illuminate trends, patterns, and variable changes.

 Perfect for showing trends over time, like tracking website traffic or how something changes.
 Types of Line Chart
1. Simple Line Chart
2. Multiple Line Chart
3. Compound Line Chart
 When to use line charts?

It help to measure how different groups relate to each

other.

It is effective for demonstrating progression, making

them suitable for scenarios like project timelines,
production cycles, or population growth.
 Best practices for line charts:
 Limit the number of lines

 Use distinct colors and styles

 Clear axis labels and scales

 Highlight significant data points

 Consider interactivity

 Provide context
 Ex. Plotting Temperature against Height
import [Link] as plt
#list storing date in string format
date=["25/12","26/12","27/12"]
#list storing temperature values
temp=[8.5,10.5,6.8]
#create a figure plotting temp versus date
[Link](date, temp)
#show the figure
[Link]()
import pandas as pd
import [Link] as plt
# reads "[Link]" to df by giving path to
the file
df=pd.read_csv("[Link]")
#create a line plot of different color for each
week
[Link](kind='line',
color=['red','blue','brown'])
# Set title to "Mela Sales Report"
[Link]('Mela Sales Report')
# Label x axis as "Days"
[Link]('Days')
# Label y axis as "Sales in Rs"
[Link]('Sales in Rs')
#Display the figure
[Link]()
import [Link] as plt
import numpy as np
x = [Link]([1, 2, 3, 4])
y = x*2
[Link](x, y)
x1 = [2, 4, 6, 8]
y1 = [3, 5, 7, 9]
[Link](x, y1, '-.')
[Link]("X-axis data")
[Link]("Y-axis data")
[Link]('multiple plots')
plt.fill_between(x, y, y1, color='green', alpha=0.5)
[Link]()
Advantages and Disadvantages of Line Chart

Advantage Disadvantage

Line charts help in gathering data when a

Usage of zero value baseline.
good measure of the interval is seen.

Overuse of lines will make the line chart very

Can use more than one line to plot data
messy.

Line charts can have both straight lines Cannot have both straight lines and curved
and curve lines. lines together on one line chart.

Cannot have dual axis in one line chart.

When to avoid line charts
 Discrete or categorical data

 Limited data points

 Complex data relationships

 Detailed price analyses

Bar charts
 Bar graphs are also known as bar charts and it is a pictorial representation
of grouped data. It is one of the ways of data handling.
 Bar graph is an excellent tool to represent data that are:
 independent of one another and
 that do not need to be in any specific order while being represented.
Types of Bar Graphs
 Uses of Bar Graph

 used in mathematics and statistics

 The comparisons between different categories are easy and convenient.
 It is the easiest diagram to prepare and does not require too much effort.
 It is the most widely used method of data representation. Therefore, it is
used by various industries.
 It is used to compare data sets that are independent of one another.
 It helps in studying patterns over long periods of time.
 Best practices for bar charts

Clearly label each bar and axis with concise labels

Limit the number of bars and categories to avoid cognitive

overload

Purposely use colors to highlight key points and convey meaning

 Bar Plot in Matplotlib
import [Link] as plt
import numpy as np
fruits = ['Apples', 'Bananas', 'Cherries', 'Dates']
sales = [400, 350, 300, 450]
[Link](fruits, sales)
[Link]('Fruit Sales')
[Link]('Fruits')
[Link]('Sales')
[Link]()
import pandas as pd
df= pd.read_csv('[Link]')
import [Link] as plt
# plots a bar chart with the column
"Days" as x axis
[Link](kind='bar',x='Day',title='Mela
Sales Report')
#set title and set ylabel
[Link]('Sales in Rs')
[Link]()
Advantages of Bar Graphs
 Highlighting Trends
 Customizations
 Space Efficiency

Disadvantages of Bar Graphs

 Limited Details
 Misleading Scaling
 Overcrowding
 Area chart

 Area charts are similar to line graphs but with the area below the line filled in
with color.

 They are used to represent cumulative totals or stacked data over time.

 Area charts are effective for showing changes in composition over time and
comparing the contributions of different categories to the total.
 Types of area charts
Simple area chart

Stacked area chart

100% stacked area chart

3D area chart

 When to use area charts
Displaying trends over time.
Emphasizing cumulative values
Representing part-to-whole relationships
Comparing multiple data series
Visualizing magnitude
 Area Plots in Python
import [Link] as px
df = [Link]()
fig = [Link](df, x="sepal_width", y="sepal_length",
color="species",
hover_data=['petal_width'],)
[Link]()
import pandas as pd [Link]("X-axis")
import [Link] as plt [Link]("Y-axis")
# Sample data [Link]()
df = [Link]({ [Link]()
'x': list(range(1, 11)),
'Category A': [1, 3, 2, 4, 5, 7, 6, 8, 9, 10],
'Category B': [2, 4, 3, 5, 6, 8, 7, 9, 10, 11],
'Category C': [3, 5, 4, 6, 7, 9, 8, 10, 11, 12] })
# Define custom colors for each category
colors = ['yellow', 'purple', 'pink']
# Create the stacked area line plot with custom colors
[Link](df['x'], df['Category A'], df['Category B'], df['Category
C'], colors=colors, alpha=0.7)
# Plot lines for each category with custom colors
[Link](df['x'], df['Category A'], color='blue', alpha=0.5,
linewidth=0.9)
[Link](df['x'], df['Category B'], color='green', alpha=0.5,
linewidth=0.9)
[Link](df['x'], df['Category C'], color='red', alpha=0.5,
linewidth=0.9)
[Link]("Stacked Area Line Plot with Custom Colors")
Advantages of Using Area Charts
 Visually Appealing
 Great for Trends
 Compares Well
Highlight trends
Illustrate cumulative totals
Visually appealing

Disadvantages of Using Area Charts

 Not for Precise
 Limited Data Sets
 Occlusion
 Clutter with too many categories
 Not ideal for negative values
Area Chart – When to avoid?
Obviously, similarly to line charts, area charts are not suitable for
representing parts of a whole over a single period.
 In our example, we can’t use an area chart to show the proportion
of revenues each division generated in say, 2018 alone. So that’s a
situation where we can’t use an area chart.
In general, I would stay away from the classical area chart too. It
can be very confusing and even Microsoft themselves recommend
avoiding it and to consider using a simple line chart.
 Pie charts
 A pie chart is a type of graph that records data in a circular manner that is further divided into
sectors for representing the data of that particular part out of the whole part.
 Each of these sectors or slices represents the proportionate part of the whole.
 Pie charts, also commonly known as pie diagrams help in interpreting and representing the data
more clearly.
 It is also used to compare the given data.
 When to use pie charts?

This classic chart type is effective when you want to illustrate the
proportion of each category in the dataset.

However, remember not to use these types of charts for large

datasets, as too many slices can create confusion.

 The chart is suitable when you have limited categories, ideally less
than six or seven.
Best practices for pie chart

Keep the number of slices limited to maintain clarity

Clearly label each slice with clear text.

Ensure consistency so viewers associate colors with specific

categories
 Pie Chart Types
 Simple Pie Chart

 3D Pie Chart
 Exploded Pie Chart

 Donut Chart  Multi-Level Pie Chart

Pie chart in python
import [Link] as plt
import numpy as np

y = [Link]([35, 25, 25, 15])

mylabels = ["Apples", "Bananas", "Cherries", "Dates"]

[Link](y, labels = mylabels, startangle = 90)

[Link]()
import [Link] as plt
import numpy as np

y = [Link]([35, 25, 25, 15])

mylabels = ["Apples", "Bananas", "Cherries", "Dates"]
myexplode = [0.2, 0, 0, 0]

[Link](y, labels = mylabels, explode = myexplode)

[Link]()
 Pie Chart – When to avoid?
We can’t use a pie chart in situations when we would like to show
how one or more variables develop over time.

Pie charts are a definite no-go in these cases. Moreover, as mentioned

earlier, a pie chart would be misleading if we don’t consider all
values.

 In the context of example, we shouldn’t create a pie chart that

includes revenue of only two of the firm’s three divisions.
Pie Chart Advantages
 A pie chart is a simple and easy-to-understand method to represent the data
visually as a fractional part of a whole.
 It provides an effective communication tool visually simpler than other types of
graphs.
 Pie charts offer a visually appealing yet straightforward way of presenting data.
 Pie charts allow you to make quick comparisons between categories.

Pie Chart Disadvantages

 A pie chart cannot show more than a few values without separating the visual
encoding from the data they represent, that is they are not very effective when the
number of values in a data set increases.
 It does not easily reveal exact values in the data set.
 It might be difficult to compare smaller-sized slices.
 There is a chance of misinterpreting data because of factors like slice colours, slice
orders, and presentation angles.
 Bubble chart
 A bubble chart is a data visualization that extends a scatter plot by adding a
third dimension to the data points, represented by the size of the bubbles.
 This allows for the visualization of relationships between three numeric
variables within a single chart.
Key Features:
 X-axis and Y-axis: These represent two numeric variables, just like in a scatter
plot.
 Bubble Size: The size of each bubble corresponds to a third numeric variable,
with larger bubbles indicating higher values.
 Visual Representation: Bubble charts help in identifying patterns and
relationships between the three variables, making it easier to understand
complex data.
When to use a bubble chart
Comparing data points

Identifying outliers and clusters

Revealing trends

Strategic decision making

 Types of bubble chart
Labelled Bubble Charts 3D Bubble Charts
Bubble Maps (Cartograms)
import pandas as pd
# Sample data
data = {
'x': [1, 3, 4, 6, 8],
'y': [10, 25, 40, 35, 50],
'size': [100, 300, 500, 200, 400],
'color': ['red', 'blue', 'green', 'yellow', 'orange'],
'label': ['A', 'B', 'C', 'D', 'E']
}
# Creating DataFrame
df = [Link](data)
fig = [Link](df, x='x', y='y', size='size', color='color', hover_data=['label'],
width=800, height=500)
[Link]()
 Best practices for creating effective bubble
charts
Choose the right data
Limit the number of data points
Scale bubble area by value
Provide clear labels and legends
Consider interactivity
 Limitations of bubble charts
Difficulty in ascertaining exact values
Overlapping bubbles
Not ideal for depicting negative values
Can be difficult to interpret
 Examples of bubble chart applications

Financial analysis

Sales and marketing

Project management

Healthcare analysis
Candlestick Chart
Candlestick charts are a type of financial chart used to represent the
price movements of a security, derivative, or currency over a period
of time.

They display the open, high, low, and closing prices for a specific
period, often used in technical analysis for trading
Structure of a Candlestick:
Types of Candlestick patterns
How to Use Candlestick Charts:

1. Identify Trends

2. Recognize Reversal Patterns

3. Combine with Other Indicators

4. Timeframe Selection

5. Risk Management
Where Are Candlestick Charts Used?

Stock market analysis

Crypto trading

Forex markets

Technical analysis
Tools to Create Candlestick Charts:

 Python: matplotlib, plotly, or mplfinance

 Excel: Stock chart type

 Platforms: TradingView, Yahoo Finance, MetaTrader, etc.

import [Link] as plt ohlc['date'] =
ohlc['date'].apply(mpl_dates.date2num)
# Importing all the required libraries
ohlc = [Link](float)
from mpl_finance import candlestick_ohlc
# Creating Subplots
import pandas as pd
fig, ax = [Link]()
import [Link] as mpl_dates
candlestick_ohlc(ax, [Link], width=0.6,
import numpy as np
colorup='blue',
import datetime
colordown='green', alpha=0.4)
# Defining a dataframe showing stock prices
# Setting labels & titles
# of a week
ax.set_xlabel('Date')
stock_prices = [Link]({'date':
ax.set_ylabel('Price')
[Link]([[Link](2021, 11, i+1)
[Link]('Stock Prices of a week')
for i in range(7)]),
# Formatting Date
'open': [36, 56, 45, 29, 65, 66, 67],
date_format = mpl_dates.DateFormatter('%d-%m-
'close': [29, 72, 11, 4, 23, 68, 45],
%Y')
'high': [42, 73, 61, 62, 73, 56, 55],
[Link].set_major_formatter(date_format)
'low': [22, 11, 10, 2, 13, 24, 25]})
fig.autofmt_xdate()
ohlc = stock_prices.loc[:, ['date', 'open', 'high', 'low',
fig.tight_layout()
'close']]
[Link]()
ohlc['date'] = pd.to_datetime(ohlc['date'])
 Benefits of Candlestick Charts

 Enhanced Insights into Market Sentiment

 Dynamic Representation of Price Movements

 Quick Interpretation

Limitations & Considerations of Candlestick Charts

 False Signals and Subjectivity

 Context Matters
Surface charts
 3D surface plots, are a type of data visualization used to represent the relationship between
three variables in a three-dimensional space.

 They are particularly useful for visualizing trends and patterns within large datasets that
might otherwise be difficult to discern.

 Imagine a rubber sheet stretched over a 3D column chart: that's essentially what a surface
chart depicts. The x and y axes represent two independent variables, while the z-axis (the
height of the surface) represents the dependent variable, showing how it changes in
relation to the other two.
 Types of surface charts
 3D Surface Chart

 Wireframe 3D Surface Chart

 Contour Chart

 Wireframe Contour Chart

When interpreting a surface chart, focus on the following:
 Color Gradients: The color variations indicate changes in value across the
data points.

 Peaks and Valleys: These represent high and low points in the dataset.

 Chart Rotation: Rotating the chart can provide different perspectives and
reveal hidden insights.

 Legends and Axis Labels: Ensure clear and descriptive labels for better
understanding.
# Import libraries from mpl_toolkits
import mplot3d
import numpy as np
import [Link] as plt
# Creating dataset
x = [Link]([Link](-3, 3, 32), [Link](32))
y = [Link]().T # transpose
z = ([Link](x **2) + [Link](y **2) )
# Creating figure
fig = [Link](figsize =(14, 9))
ax = [Link](projection ='3d')
# Creating plot
ax.plot_surface(x, y, z)
# show plot
[Link]()
 Advantages
Complex data patterns
Highlighting high and low values
Optimal combinations
Data comparison.
Enhanced data perception and interaction:
Advanced analysis capabilities
Better data storytelling and aesthetic appeal
 Disadvantages
Visual complexity
Data requirements
Time-consuming
Limited for discrete data
Potential for misrepresentation
Difficulty in unwrapping or viewing hidden data
 Map charts
 Map charts, also known as thematic maps or cartograms, are powerful data
visualization tools that display information overlaid onto geographical maps.
 They provide a visual way to understand data in the context of location, allowing
users to identify patterns, trends, and relationships related to geographical areas.
When to Use Map Charts
 Comparing regional sales or market share.
 Showing disease outbreaks, internet penetration, or voter turnout.
 Mapping logistics, customer locations, or geospatial patterns.
 Types of Map Charts

Type Description Example Use

Choropleth Map Regions colored based on values Population, sales, election results

Bubble Map Circles on a map sized by value COVID-19 cases per city

Heat Map (Geo Heat) Colors show intensity over regions Temperature, foot traffic

Symbol Map Custom icons or markers on map Store locations, delivery points

Trajectory/Path Map Lines showing movement Flight paths, vehicle tracking

Tools to Create Map Charts
 Python: GeoPandas + Matplotlib, plotly, or Basemap / Cartopy
 Excel: Built-in Map Chart under Insert → Charts → Maps
 Power BI / Tableau
 Online Tools
 Datawrapper: Free, simple choropleth and symbol maps.
 Mapbox Studio: Highly customizable and developer-friendly.
 Flourish: Interactive, beautiful maps for storytelling.
 Examples of map chart applications
 Sales Performance:
 Visualizing sales figures for different regions to identify top-performing areas and underperforming regions.
 Election Results:
 Displaying voting patterns and results for different districts or regions.
 Population Density:
 Mapping population distribution to understand spatial concentrations and identify areas with high or low population
density.
 Disease Outbreaks:
 Tracking the spread of diseases across geographic regions to identify hotspots and potential areas of concern.
 Weather Patterns:
 Visualizing temperature, rainfall, or other weather data across different locations.
 Resource Distribution:
 Mapping the distribution of natural resources, such as minerals or water, across a region.
import [Link] as px
import pandas as pd
# Import data from USGS
data =pd.read_csv('[Link]
# Drop rows with missing or invalid values in the 'mag' column
data = [Link](subset=['mag'])
data = data[[Link] >= 0]
# Create scatter map
fig = px.scatter_geo(data, lat='latitude', lon='longitude', color='mag', hover_name='place',
#size='mag', title='Earthquakes Around the World')
[Link]()
Advantages of map charts

Clarity and Simplicity

 Geographical Context

Enhanced Spatial Analysis

Improved Decision-Making

Identification of Patterns and Trends

Interactive Engagement
Disadvantages of map charts
Potential for Distortion
Overcrowding and Complexity
Limited Detail
Static Representation (for some types)
Data Accuracy is Crucial
 Infographics
 Infographics are visual representations of information, data, or knowledge designed to present complex
content quickly and clearly.
 They combine text, charts, icons, images, and graphics to tell a story or explain a concept in a visually
appealing way.
 Key Components of an Infographic

 Headline – Grabs attention and summarizes the topic.

 Sections – Break the content into logical parts.

 Visuals – Icons, illustrations, charts, timelines, etc.

 Text – Concise, easy-to-read explanations.

 Data Visualization – Charts (bar, pie, map, etc.) to back up claims.

 Call to Action – (optional) What you want the viewer to do next.

Key aspects of infographics in data visualization:
 Combining visual elements
 Storytelling
 Accessibility
 Engagement
 Versatility
 Presenting survey results
 Comparing data across categories or time periods
 Visualizing trends and patterns
 Simplifying complex data
 Enhancing reports and presentations
 Types of Infographics
Type Purpose Example Use

Statistical Showcase data, surveys, reports Marketing stats, research data

Company history, project
Timeline Show chronological progression
milestones
Process Explain steps or workflows How-to guides, tutorials

Comparison Compare two or more things Product A vs. Product B

Regional sales, demographic
Geographic/Map Display location-based information
spread

Hierarchical Rank or organize items Organizational chart, food chain

Present a series of tips, items, or

List "5 ways to save energy"
facts
 Tools to Create Infographics
Design Tools (No Code)
•Canva – Drag-and-drop tool, ideal for beginners.
•Piktochart – Specializes in infographics and data visualization.
•Venngage – Templates for business, education, and marketing.
•Adobe Express – Fast and polished, with Adobe’s design power.
•Visme – Good for infographics, presentations, and reports.
🧑 💻 Code-Based Tools (Customizable)
•Python – Use matplotlib, seaborn, or plotly to create data visuals,
then combine with images using Pillow or export for design.
•[Link] (JavaScript) – Interactive, custom web-based infographics.
•R (ggplot2 + patchwork) – For combining multiple charts into a single layout.
Infographics vs. Regular Charts

Feature Infographic Chart/Graph

Purpose Tell a story visually Present data visually

Design Focus High (fonts, colors, layout) Moderate

Interactivity Often static Can be interactive (in web apps)

Audience General/public Analysts, data scientists

 QUESTIONS:
 Discuss in brief Area Chart.
 Differentiate between static charts and dynamic charts with suitable example.
 Explain the usage and benefits of the following advanced visualizations:
a) Surface Charts
b) Map Charts
c) Infographics
 What is a candlestick chart and where is it commonly used?
 Define bubble charts and explain how they differ from scatter plots.
 Differentiate between infographics and traditional charts.
 Which component of an infographic ensures the credibility of the data presented?
 Explain the basic structure of a candlestick in a candlestick chart. Describe the significance
of each part (body, wick/shadow, color).
 Discuss the advantages and limitations of using surface charts in data visualization. Include
one example where surface charts would be ideal and one where they would not.
 Discuss the advantages and limitations of using pie charts in data visualization.
 Course Contents
 Acquiring and Visualizing Data from Text Files (.txt, .csv, XML),

 Displaying JSON content Outputting Basic Table Data (Building a

table, Using Semantic Table, Configuring the columns),

 Assuring Maximum readability (Styling your table, Increasing

readability, Adding dynamic Highlighting),

 Including computations, Using data tables library, relating data table to

a chart.
 Acquiring and Visualizing Data from Text Files
.txt (Plain Text Files):
Steps for Reading a File in Python
To read a file, Please follow these steps:
1. Find the path of a file.
An absolute path contains the complete directory list required to locate the file.
A relative path contains the current directory and then the file name.
2. Open file in Read Mode
To open a file Pass file path and access mode to the open() function. The access mode specifies the
operation you wanted to perform on the file, such as reading or writing. For example, r is for reading.
For example, fp= open(r'File_Path', 'r')
3. Read content from a file.
Once opened, we can read all the text or content of the file using the read() method. You can also use
the readline() to read file line by line or the readlines() to read all lines.
For example, content = [Link]()
4. Close file after completing the read operation
We need to make sure that the file will be closed properly after completing the file operation.
Use [Link]() to close a file.
Different modes for reading the file

File Mode Definition

r The default mode for opening a file to read the contents of a text file.

Open a file for both reading and writing. The file pointer will be placed at the beginning
r+
of the file.

Opens the file for reading a file in binary format. The file pointer will be placed at the
rb
beginning of the file.

Opens a file for both writing as well as reading. The file pointer will be placed in the
w+
beginning of the file. For an existing file, the content will be overwritten.

Open the file for both the reading and appending. The pointer will be placed at the end
a+
of the file and new content will be written after the existing content.
 File Read Methods

Method When to Use?

Returns the entire file content and it accepts the optional size parameter that mentions
the bytes to read from the file.
read() with open('[Link]') as f:
print([Link](17))

The readline() method reads a single line from a file at a time. Accepts optional size
parameter that mentions how many bytes to return from the file.
readline() with open('[Link]') as f:
print([Link]())

The readlines() method returns a list of lines from the file.

readlines() with open(‘[Link]') as f:
lines = [Link]()
# read file with absolute path
try:
fp = open(r"E:\demos\files\read_demo.txt", "r")
print([Link]())
[Link]()
except FileNotFoundError:
print("Please check the path")
Reading a File Using the with Statement
with open(__file__, accessmode) as f:
Main advantages of opening a file using ‘with’ statement
 The with statement simplifies exception handling by
encapsulating common preparation and cleanup tasks.
 This also ensures that a file is automatically closed after
leaving the block.
 As the file is closed automatically it ensures that all the
resources that are tied up with the file are released.
# Reading files using 'with'
with open('read_demo.txt', 'r') as file:
print([Link]())
 Visualizing Data from Text Files (.txt)

 [Link]

import [Link] as plt

import numpy as np
X, Y = [Link]('[Link]', delimiter=',', unpack=True)
[Link](X, Y)
[Link]('Line Graph using NUMPY')
[Link]('X')
[Link]('Y')
[Link]()
 CSV Files (.csv)
CSV file (Comma Separated Values file) is a type of plain text file that
uses specific structuring to arrange tabular data. Because it’s a plain text
file, it can contain only actual text data.

Modes:

 ‘r’ – to read an existing file,

 ‘w’ – to create a new file if the given file doesn’t exist and write to it

 ‘a’ – to append to existing file content,

 ‘+’ – to create a new file for reading and writing

Read CSV files in Python
 Using the csv module

import csv

# Open the CSV file in read mode

with open('[Link]', 'r') as csvfile:

# Create a reader object

csv_reader = [Link](csvfile)

# Iterate through the rows in the CSV file

for row in csv_reader:

# Access each element in the row
print(row)
Read CSV file using .readlines() function
with open('Salary_Data.csv') as file:
content = [Link]()
header = content[:1]
rows = content[1:]
print(header)
print(rows)
Read CSV file using Pandas
import pandas as pd
# Read the CSV file into a DataFrame
df = pd.read_csv('[Link]')
df
# Access data in the DataFrame using column names
or indexing
print(df['column_name'])
print([Link][0])
Read CSV file using [Link]

import csv
with open('Salary_Data.csv', 'r') as csvfile:
reader = [Link](csvfile)
for row in reader:
print(row)
 Visualizing Data from CSV Files
import [Link] as plt
import csv
x = []
y = []
with open('[Link]','r') as csvfile:
plots = [Link](csvfile, delimiter = ',')
for row in plots:
[Link](row[0])
[Link](int(row[2]))
[Link](x, y, color = 'g', width = 0.72, label = "Age")
[Link]('Names')
[Link]('Ages')
[Link]('Ages of different persons')
[Link]()
[Link]()
Visualizing Student marks in different subjects using a pie plot

import [Link] as plt

import csv
Subjects = []
Scores = []
with open('[Link]', 'r') as csvfile:
lines = [Link](csvfile, delimiter = ',')
for row in lines:
[Link](row[0])
[Link](int(row[1]))
[Link](Scores,labels = Subjects,autopct = '%.2f%%')
[Link]('Marks of a Student', fontsize = 20)
[Link]()
 Read XML files in Python
 XML stands for Extensible Markup Language.
 It is a markup language and file format that helps in storing and transporting of
data.
 It is designed to carry data and not just to display data as it is self descriptive.
 It was formed from extracting the properties of SGML (Standard Generalized
Markup Language).
 It supports exchanging of information between computer systems. They can be
websites, databases, and any third-party applications.
 It consists of predefined rules which makes it easy to transmit data as XML files
over any network.
 To read the local XML file in Python
we can give the absolute path of the file:
import pandas as pd
df = pd.read_xml('[Link]')

 We can read remote files the same way:

import pandas as pd
df = pd.read_xml( f'[Link]
 What is JSON
 JSON (an acronym for JavaScript Object Notation) is a data-interchange
format and is most commonly used for client-server communication.
 A JSON is an unordered collection of key and value pairs, resembling
Python's native dictionary.
 Keys are unique Strings that cannot be null.
 Values can be anything from a String, Boolean, Number, list, or even
null.
 A JSON can be represented by a String enclosed within curly braces
with keys and values separated by a colon, and pairs separated by a
comma
 Why Convert JSON to Tabular Format?
 Data Analysis: Many data analysis tools and libraries work
more efficiently with tabular data.

 Readability: Tabular formats are often easier for humans to

read and interpret, especially for large datasets.

 Compatibility: Some systems or applications may require data

in a tabular format for import or processing.

 Visualization: Creating charts and graphs is typically easier

with tabular data.
 Various methods to convert JSON to tabular
format
Method 1: Using Python and Pandas
Step 1: Install Required Libraries
pip install pandas
Step 2: Read JSON Data
import pandas as pd
# Read JSON file
df = pd.read_json('[Link]')
# If the JSON is nested, you may need to normalize it
df = pd.json_normalize(df)

Step 3: Convert to Tabular Format

Once the data is in a Pandas DataFrame, you can easily convert it to various tabular formats:
 On macOS with Homebrew:
brew install jq csvkit
 On Ubuntu:
sudo apt-get install jq csvkit
 Method 2: Using Online Converters
Some popular online converters include:
 JSON to CSV Converter
 ConvertCSV
 JSON Editor Online
 To use these tools:
 Copy your JSON data or upload your JSON file.
 Select the desired output format (CSV, Excel, etc.).
 Click the convert button.
 Download the resulting tabular file.
Method 3: Using Command-Line Tools
For users comfortable with the command line, several tools can
convert JSON to tabular format directly from the terminal.
 Using jq and csvkit
On macOS with Homebrew:
brew install jq csvkit
On Ubuntu:
sudo apt-get install jq csvkit
 Use the following command to convert JSON to CSV:

Replace .field1, .field2, etc., with the actual field names from your JSON data.
 Method 4: Using Database System
Using PostgreSQL
 PostgreSQL has excellent support for JSON data types and provides
functions to convert JSON to tabular format.
 Create a table to store the JSON data:

 Import the JSON data into the table.

 Use JSON functions to extract and convert the data:
Method 5: Using Spreadsheet Software
In Microsoft Excel:
Go to the “Data” tab and click “Get Data” > “From File” > “From
JSON”.
Select your JSON file and click “Import”.
In the Power Query Editor, you can then transform the data as needed.
Load the data into your spreadsheet.
In Google Sheets:
[Link] the IMPORTDATA function to import JSON from a URL:

2. Use Apps Script to parse more complex JSON structures.

Best Practices for JSON to Tabular Conversion
 Handle Nested Structures: JSON can contain nested objects and
arrays. Decide how to flatten these structures into columns.

 Deal with Missing Data: JSON objects may not always contain all
fields. Decide how to handle missing data in your tabular format.

 Preserve Data Types: Ensure that data types (numbers, dates,

booleans) are correctly preserved during the conversion.

 Handle Large Datasets: For very large JSON files, consider using
streaming parsers or database systems to avoid memory issues.

 Validate the Output:Always verify that the converted tabular data

accurately represents the original JSON data.
Challenges in JSON to Tabular Conversion
 Loss of Hierarchy: Tabular formats are inherently flat, so you may lose some
of the hierarchical structure present in JSON.

 Handling Arrays: JSON arrays can be tricky to represent in tabular format.

You may need to decide between creating multiple rows or concatenating
array elements into a single cell.

 Inconsistent Structures: If your JSON data doesn’t have a consistent

structure across all objects, conversion can be more complex.

 Data Type Inference: Automatic conversion tools may not always correctly
infer data types, especially for dates or complex numbers.
Example
import pandas as pd
import json
json_data = """ [ { "id": 1, "name": "Alice", "details": { "age": 30, "city": "New
York" }, "hobbies": ["reading", "hiking"] },
{ "id": 2, "name": "Bob", "details": { "age": 25, "city": "London" }, "hobbies":
["coding", "gaming"] } ] """

# Load JSON data

data = [Link](json_data)

# Normalize the JSON to handle nested structures

df = pd.json_normalize(data)
# Print the resulting DataFrame
print(df)
QUESTIONS:
 Why is it important to convert JSON data to tabular format? Discuss
various methods used for this conversion, along with relevant tools or
programming examples.
 Describe the steps to import data from .csv, .xml, and .txt files and
visualize it using Python (e.g., Pandas and Matplotlib).
 Write a Python function that loads JSON data from a file and displays
selected fields in tabular form.
 Describe in detail about challenges you may encounter while
converting JSON to tabular format, and suggest possible solutions for
each.
 What is the purpose of including computations in a data table? Give
two examples of computations that can be performed.