SHREE UTTAR GUJARAT BCA COLLEGE

Unit-2 Automate EDA (Exploratory Data Analysis)

2.1 Python Libraries to Automate Exploratory Data Analysis

2.1.1 Pandas and Numpy
2.2 Regression
2.2.1 Characteristics of Regression
2.2.2 Dependent and Independent variables
2.2.3 Covariance and Correlation
2.3 Machine Learning Basics:
2.3.1 Concepts of Machine learning
[Link] Understanding machine learning
[Link] Benefit of machine learning
[Link] Machine learning life cycle
2.4 Types of Machine Learning:
2.4.1 Supervised and Unsupervised Learning
2.4.2 Applications of ML in real-world scenarios

2.1 Python Libraries to Automate Exploratory Data Analysis (EDA)

Exploratory Data Analysis (EDA) is the first step in understanding your
data before building models. It helps you find patterns, spot errors, and
summarize the main features of the data. In Python, we use special
libraries (tools) to make EDA faster and easier.

Here is a simple diagram showing the steps in the data analysis process,
where EDA plays a key role:
Page 1
 SHREE UTTAR GUJARAT BCA COLLEGE

2.1.1 Pandas and NumPy

These are the two most important Python libraries for handling data.
NumPy: It helps with fast mathematical operations on large amounts
of numbers (arrays). Think of it as a super-fast calculator for lists of
numbers.
Pandas: Built on top of NumPy, it is used for working with tables of
data (called DataFrames). You can easily load data from files (like CSV
or Excel), clean it, filter rows, calculate summaries, etc. grouping data.

Here are the logos of Pandas and NumPy:

Page 2
 SHREE UTTAR GUJARAT BCA COLLEGE

Simple Example in Python (you can try this later):

python
import pandas as pd Import Pandas
import numpy as np Import NumPy
data = pd.read_csv('your_file.csv') Load data
print([Link]()) Show first 5 rows
print([Link]()) Automatic summary: mean, min, max, etc.
2.2 Regression
Regression is a statistical method used to predict a numerical value
based on other data. It finds the relationship between variables.
For example: Can we predict a student's marks (based on hours studied)?
Or house price (based on size)?
The most common type is Linear Regression, where the relationship is
shown as a straight line.
Here are some simple examples of linear regression graphs (scatter
points with a best-fit line):

Page 3
 SHREE UTTAR GUJARAT BCA COLLEGE

2.1 Characteristics of Regression

- It predicts a continuous value (like height, weight, price – not
categories like "yes/no").
- It shows how one or more inputs affect the output.
- The goal is to find the "best-fit" line (or curve) that minimizes
errors in prediction.
- Useful for understanding relationships and making predictions.

Page 4
 SHREE UTTAR GUJARAT BCA COLLEGE

2.2.2 Dependent and Independent Variables

- Independent Variable (X): The input or cause. We control or change
this. Also called "predictor" or "feature".
- Dependent Variable (Y): The output or result. It depends on the
independent variable. Also called "target" or "response".
Example: In predicting house price:
- Independent: Size of house (sq. feet)
- Dependent: Price of house
Here is a simple diagram explaining independent and dependent
variables:

Page 5
 SHREE UTTAR GUJARAT BCA COLLEGE

2.2.3 Covariance and Correlation

These measure how two variables change together.
- Covariance: Tells if two variables move in the same direction
(positive) or opposite (negative). But its value depends on the scale of
data, so it's hard to compare.
- Correlation: A standardized version of covariance. It ranges from -1
to +1.
- +1: Perfect positive relationship
- 0: No relationship
- -1: Perfect negative relationship
Correlation is better because it's scale-independent and easy to interpret.
Here are diagrams comparing covariance and correlation, and showing
different types of correlation with scatter plots:

Page 6
SHREE UTTAR GUJARAT BCA COLLEGE

Page 7
 SHREE UTTAR GUJARAT BCA COLLEGE

In Pandas, you can easily calculate them:

python
[Link]() Covariance
[Link]() Correlation

2.3 Machine Learning Basics

Machine Learning (ML) is a part of Artificial Intelligence (AI) where
computers learn from data and improve on their own without being
explicitly programmed for every task. Instead of writing rules, we give
the computer data and let it find patterns.
Here are simple diagrams showing the basic process of machine learning
(data → model → prediction):

Page 8
SHREE UTTAR GUJARAT BCA COLLEGE

Page 9
 SHREE UTTAR GUJARAT BCA COLLEGE

2.3.1 Concepts of Machine Learning

[Link] Understanding Machine Learning
Machine Learning allows computers to learn from experience (data).
There are three main ways:
- It uses algorithms (step-by-step methods) to find hidden patterns in
data.
- The computer makes predictions or decisions based on new data.
- Example: Teaching a computer to recognize cats in photos by showing
it thousands of cat pictures.

Page 10
 SHREE UTTAR GUJARAT BCA COLLEGE

[Link] Benefits of Machine Learning

Machine Learning helps in many ways:
- Automation: Handles repetitive tasks quickly (e.g., sorting emails into
spam/not spam).
- Better Decisions: Analyzes huge data faster than humans.
-Personalization: Recommends movies or products you like (like
Netflix or Amazon).
- Prediction: Forecasts weather, stock prices, or diseases.
- Efficiency: Saves time and money in businesses.
Here are info graphic showing key benefits:

Page 11
 SHREE UTTAR GUJARAT BCA COLLEGE

[Link] Machine Learning Life Cycle

The ML process follows a cycle (like a project steps). It is repeated to
improve the model.
Main steps:
1. Problem Definition: Understand what to solve (e.g., predict house
prices).
2. Data Collection: Gather relevant data.
3. Data Preparation: Clean and organize data (remove errors, handle
missing values).
4. Exploratory Data Analysis (EDA): Understand data with summaries
and charts.
5. Model Building: Choose and train an algorithm.
6. Model Evaluation: Test how good the model is.
7. Deployment: Use the model in real applications.
8. Monitoring: Check and update the model over time.
Here are clear diagrams of the Machine Learning Life Cycle:

Page 12
SHREE UTTAR GUJARAT BCA COLLEGE

Page 13
SHREE UTTAR GUJARAT BCA COLLEGE

Page 14
 SHREE UTTAR GUJARAT BCA COLLEGE

2.4 Types of Machine Learning

There are mainly three types: Supervised, Unsupervised, and
Reinforcement Learning. Here we focus on the first two.

2.4.1 Supervised and Unsupervised Learning

- Supervised Learning: The data has labels (answers). The model learns
from input-output pairs to predict for new data.
- Example: Predicting exam marks (output) based on study hours
(input). Or classifying emails as spam/not spam.
- Types: Classification (categories) and Regression (numbers).

- Unsupervised Learning: No labels. The model finds hidden patterns or

groups in data on its own.
- Example: Grouping customers by shopping habits (customer
segmentation). Or finding unusual transactions (fraud detection).

Here are diagrams comparing Supervised and Unsupervised Learning:

Page 15
SHREE UTTAR GUJARAT BCA COLLEGE

Page 16
SHREE UTTAR GUJARAT BCA COLLEGE

Page 17
 SHREE UTTAR GUJARAT BCA COLLEGE

2.4.2 Applications of ML in Real-World Scenarios

Machine Learning is used everywhere today:
- Healthcare: Predicting diseases from scans, recommending treatments.
- Finance: Detecting fraud in transactions, stock price prediction.
- E-commerce: Product recommendations, chatbots for customer service.
- Transportation: Self-driving cars, traffic prediction.
- Entertainment: Netflix suggestions, Spotify playlists.
- Social Media: Face recognition, spam filters.
- Agriculture: Predicting crop yields, detecting pests.

Here are visuals showing real-world ML applications:

Page 18
SHREE UTTAR GUJARAT BCA COLLEGE

Page 19