Unit 01 machine learning

Machine Learning -
Machine learning is a branch of Artificial Intelligence that focuses on
developing models and algorithms that let computers learn from data without
being explicitly programmed for every task. In simple words, ML teaches
systems to think and understand like humans by learning from the data.

Machine Learning is mainly divided into three core types:

 Supervised Learning: Trains models on labeled data to predict or classify
new, unseen data.
 Unsupervised Learning: Finds patterns or groups in unlabeled data, like
clustering or dimensionality reduction.
 Reinforcement Learning: Learns through trial and error to maximize
rewards, ideal for decision-making tasks.
Module 1: Machine Learning Pipeline
This section covers preprocessing, exploratory data analysis and model
evaluation to prepare data, uncover insights and build reliable models.

1. Data Preprocessing

 ML workflow
 Data Cleaning
 Data Preprocessing in Python
 Feature Scaling
 Feature Extraction
 Feature Engineering
 Feature Selection Techniques

2. Exploratory Data Analysis

 Exploratory Data Analysis

 Exploratory Data Analysis in Python
 Advance EDA
 Time Series Data Visualization
3. Model Evaluation

 Regularization in Machine Learning

 Confusion Matrix
 Precision, Recall and F1-Score
 AUC-ROC Curve
 Cross-validation
 Hyperparameter Tuning

✍️ Types of Machine Learning System-

🔹 Introduction

Machine Learning (ML) is a branch of Artificial Intelligence that enables systems to learn from
data and improve performance without explicit programming. ML systems are classified based
on how they learn, process data, and generalize knowledge.

🔹 1. Based on Supervision
(a) Supervised Learning

In supervised learning, the model is trained using labeled data, where both input and output are
known. The goal is to learn a mapping function from input to output.

 Types of problems:
o Classification (e.g., spam detection)
o Regression (e.g., house price prediction)
 Example: Predicting exam results based on study hours.

(b) Unsupervised Learning

In unsupervised learning, the model works with unlabeled data and tries to identify hidden
patterns or structures.

 Techniques:
o Clustering (grouping similar data)
o Association (finding relationships)
 o Example: Customer segmentation in marketing.

(c) Semi-Supervised Learning

This is a combination of supervised and unsupervised learning, using a small amount of labeled
data and a large amount of unlabeled data.

 Example: Image classification with limited labeled images.

(d) Reinforcement Learning

In reinforcement learning, an agent learns by interacting with an environment and receiving

rewards or penalties.

 Key components: Agent, Environment, Reward

 Example: Game playing, robotics

🔹 2. Based on Learning Style

(a) Batch Learning (Offline Learning)

The model is trained using the entire dataset at once. It does not learn continuously after
deployment.

 Advantage: Simpler implementation

 Disadvantage: Cannot adapt to new data

(b) Online Learning

The model learns incrementally from new data as it arrives.

 Advantage: Adapts to changes quickly

 Example: Real-time recommendation systems

🔹 3. Based on Generalization Approach

(a) Instance-Based Learning

The model memorizes training data and compares new instances with stored examples.

 Example: K-Nearest Neighbors (KNN)

(b) Model-Based Learning

The model builds a generalized representation of the data and uses it for predictions.

 Example: Linear Regression

🔹 4. Based on Model Structure

(a) Parametric Models

These models have a fixed number of parameters.

 Advantages: Simple and fast

 Disadvantages: Less flexible
 Example: Linear Regression

(b) Non-Parametric Models

These models do not assume a fixed number of parameters and can grow with data.

 Advantages: Flexible
 Disadvantages: Requires more data and computation
 Example: Decision Trees

✍️ Challenges in Machine Learning-

🔹 Introduction

Machine Learning (ML) enables systems to learn from data and make predictions, but building
an effective ML system is not easy. There are several challenges related to data, models,
computation, and real-world deployment that affect performance and reliability.

🔹 1. Insufficient Training Data

ML models require a large amount of data to learn patterns effectively.

 Problem: Small datasets lead to poor learning

 Impact: Low accuracy and weak generalization
🔹 2. Poor Quality Data

Data may contain:

 Noise (errors)
 Missing values
 Inconsistent formats
 Impact: Incorrect predictions and unreliable models

🔹 3. Non-Representative Data

If training data does not represent real-world scenarios:

 Problem: Model learns biased patterns

 Example: Face recognition trained on limited demographics

🔹 4. Overfitting

The model performs well on training data but poorly on new data.

 Cause: Model too complex

 Result: High variance

🔹 5. Underfitting

The model is too simple to capture patterns in data.

 Result: Poor performance on both training and test data

 Cause: High bias

🔹 6. Feature Engineering Challenges

Selecting the right features is difficult.

 Problem: Irrelevant or redundant features reduce accuracy

 Solution: Feature selection and extraction techniques
🔹 7. Curse of Dimensionality

When the number of features increases:

 Data becomes sparse

 Model becomes complex
 Impact: Reduced performance

🔹 8. Computational Complexity

Training ML models requires high computational power.

 Issue: Large datasets need more time and memory

 Example: Deep learning models require GPUs

🔹 9. Model Interpretability

Some models (like deep neural networks) are difficult to understand.

 Problem: Lack of transparency

 Impact: Difficult to trust decisions

🔹 10. Data Privacy and Security

ML systems often use sensitive data.

 Concern: Data leakage, misuse

 Example: Medical or financial data

🔹 11. Bias and Fairness Issues

Models can inherit bias from data.

 Problem: Discrimination in predictions

 Example: Hiring systems favoring certain groups
🔹 12. Dynamic Environments (Concept Drift)

Real-world data changes over time.

 Problem: Model becomes outdated

 Solution: Continuous retraining

Supervised Learning in Machine Learning

🔹 Introduction

Supervised Learning is one of the most important types of Machine Learning in which the model
is trained using labeled data. Each training example consists of an input along with its
corresponding correct output. The goal is to learn a mapping function that can predict outputs for
new unseen data.

🔹 Definition

Supervised Learning is a technique where a model learns from a dataset (X,Y)(X, Y)(X,Y),
where:

 XXX = Input features

 YYY = Target/output labels

The model tries to approximate a function:

Y=f(X)Y = f(X)Y=f(X)

🔹 Working of Supervised Learning

1. Collect labeled data

2. Split data into training and testing sets
3. Train the model using training data
4. Evaluate using test data
5. Use the model for prediction
🔹 Types of Supervised Learning
(a) Classification

 Output is categorical (labels)

 Predicts class/category

📌 Examples:

 Spam vs Not Spam

 Disease detection (Yes/No)

(b) Regression

 Output is continuous (numeric values)

📌 Examples:

 House price prediction

 Temperature forecasting

🔹 Common Algorithms

 Linear Regression
 Logistic Regression
 Decision Trees
 Support Vector Machines (SVM)
 K-Nearest Neighbors (KNN)
 Neural Networks

🔹 Advantages

1. Easy to understand and implement

2. High accuracy with sufficient data
3. Clear objective (predict known outputs)

🔹 Disadvantages

1. Requires large labeled datasets

2. Data labeling is time-consuming
3. Can suffer from overfitting
 4. Not suitable when labeled data is unavailable

🔹 Applications

 Email spam filtering

 Image classification
 Speech recognition
 Medical diagnosis
 Stock price prediction

🔹 Example

If we want to predict house prices:

 Input: Size, location, number of rooms

 Output: Price

The model learns from past data and predicts price for new houses.

✍️ Unsupervised Learning
🔹 Introduction

Unsupervised Learning is a type of Machine Learning in which the model is trained on

unlabeled data. Unlike supervised learning, there are no predefined outputs. The goal is to
discover hidden patterns, structures, or relationships within the data.

🔹 Definition

Unsupervised Learning is a technique where a model analyzes input data XXX without
corresponding output labels and tries to learn the underlying structure of the data.

🔹 Working of Unsupervised Learning

1. Collect unlabeled data

2. Identify patterns or similarities
3. Group or organize data based on structure
4. Extract meaningful insights
🔹 Types of Unsupervised Learning
(a) Clustering

Clustering groups similar data points into clusters based on similarity.

📌 Examples:

 Customer segmentation
 Grouping news articles

📌 Algorithms:

 K-Means
 Hierarchical Clustering
 DBSCAN

(b) Association Rule Learning

Finds relationships between variables in large datasets.

📌 Example:

 Market basket analysis (e.g., “people who buy bread also buy butter”)

📌 Algorithms:

 Apriori Algorithm
 FP-Growth

(c) Dimensionality Reduction

Reduces the number of features while preserving important information.

📌 Purpose:

 Simplify data
 Improve performance

📌 Algorithms:

 Principal Component Analysis (PCA)

  t-SNE

🔹 Advantages

1. No need for labeled data

2. Useful for discovering hidden patterns
3. Helps in data exploration and preprocessing
4. Can work with large datasets

🔹 Disadvantages

1. Difficult to evaluate results

2. No clear output labels
3. Results may be less accurate
4. Requires domain knowledge for interpretation

🔹 Applications

 Customer segmentation
 Recommendation systems
 Anomaly detection (fraud detection)
 Image compression
 Pattern recognition

🔹 Example

A company wants to group its customers based on purchasing behavior:

 Input: Purchase history

 Output: Customer groups (clusters)

This helps in targeted marketing.

✍️ Training–Test Loss Tradeoff, Estimating
Risk & Sampling Distribution-
🔹 Introduction

In statistical learning, the goal is to build models that perform well not only on training data but
also on unseen data. Concepts like training vs test loss, risk estimation, and sampling
distribution of estimators are fundamental to evaluating and improving model performance.

🔹 1. Training Loss vs Test Loss Tradeoff

📌 Definitions

 Training Loss: Error calculated on training data

 Test Loss: Error calculated on unseen (test) data

📌 Behavior of Loss

 As model complexity increases:

o Training loss decreases continuously
o Test loss first decreases, then increases

👉 This leads to the concept of overfitting and underfitting

📌 Tradeoff Explanation

 Underfitting (High Bias)

o Model too simple
o High training & test loss
 Overfitting (High Variance)
o Model too complex
o Low training loss but high test loss
📌 Goal

Choose a model with minimum test loss for good generalization.

🔹 2. Bias–Variance Tradeoff (Related

Concept)
E[(Y−f^(X))2]=Bias2+Variance+Irreducible ErrorE[(Y - \hat{f}(X))^2] = Bias^2 + Variance +
Irreducible\ ErrorE[(Y−f^(X))2]=Bias2+Variance+Irreducible Error

📌 Explanation

 Bias → error due to wrong assumptions

 Variance → sensitivity to training data
 Irreducible error → noise in data

👉 A good model balances bias and variance.

🔹 3. Estimating Risk
📌 Definition

Risk measures how well a model performs on new data.

🔸 Expected Risk (True Risk)

R(f)=E[L(Y,f(X))]R(f) = E[L(Y, f(X))]R(f)=E[L(Y,f(X))]

 Average loss over entire data distribution

 Cannot be computed directly (unknown distribution)

🔸 Empirical Risk

R^(f)=1n∑i=1nL(yi,f(xi))\hat{R}(f) = \frac{1}{n} \sum_{i=1}^{n} L(y_i, f(x_i))R^(f)=n1

∑i=1nL(yi,f(xi))
  Calculated using training data
 Used in practice

👉 Machine learning aims to minimize empirical risk (Empirical Risk Minimization - ERM)

🔹 4. Statistical Sampling Distribution of an

Estimator
📌 Definition

The sampling distribution of an estimator is the probability distribution of that estimator over
repeated samples from the population.

📌 Example: Sample Mean

 Estimator:

Xˉ=1n∑Xi\bar{X} = \frac{1}{n} \sum X_iXˉ=n1∑Xi

Properties:

 Mean = Population mean (μ)(\mu)(μ)

 Variance = σ2/n\sigma^2 / nσ2/n

📌 Importance

 Helps understand variability of estimates

 Used in confidence intervals and hypothesis testing
 Larger sample size → more accurate estimates

🔹 5. Properties of a Good Estimator

1. Unbiasedness

E[θ^]=θE[\hat{\theta}] = \thetaE[θ^]=θ
2. Consistency
o Converges to true value as sample size increases
3. Efficiency
o Minimum variance among estimators