UNIT-I Introduction to Machine Learning
UNIT-I INTRODUCTION TO MACHINE
LEARNING
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 1
UNIT-I Introduction to Machine Learning
Syllabus
- 1.1 Basics of Machine Learning
1.2 Types of Machine Learning
1.3 Applications of Machine Learning
1.4 Challenges in Machine Learning
1.5 Python for Machine Learning
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 2
UNIT-I Introduction to Machine Learning
1.1 Basics of Machine Learning
1.1.1 Definition of Machine Learning
Machine Learning is a branch of Artificial Intelligence that focuses on building systems that can
learn automatically from data and past experience, identify patterns, and make decisions with
minimal human intervention.
Unlike traditional programming, where rules are explicitly written by a programmer, machine
learning systems derive rules automatically from data.
Formal Definition (Tom Mitchell):
A computer program is said to learn from experience (E) with respect to some task (T) and
performance measure (P) if its performance at task T, as measured by P, improves with
experience E.
Example:
• Task (T): Predicting house prices
• Experience (E): Historical housing data
• Performance (P): Mean Squared Error (MSE)
1.1.2 How Machine Learning Works
The machine learning process follows a cyclic and iterative workflow:
Step 1: Data Collection
• Data is gathered from:
o Databases
o Sensors
o Web applications
o Logs
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 3
UNIT-I Introduction to Machine Learning
o User interactions
• Data can be:
o Structured (tables)
o Semi-structured (JSON, XML)
o Unstructured (text, images)
Step 2: Data Preparation
This is one of the most critical stages.
Includes:
• Removing noise
• Handling missing values
• Normalization
• Feature selection
• Feature extraction
Poor data quality leads to poor model performance.
Step 3: Model Training
• An ML algorithm is selected
• The model learns patterns from training data
• Learning is achieved by minimizing error or maximizing reward
Step 4: Model Evaluation
• Model is tested on unseen data
• Performance metrics are used
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 4
UNIT-I Introduction to Machine Learning
• If accuracy is not acceptable, retraining is done
Step 5: Deployment
• Model is deployed in real-world systems
• Continuous learning may occur using new data
1.1.3 Traditional Programming vs Machine Learning
Feature Traditional Programming Machine Learning
Logic Explicitly written Learned from data
Flexibility Low High
Adaptability Manual updates Automatic improvement
Accuracy Rule-dependent Data-dependent
Example Calculator Recommendation system
Explanation:
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 5
UNIT-I Introduction to Machine Learning
• In traditional systems, rules are fixed.
• In ML systems, rules evolve as data increases.
Example:
• Task (T): Email spam classification
• Experience (E): Previously labeled emails
• Performance (P): Accuracy of spam detection
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 6
UNIT-I Introduction to Machine Learning
1.1.4 Role of Machine Learning in Artificial Intelligence
Machine Learning is the core driving force behind modern AI systems.
ML enables AI to:
• Learn from experience
• Adapt to new environments
• Improve decision-making
Examples:
• Speech recognition
• Autonomous vehicles
• Face recognition
Without ML, AI systems would remain static and rule-based.
1.1.5 Role of Machine Learning in Data Science
In Data Science, ML is used to:
• Discover patterns
• Build predictive models
• Automate decision processes
ML bridges the gap between raw data and intelligent insights.
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 7
UNIT-I Introduction to Machine Learning
1.2 Types of Machine Learning
Machine Learning is broadly classified into three major types:
1.2.1 Supervised Learning
Definition
Supervised Learning is a type of ML where the model is trained using a labeled dataset, i.e.,
data that contains input–output pairs.
The dataset acts as a teacher guiding the learning process .
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 8
UNIT-I Introduction to Machine Learning
General Objective
To learn a function:
𝑓(𝑋) → 𝑌
that can correctly predict output 𝑌for unseen input data 𝑋.
Working Principle
1. Training data consists of:
o Inputs (features)
o Targets (labels)
2. Model learns mapping:
𝑓(𝑋) → 𝑌
3. When new input data is given, the model predicts output.
Examples of Supervised Learning
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 9
UNIT-I Introduction to Machine Learning
A. Classification
• Output variable is categorical
• Examples:
o Spam / Not Spam
o Disease / No Disease
o Male / Female
1. K-Nearest Neighbors (KNN)
Definition
KNN is a non-parametric, instance-based learning algorithm that classifies a data point based
on the majority class of its K nearest neighbors.
Working Principle
1. Choose value of K.
2. Calculate distance between test point and all training points.
3. Select K nearest neighbors.
4. Assign the most common class.
Distance Metrics
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 10
UNIT-I Introduction to Machine Learning
• Euclidean distance
• Manhattan distance
• Minkowski distance
• Hamming distance
Applications
• Pattern recognition
• Recommendation systems
Advantages
• Simple and intuitive
• No training phase
Disadvantages
• Computationally expensive
• Sensitive to noise and K value
2. Naive Bayes Classifier
Definition
Naive Bayes is a probabilistic classifier based on Bayes’ Theorem with the assumption that
features are independent.
Bayes’ Theorem
𝑃(𝐵 ∣ 𝐴)𝑃(𝐴)
𝑃(𝐴 ∣ 𝐵) =
𝑃(𝐵)
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 11
UNIT-I Introduction to Machine Learning
Working Principle
1. Calculate prior probabilities.
2. Compute likelihood for each feature.
3. Predict class with maximum posterior probability.
Types
• Gaussian Naive Bayes
• Multinomial Naive Bayes
• Bernoulli Naive Bayes
Applications
• Spam filtering
• Sentiment analysis
• Document classification
Advantages
• Fast and scalable
• Works well with high-dimensional data
Disadvantages
• Strong independence assumption
• Less accurate when features are correlated
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 12
UNIT-I Introduction to Machine Learning
3. Decision Tree
Definition
Decision Tree is a supervised learning algorithm that uses a tree-like structure to make
decisions based on feature values.
Structure
• Root node
• Internal decision nodes
• Leaf nodes (class labels)
Splitting Criteria
• Entropy
• Information Gain
• Gain Ratio
• Gini Index
Algorithms
• ID3
• C4.5
• CART
Applications
• Medical diagnosis
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 13
UNIT-I Introduction to Machine Learning
• Customer churn prediction
Advantages
• Easy to understand
• Handles both numerical and categorical data
Disadvantages
• Overfitting
• Sensitive to data changes
4. Support Vector Machine (SVM)
Definition
SVM is a supervised learning algorithm that finds an optimal hyperplane that best separates
data points of different classes.
Key Concepts
• Hyperplane
• Support vectors
• Margin
Kernel Functions
• Linear
• Polynomial
• Radial Basis Function (RBF)
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 14
UNIT-I Introduction to Machine Learning
• Sigmoid
Working Principle
• Maximizes margin between classes.
• Uses kernel trick for non-linear data.
Applications
• Image classification
• Bioinformatics
• Text classification
Advantages
• High accuracy
• Effective in high-dimensional spaces
Disadvantages
• Computationally expensive
• Hard to tune parameters
5. Random Forest (Ensemble Supervised Algorithm)
Definition
Random Forest is an ensemble learning method that builds multiple decision trees and
combines their outputs.
Working Principle
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 15
UNIT-I Introduction to Machine Learning
1. Random sampling of data.
2. Build multiple decision trees.
3. Aggregate predictions using voting or averaging.
Applications
• Fraud detection
• Recommendation systems
Advantages
• Reduces overfitting
• High accuracy
Disadvantages
• Less interpretable
• High computation cost
Summary Table
Algorithm Type Output Key Feature
Linear Regression Regression Continuous Linear model
Logistic Regression Classification Binary Probability-based
KNN Classification Categorical Distance-based
Naive Bayes Classification Categorical Probabilistic
Decision Tree Classification Categorical Rule-based
SVM Classification Categorical Margin maximization
Random Forest Classification Categorical Ensemble
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 16
UNIT-I Introduction to Machine Learning
B. Regression
• Output variable is continuous
• Examples:
o House price prediction
o Salary estimation
o Temperature forecasting
1. Simple Linear Regression
Definition
Simple Linear Regression is a supervised learning algorithm used to model the linear
relationship between one independent variable (X) and one dependent variable (Y).
It assumes that the relationship between X and Y can be represented by a straight line.
Mathematical Model
𝑌 = 𝑏0 + 𝑏1 𝑋
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 17
UNIT-I Introduction to Machine Learning
Where:
• 𝑌→ Dependent variable (target)
• 𝑋→ Independent variable (feature)
• 𝑏0 → Intercept
• 𝑏1 → Slope (regression coefficient)
Working Principle
1. Plot data points on X–Y plane.
2. Find the best-fit line such that the sum of squared errors between actual and predicted
values is minimized.
3. The model predicts Y for any new X.
Assumptions
• Linear relationship
• No multicollinearity
• Homoscedasticity
• Normally distributed errors
Applications
• Salary prediction
• Sales forecasting
• Temperature estimation
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 18
UNIT-I Introduction to Machine Learning
Advantages
• Simple and interpretable
• Low computational cost
Disadvantages
• Works only for linear relationships
• Sensitive to outliers
2. Multiple Linear Regression
Definition
Multiple Linear Regression extends simple linear regression by using more than one
independent variable to predict the dependent variable.
Mathematical Model
𝑌 = 𝑏0 + 𝑏1 𝑋1 + 𝑏2 𝑋2 + ⋯ + 𝑏𝑛 𝑋𝑛
Working Principle
• Each feature contributes independently to the prediction.
• Model learns multiple coefficients corresponding to multiple features.
Applications
• House price prediction (area, location, rooms)
• Business analytics
Advantages
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 19
UNIT-I Introduction to Machine Learning
• Handles complex relationships better than simple regression
Disadvantages
• Multicollinearity issues
• Harder to interpret
3. Logistic Regression
Definition
Logistic Regression is a classification algorithm used to predict binary or multiclass outcomes
using a probabilistic approach.
Despite the name, it is used for classification, not regression.
Sigmoid Function
1
𝜎(𝑧) =
1 + 𝑒 −𝑧
The output lies between 0 and 1, representing probability.
Working Principle
1. Linear combination of inputs is computed.
2. Sigmoid function converts output into probability.
3. A threshold (usually 0.5) is used for classification.
Types
• Binary Logistic Regression
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 20
UNIT-I Introduction to Machine Learning
• Multinomial Logistic Regression
Applications
• Disease prediction
• Spam detection
• Credit approval
Advantages
• Probabilistic interpretation
• Efficient for linearly separable data
Disadvantages
• Not suitable for complex nonlinear patterns
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 21
UNIT-I Introduction to Machine Learning
1.2.2 Unsupervised Learning
Definition
Unsupervised Learning is a type of ML where the model learns from unlabeled data and
identifies hidden patterns or structures automatically .
Working Principle
1. Data contains only input features.
2. Algorithm analyzes similarities and differences.
3. Groups or structures are formed without labels.
Examples of Unsupervised Learning
1. Clustering
• Grouping similar data points.
• Examples:
o Customer segmentation
o Document clustering
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 22
UNIT-I Introduction to Machine Learning
o Image grouping
1. K-Means Clustering
Definition
K-Means is a partition-based clustering algorithm that divides the dataset into K distinct, non-
overlapping clusters, where each data point belongs to the cluster with the nearest mean
(centroid).
Objective Function
K-Means aims to minimize the within-cluster sum of squares (WCSS):
𝑘
𝐽 = ∑ ∑ ∣∣ 𝑥 − 𝜇𝑖 ∣∣2
𝑥∈𝐶𝑖
𝑖=1
Where:
• 𝐶𝑖 = cluster i
• 𝜇𝑖 = centroid of cluster i
Algorithm Steps
1. Choose the number of clusters K.
2. Initialize K centroids randomly.
3. Assign each data point to the nearest centroid.
4. Recalculate centroids.
5. Repeat steps 3 and 4 until convergence.
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 23
UNIT-I Introduction to Machine Learning
Distance Metric
• Euclidean distance (most common)
Advantages
• Simple and fast
• Easy to implement
• Works well for spherical clusters
Disadvantages
• Need to predefine K
• Sensitive to outliers
• Poor performance for non-spherical clusters
Applications
• Customer segmentation
• Image compression
• Market analysis
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 24
UNIT-I Introduction to Machine Learning
2. K-Modes Clustering
Definition
K-Modes is an extension of K-Means designed to work with categorical data instead of
numerical data.
Key Difference from K-Means
Aspect K-Means K-Modes
Data type Numerical Categorical
Centroid Mean Mode
Distance Euclidean Matching dissimilarity
Distance Measure
• Simple matching dissimilarity:
𝑑(𝑥, 𝑦) = number of mismatches
Applications
• Customer profiling using categorical attributes
• Survey data analysis
Advantages
• Handles categorical data
• Efficient
Disadvantages
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 25
UNIT-I Introduction to Machine Learning
• Sensitive to initialization
• Local minima problem
3. K-Prototypes Clustering
Definition
K-Prototypes is a hybrid clustering algorithm designed for mixed data types, i.e., datasets
containing both numerical and categorical features.
Distance Function
Combination of:
• Euclidean distance (numerical)
• Matching dissimilarity (categorical)
𝑑 = 𝑑𝑛𝑢𝑚 + 𝛾𝑑𝑐𝑎𝑡
Where 𝛾is a weighting factor.
Applications
• Real-world datasets with mixed attributes
• Customer databases
Advantages
• Handles mixed data efficiently
Disadvantages
• Parameter tuning required
• Sensitive to scaling
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 26
UNIT-I Introduction to Machine Learning
2. Dimensionality Reduction
Used to reduce number of features while preserving important information.
Dimensionality Reduction Techniques
1. Principal Component Analysis (PCA)
Definition
PCA is a linear dimensionality reduction technique that transforms original correlated features
into uncorrelated principal components.
Objective
Maximize variance while minimizing information loss.
Steps
1. Standardize data
2. Compute covariance matrix
3. Compute eigenvalues and eigenvectors
4. Select top components
5. Transform data
Advantages
• Reduces noise
• Improves performance
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 27
UNIT-I Introduction to Machine Learning
Disadvantages
• Loss of interpretability
• Linear only
2. Linear Discriminant Analysis (LDA)
Definition
LDA is a dimensionality reduction technique that maximizes class separability.
Objective Function
Maximize:
Between-class variance
Within-class variance
Applications
• Face recognition
• Pattern recognition
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 28
UNIT-I Introduction to Machine Learning
1.2.3 Reinforcement Learning
Definition
Reinforcement Learning (RL) is a learning paradigm where an agent interacts with an
environment and learns to maximize rewards through trial and error .
Characteristics of Reinforcement Learning
• Learning is based on experience
• No explicit supervisor or teacher
• Decisions affect future states
• Reward signal guides learning
• Focus is on long-term rewards, not immediate gains
Basic Components of Reinforcement Learning
Reinforcement Learning consists of the following fundamental elements:
3.1 Agent
• The learner or decision-maker
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 29
UNIT-I Introduction to Machine Learning
• Interacts with the environment
• Chooses actions based on policy
Examples:
• Robot
• Game player
• Autonomous vehicle
3.2 Environment
• Everything the agent interacts with
• Responds to agent’s actions
• Provides new state and reward
3.3 State (S)
• Represents the current situation of the agent in the environment
• Can be:
o Fully observable
o Partially observable
3.4 Action (A)
• A decision taken by the agent
• Changes the state of the environment
3.5 Reward (R)
• Numerical feedback received after an action
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 30
UNIT-I Introduction to Machine Learning
• Can be:
o Positive (reward)
o Negative (penalty)
Purpose:
To guide the agent toward desirable behavior.
3.6 Policy (π)
• A strategy that defines what action to take in a given state
𝜋(𝑠) → 𝑎
Goal of reinforcement learning is to learn an optimal policy.
Working Principle of Reinforcement Learning
The reinforcement learning process follows a continuous loop:
1. Agent observes the current state.
2. Agent selects an action based on policy.
3. Action is executed in the environment.
4. Environment returns:
o New state
o Reward
5. Agent updates its policy based on reward.
6. Process repeats until optimal behavior is learned.
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 31
UNIT-I Introduction to Machine Learning
Policy Learning
• Policy defines the mapping:
State → Action
• Goal is to learn optimal policy.
Examples
• Game playing (Chess, Go)
• Robotics
• Self-driving cars
Types of Reinforcement Learning
1. Positive Reinforcement Learning
• Reward is given for correct actions
• Encourages repetition of good behavior
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 32
UNIT-I Introduction to Machine Learning
• Increases performance
Example:
Winning a game gives points.
2. Negative Reinforcement Learning
• Penalty is given for incorrect actions
• Discourages bad behavior
Example:
Robot collides with obstacle → penalty.
Comparison of ML Types with Real-World Applications
Learning Type Data Output Applications
Supervised Labeled Known Spam detection, disease diagnosis
Unsupervised Unlabeled Unknown Customer segmentation, clustering
Reinforcement Reward-based Sequential Robotics, gaming, navigation
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 33
UNIT-I Introduction to Machine Learning
1.3 Applications of Machine Learning
Healthcare
• Disease prediction
• Medical imaging
• Drug discovery
Finance
• Credit scoring
• Fraud detection
• Algorithmic trading
E-Commerce
• Recommendation systems
• Demand forecasting
• Customer behavior analysis
Other Domains
• Speech recognition
• Computer vision
• Smart cities
• Cybersecurity
Challenges in Machine Learning
1. Data quality issues
2. Overfitting and underfitting
3. High computational cost
4. Interpretability of models
5. Ethical and privacy concerns
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 34
UNIT-I Introduction to Machine Learning
1.4 Introduction to Python for Machine Learning
Basics of Python Programming Relevant to ML
• Variables and data types
• Lists, tuples, dictionaries
• Loops and conditional statements
• Functions and modules
Overview of Required Python Libraries
NumPy
• Numerical computing
• Arrays and matrices
• Linear algebra operations
Pandas
• Data manipulation
• DataFrames and Series
• Handling missing data
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 35
UNIT-I Introduction to Machine Learning
Matplotlib
• Data visualization
• Line plots, bar charts, histograms
Scikit-learn
• ML algorithms
• Model training and evaluation
• Preprocessing utilities
Library What it provides Typical use in ML
NumPy Fast n-dimensional arrays, linear-algebra ops Underlying data structures for all algorithms
Pandas Tabular data handling, missing-value imputation, Load CSV/Excel, clean & transform datasets
feature engineering
Matplotlib / Plotting & visualisation Explore data distributions, model diagnostics
Seaborn
Scikit-learn Ready-made implementations of regression, End-to-end pipeline: train_test_split → fit → predict
classification, clustering, model-selection utilities → evaluate
Writing and Executing Simple ML Scripts
Steps:
1. Import libraries
2. Load dataset
3. Preprocess data
4. Train model
5. Test model
6. Evaluate performance
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 36
UNIT-I Introduction to Machine Learning
import pandas as pd
from sklearn.model_selection import train_test_split
from [Link] import RandomForestClassifier
from [Link] import accuracy_score
# Load and prepare data
df = pd.read_csv('[Link]') # assumes a column named
'target'
X = [Link]('target', axis=1)
y = df['target']
# Split into training / test sets
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42)
# Train a model
clf = RandomForestClassifier(n_estimators=200, random_state=42)
[Link](X_train, y_train)
# Predict & evaluate
y_pred = [Link](X_test)
print('Test accuracy:', accuracy_score(y_test, y_pred))
Machine Learning by Prof. Vishal Jadhav Sir’s (VJTech Academy, contact us: +91-7743909870) 37