Module 3
ML Foundation
Why is Machine Learning Important?
Machine Learning is transforming industries by enabling computers to analyze large
amounts of data and make intelligent decisions quickly.
Importance of Machine Learning are:
• Identifies hidden patterns in data.
• Handles large and complex datasets.
• Supports data-driven business decisions.
• Automates decision-making processes.
• Improves prediction accuracy.
• Reduces manual effort and operational costs.
• Enables intelligent automation in various industries.
Key Components of Machine Learning
1. Data in Machine Learning
• Data consists of facts, observations, measurements, or records
collected from various sources.
• Machine learning models analyze this data to identify patterns,
relationships, and trends, which are then used to make predictions or
informed decisions.
• Without sufficient and high-quality data, a machine learning model
cannot learn effectively or produce accurate results.
Types of Data in Machine Learning
1. Structured Data
Structured data is organized in a predefined format, usually in rows and
columns, making it easy to store, search, and analyze.
Some of its Characteristics are:
• Highly organized Sample Structured Data
• Stored in relational databases
• Easy to query using SQL Customer
Age Income Purchased
• Suitable for traditional ML algorithms ID
Examples of structured data 101 28 $40,000 Yes
• Excel spreadsheets 102 35 $55,000 No
• SQL databases
103 42 $68,000 Yes
• Banking transaction records
• Employee databases
• Student records Structured Data are used in the following Applications:
•Sales prediction
•Customer churn prediction
•Credit scoring
•House price prediction
Types of Data in Machine Learning
2. Unstructured Data
• Unstructured data does not have a predefined format or structure. It is more
complex and requires advanced techniques such as Deep Learning to analyze.
Some of its Characteristics are:
Characteristics
• Large in volume Unstructured Data are used in the following
• Difficult to organize Applications:
• Rich in information
• Face recognition
• Requires preprocessing before analysis
• Speech recognition
Examples of unstructured data: • Image classification
• Images • Sentiment analysis
• Videos • Medical diagnosis
• Audio recordings • Chatbots
• Emails
• Social media posts
• Medical images (MRI, CT scans)
• Documents and PDFs
Types of Data in Machine Learning
3. Semi-Structured Data
Semi-structured data contains some organizational properties, such as
tags or keys, but does not follow a rigid table format like structured data.
Some of its Characteristics are:
• Flexible structure
Semi-structured Data are used in the following
• Easier to process than unstructured data
Applications:
• Commonly used in web and cloud applications
Examples of semi-structured data • Web applications
• XML files • IoT devices
• APIs
• HTML documents
• Cloud computing
• Log files • Data exchange between systems
• Email messages
Key Components of Machine Learning
2. Features (Input Variables)
Features are the input variables or attributes that describe the
characteristics of an object, event, or observation. They provide the
information that a machine learning model uses to learn patterns and
make predictions.
Types of Features:
1. Numerical Features
2. Categorical Features
3. Ordinal Features
4. Binary Feature
Types of Features
1. Numerical Features 3. Ordinal Features
Contain numerical values. Categorical values with a meaningful order.
Examples Examples
•Age •Education Level (High School < Bachelor's < Master's < PhD)
•Salary •Customer Satisfaction (Poor < Fair < Good < Excellent)
•Temperature
•Height
•Annual income
2. Categorical Features 4. Binary Features
Represent categories or labels. Have only two possible values.
Examples Examples
•Gender (Male/Female) •Yes / No
•Blood Group (A, B, AB, O) •True / False
•City •Purchased / Not Purchased
•Occupation
Example of features
Example 1: House Price Prediction (Suppose we want to predict the price of a house)
Features are: Age, Bedrooms, Bathrooms, parking, location, age of the house
Example 2: Disease Prediction
Features are: Age, gender, Blood Pressure, Blood Sugar, Cholesterol Level, BMI, Heart Rate
Example 3: Plant Disease Detection
Features are: Leaf Color, Leaf Texture, Temperature, Humidity, Soil, Moisture, Soil pH
3. Labels (Target Variables)
• A label (also called the target variable, output variable, or dependent variable) is the
correct answer or outcome that the machine learning model is trained to predict.
• In supervised learning, every training example has both features and a corresponding
label.
• We can consider label of a dataset as predicted output.
Example 2: Email Spam Detection
Email Content (Feature) Label
"Congratulations! You won a prize." Spam
"Meeting scheduled for tomorrow." Not Spam
The model learns to classify emails based on the provided labels.
Classification and Regression
• Classification and Regression are the two primary
types of supervised learning in Machine Learning. In
supervised learning, the model learns from labeled
data, where each input (features) has a
corresponding correct output (label).
• The main difference is the type of output they
predict:
• Classification predicts categorical (discrete)
values.
• Regression predicts continuous (numerical)
values.
1. Classification
• Classification is the process of predicting the category or class to
which a new data point belongs.
• Classification follows the below steps:
• Collect labeled data.
• Extract relevant features.
• Train a classification algorithm.
• Learn the relationship between features and class labels.
• Predict the class of new, unseen data.
Classification Workflow
Input Dataset(D) Image
Data Preprocessing
Apply Train data to train Dataset Splitting Train/Validation/Test
each model
Classification Algorithms
Validation Phase: compute Accuracy/ Loss
Testing Phase: evaluate on test
data
Output
Data Preprocessing
1. Data Cleaning: Data cleaning removes errors, duplicate records, inconsistent data, and irrelevant
information from the dataset.
2. Handling Missing Values: Missing values occur when some observations do not contain complete
information. These values must be handled before training the model. (by using mean/median
values)
3. Handling Outliers: Outliers are extreme values that differ significantly from the rest of the data.
(Use Interquartile Range (IQR) or Z-score method)
4. Encoding Categorical Data: Machine learning algorithms work with numerical data, so categorical
values must be converted into numbers. (Label Encoding, one hot encoding)
5. Feature Scaling : Feature scaling ensures that numerical features have similar ranges so that no
feature dominates others. (Min-Max Normalization, Standardization (Z-score))
6. Feature Selection: Selecting the most relevant features from the dataset while removing
unnecessary ones.
7. Feature Extraction: Creating new features from existing data.
Common Classification Algorithms
Decision Tree
Random Forest
Support Vector Machine (SVM)
K-Nearest Neighbors (KNN)
Naïve Bayes
Neural Networks
Types of Classification
1. Binary Classification
Predicts one of two possible classes.
Examples
• Spam or Not Spam
• Pass or Fail
• Disease or No Disease
• Fraud or Genuine
• Yes or No
Types of Classification
2. Multi-Class Classification
Predicts one class from more than two categories.
Examples: Handwritten digit recognition (0–9), Plant disease
classification, Animal recognition, Weather prediction (Sunny,
Rainy, Cloudy)
One vs. one multi-class classification One vs. rest multi-class classification
Types of Classification
3. Multi-Label Classification
Each sample can belong to multiple classes simultaneously.
Examples
• Movie genres (Action, Comedy, Drama)
• Medical diagnosis (Diabetes + Hypertension)
Evaluation Metrics for Classification
Accuracy: Percentage of correct predictions
Precision: Correct positive predictions among predicted positives
Recall: Correct positive predictions among actual positives
F1-Score: Harmonic mean of Precision and Recall
ROC-AUC: Measures classifier performance across thresholds
Confusion Matrix: Shows TP, TN, FP, and FN
Accuracy {TP+TN}/{TP+TN+FP+FN}
Precision {TP}/{TP+FP}
Recall (Sensitivity) {TP}/{TP+FN}
Specificity {TN}/{TN+FP}
F1-Score
[Link]
• Regression is a supervised machine learning technique used to
predict continuous numerical values based on input features.
Training Data (Features + Numeric Target)
Regression Algorithm
Trained Regression Model
Predict Continuous Value
Types of Regression
1. Linear Regression
When we can model the relationship between variables using a straight line.
Y=mX+c
where, Y = Predicted value, X = Input feature, m = Slope, c = Intercept
Examples
• House price prediction
• Salary prediction
• Sales forecasting
Types of Regression
2. Multiple Linear Regression
We use multiple input features to predict one numerical output.
Example
Predicting house price using:
• Area
• Bedrooms
• Location
• Parking
• Age of House
Types of Regression
3. Polynomial Regression
When we model non-linear relationships using polynomial equations, then
polynomial regression is used.
Example
Predicting crop yield based on fertilizer amount where the relationship is
curved rather than linear.
4. Ridge and Lasso Regression
It is used to reduce overfitting by adding regularization.
• Ridge Regression (L2): Shrinks coefficients but keeps all features. It is used for
financial forecasting.
• Lasso Regression (L1): Shrinks coefficients and can remove less important
features. It is used for feature selection.
Evaluation Metrics for Regression
Mean Absolute Error (MAE) Average absolute prediction error
Mean Squared Error (MSE) Average squared prediction error
Root Mean Squared Error (RMSE) Square root of MSE
Measures how well the model explains
R² Score
data variability
References
• [1] C. M. Bishop, Pattern Recognition and Machine Learning. New York, NY,
USA: Springer, 2006.
• [2] T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical
Learning: Data Mining, Inference, and Prediction, 2nd ed. New York, NY,
USA: Springer, 2009.
• [3] K. P. Murphy, Machine Learning: A Probabilistic Perspective. Cambridge,
MA, USA: MIT Press, 2012.
• [4] A. Géron, Hands-On Machine Learning with Scikit-Learn, Keras &
TensorFlow, 3rd ed. Sebastopol, CA, USA: O'Reilly Media, 2022.
• [5] E. Alpaydin, Introduction to Machine Learning, 4th ed. Cambridge, MA,
USA: MIT Press, 2020.