Machine Learning

Introduction
Module I
 Module 1 - Syllabus
MODULE-1 Introduction to Machine Learning:
Understanding Machine Learning: Definition and Types of Machine Learning-
Application of Machine Learning- Machine Learning Algorithms: Supervised,
Unsupervised, and Semi-Supervised Learning Algorithms. Machine Learning Models
Model Evaluation Metrics: Confusion Matrix, Precision, Recall, F1 Score -ROC
Curve and AUC-ROC.
Advanced Techniques: Feature Scaling and Normalization –Encoding Categorical
Variables-Train-test Split and Cross-validation
Introduction to Machine Learning
• In algorithm based solution data
and program are fed into the
computer, program gets
executed and the desired output
is produced.
• In ML based solution data and
target labels(output) is fed into
the system, system learns from
this data and build the
model(program).
Figure 1: Diagram representing difference between
Algorithmic and ML based solution
Definition of Machine Learning
• Machine Learning explores algorithms that can
• learn from data / build a model from data
• use the model for prediction, decision making or solving some tasks
• Machine learning (ML) is defined as a discipline of artificial
intelligence (AI) that provides machines the ability to automatically
learn from data and past experiences to identify patterns and make
predictions with minimal human intervention.
• A weather prediction model trained on historical weather data can
predict future weather based on current conditions.
• Clustering customers based on purchasing behavior to identify market
segments.
Terminologies in Machine Learning
• Data set : Dataset is a collection of various types of data stored in a digital
format.
• Features: A feature is one column of the data in your input set. feature is
input;
• The key identified properties based on which we would like to predict the result of
our problem statement.
• Labels are also known as tags, which are used to give an identification to a
piece of data and tell some information about that element.
-A label is the thing we're predicting
-Eg: Price of stock
• Labels are also referred to as the final output for a prediction. For example,
as in the below image, we have labels such as a cat and dog, etc.
• label is output. This applies to both classification and regression problems.
Terminologies in Machine Learning
• Training data is the initial dataset you use to teach a machine learning
application to recognize patterns or perform to your criteria.
• Testing or validation data is used to evaluate your model's accuracy.
• An Algorithm is a set of rules that a machine follows to achieve a
particular goal.
• An algorithm can be considered as a recipe that defines the inputs, the
output and all the steps needed to get from the inputs to the output.
• Model: System that has been trained to recognize certain type of
patterns.
• A model defines the relationship between features and label.
• Train the model over set of data by providing algorithm to learn from the data.
• Regression vs. classification
-A regression model predicts continuous values.
• -Eg:What is the value of a house in Bangalore?(price of a
house)
-A classification model predicts discrete values.
Eg: Is a given email message spam or not spam?
Applications of Machine learning
1. Image Recognition:
• It is used to identify objects, persons, places, digital images, etc. The
popular use case of image recognition and face detection is, Automatic
friend tagging suggestion:
• Facebook provides us a feature of auto friend tagging suggestion.
Whenever we upload a photo with our Facebook friends, then we
automatically get a tagging suggestion with name, and the technology
behind this is machine learning's face detection and recognition
algorithm.
• It is based on the Facebook project named "Deep Face," which is
responsible for face recognition and person identification in the picture.
2. Speech Recognition
• While using Google, we get an option of "Search by voice," it comes
under speech recognition, and it's a popular application of machine
learning.
• Speech recognition is a process of converting voice instructions into
text, and it is also known as "Speech to text", or "Computer speech
recognition."
• At present, machine learning algorithms are widely used by various
applications of speech recognition. Google assistant, Siri, Cortana,
and Alexa are using speech recognition technology to follow the voice
instructions.
3. Traffic prediction:
• If we want to visit a new place, we take help of Google Maps, which
shows us the correct path with the shortest route and predicts the traffic
conditions.
• It predicts the traffic conditions such as whether traffic is cleared, slow-
moving, or heavily congested with the help of two ways:
• Real Time location of the vehicle form Google Map app and sensors
• Average time has taken on past days at the same time.
• Everyone who is using Google Map is helping this app to make it
better. It takes information from the user and sends back to its database
to improve the performance.
4. Product recommendations:
• Machine learning is widely used by various e-commerce and
entertainment companies such as Amazon, Netflix, etc., for product
recommendation to the user.
• Whenever we search for some product on Amazon, then we started
getting an advertisement for the same product while internet surfing on
the same browser and this is because of machine learning.
• Once you’ve shown interest in a product, Amazon and other
companies leverage machine learning algorithms to retarget you
with ads related to the same product or similar items.
• As similar, when we use Netflix, we find some recommendations for
entertainment series, movies, etc., and this is also done with the help of
machine learning
[Link]-driving cars:
• Machine learning plays a significant role in self-driving cars. Tesla,
the most popular car manufacturing company is working on self-
driving car.
• It is using unsupervised learning method to train the car models to
detect people and objects while driving.
• 6. Email Spam and Malware Filtering:
• Whenever we receive a new email, it is filtered automatically as important, normal,
and spam. We always receive an important mail in our inbox with the important
symbol and spam emails in our spam box, and the technology behind this is Machine
learning.
• Below are some spam filters used by Gmail:
• Content Filter: This filter scans the content of the email, including the body and attachments, for
keywords, phrases, or patterns commonly associated with spam
• Header filter: This filter examines the metadata in the email headers, such as the sender's
address, subject line, and other routing information.
• General blacklists filter: Gmail maintains and utilizes blacklists of known spam sources, including
IP addresses, domains, and email addresses.
• Rules-based filters: A user might create a rule to automatically move all emails from a particular
sender or containing certain keywords to the spam folder.
• Permission filters: If a user has previously marked emails from a particular sender as spam,
Gmail will likely continue to filter similar emails from that sender into the spam folder.
• Some machine learning algorithms such as Multi-Layer Perceptron, Decision tree,
and Naïve Bayes classifier are used for email spam filtering and malware detection.
7. Virtual Personal Assistant:
• We have various virtual personal assistants such as Google
assistant, Alexa, Cortana, Siri. As the name suggests, they help us in
finding the information using our voice instruction. These assistants
can help us in various ways just by our voice instructions such as Play
music, call someone, Open an email, Scheduling an appointment,
etc.
• These virtual assistants use machine learning algorithms as an
important part.
• These assistant record our voice instructions, send it over the server
on a cloud, and decode it using ML algorithms(NLP, Speech
recognition, Connect to 3rd party Services) and act accordingly.
8. Online Fraud Detection:
• Machine learning is making our online transaction safe and secure by
detecting fraud transaction.
• It involves using machine learning algorithms to identify and prevent
fraudulent activities in real-time within online systems, such as
banking, e-commerce, and payment processing platforms.
• Machine learning models are trained to recognize normal transaction
patterns and detect anomalies or deviations that could indicate
fraudulent behavior.
• Predictive models use historical data to forecast the likelihood of
fraud based on various features of transactions, such as transaction
amount, frequency, and user behavior.
9. Stock Market trading:
• Machine learning is widely used in stock market trading. In the stock
market, there is always a risk of up and downs in shares, so for this
machine learning's long short term memory neural
network (LSTM)is used for the prediction of stock market trends.
• High-frequency trading firms use machine learning models to place
buy and sell orders automatically based on real-time data analysis.
• Predicting the future price of a stock by analyzing historical price data,
trading volumes, and other financial indicators.
10. Medical Diagnosis:
• In medical science, machine learning is used for diseases diagnoses. With
this, medical technology is growing very fast and able to build 3D models
that can predict the exact position of lesions in the brain.
• Algorithms can be trained to classify medical images, lab results, or
patient data into different categories, such as identifying whether a patient
has a particular disease.
• ML models can analyze mammograms to detect signs of breast cancer or
assess CT scans to identify tumors.
• Predicting when medical equipment is likely to fail or require maintenance
based on usage patterns and historical data.
11. Automatic Language Translation:
• NMT is a type of machine learning model that uses neural networks
to translate text. Unlike earlier models that translated text word-by-
word, NMT systems use deep learning to understand and generate
entire sentences in context.
• Google's GNMT (Google Neural Machine Translation) provide this
feature, which is a Neural Machine Learning that translates the text
into our familiar language, and it called as automatic translation.
• Technology used : Sequence-to-Sequence Models:
• These models encode the input sentence into a fixed-length vector
and then decode this vector into a sentence in the target language.
They use techniques like Long Short-Term Memory (LSTM) or
Transformer networks.
Types of Machine Learning
 Supervised Machine Learning
• Supervised learning is a type of machine learning where an algorithm
is trained on a labeled dataset.
• In supervised learning, the model learns from input-output pairs,
meaning it is provided with both the input data (features) and the
corresponding correct output (labels).
• Supervised learning algorithms are used when the output is classified
or labeled.
• These algorithms learn from the past data that is inputted, called
training data, runs its analysis and uses this analysis to predict future
events of any new data within the known classifications.
• The accurate prediction of test data requires large data to have a
sufficient understanding of the patterns.
• The algorithm can be trained further by comparing the training
outputs to actual ones and using the errors for modification of the
algorithms.
Advantages of Supervised Learning
• Since supervised learning works with the labeled dataset, we can have an
exact idea about the classification of objects and other variables that we feed
in as input.
• These algorithms are helpful in predicting the output on the basis of prior
experience and trained data.
Disadvantages of Supervised Learning
• These algorithms are not able to solve complex tasks due to a lack of data.
• It may predict the wrong output if the test data is different from the training
data or the training data has some noise.
• It requires lots of computational time to train the algorithm and a huge load
of trained data.
Algorithms Used:

• Linear Regression (for regression tasks)

• Logistic Regression (for classification tasks)
• Decision Trees
• Random Forest
• Support Vector Machines (SVM)
• Examples:
• Spam detection (classifying emails as spam or not)
• Image classification (identifying objects in images)
• Predicting stock prices (regression tasks)
• Sentiment analysis (categorizing text as positive, neutral, or negative)
Unsupervised Machine Learning

• In unsupervised machine learning, we train the machine using the

unlabeled or untrained dataset, and the machine predicts the output
without any supervision of such data.
• Unsupervised learning is a type of machine learning where the
algorithm is given data without explicit labels or output variables.
• The goal is for the model to find hidden patterns, structures, or
relationships within the data on its own.
• Tasks:
• Clustering: Grouping similar data points together.
• Dimensionality Reduction: Simplifying the data by reducing the number of
variables while preserving important information.
• Anomaly Detection: Identifying unusual or rare data points.
Advantages of Unsupervised Learning
• We can use these algorithms for complicated tasks as compared to the
supervised ones because these algorithms work on unlabeled datasets and
do not require large datasets.
• Unsupervised algorithms are preferable for various tasks as getting the
unlabeled dataset is easier as compared to the labeled dataset for the
training of these algorithms.
Disadvantages of Unsupervised Learning
• An unsupervised algorithm may result in less accurate outputs as the dataset
is not labeled, and we have not trained the algorithms with the exact output
beforehand.
• Working with Unsupervised learning is more difficult as compared to other
types as it works with the unlabelled dataset that does not map with the
output precisely.
• The goal of unsupervised learning is to find the underlying structure
of dataset, group that data according to similarities, and represent
that dataset in a compressed format.
Reinforcement Learning

• Reinforcement learning operates on a feedback-based process, in

which an Artificial Intelligence agent automatically explores its
surroundings by hit and trial methods.
• It takes action, learns from experiences, and improves its
performance. The algorithm rewards such agents for each good
action and punishes it for each bad action.
• Hence, the reinforcement learning agent aims to maximize the
rewards and minimize the punishments.
• In reinforcement learning, the algorithm does not require any labeled
data like supervised learning, and agents solely learn from their
experiences.
Advantages of Reinforcement Learning
• This type of learning assists us in solving complex real-world problems
which are difficult to be solved by general techniques that we use
conventionally.
• The learning model of Reinforcement Learning is similar to the
learning process of human beings; therefore, we can look for the
most accurate results.
• This type of learning helps us in achieving long-term results.
Disadvantages of Reinforcement Learning
• We generally do not prefer these algorithms for simple problems.
• These algorithms require huge data and high computational powers.
• Too much reinforcement learning can lead to an overload of states
which can weaken the results defying the purpose of deploying them.
Semi-Supervised Learning

• Semi-Supervised learning is a type of Machine Learning algorithm that

represents the intermediate ground between Supervised and Unsupervised
learning algorithms.
• It uses the combination of labeled and unlabeled datasets during the
training period.
• The basic disadvantage of supervised learning is that it requires hand-
labeling by ML specialists or data scientists, and it also requires a high cost
to process.
• Unsupervised learning also has a limited spectrum for its applications.
• To overcome these drawbacks of supervised learning and unsupervised
learning algorithms, the concept of Semi-supervised learning is
introduced.
Semi-Supervised Learning
• In semi-supervised learning algorithm, training data is a combination
of both labeled and unlabeled data.
• However, labeled data exists with a very small amount while it
consists of a huge amount of unlabeled data.
• Initially, similar data is clustered along with an unsupervised learning
algorithm, and further, it helps to label the unlabeled data into
labeled data.
Advantages of semi-supervised learning
• It is quite simple and easy to understand the algorithm and does not encounter
anomalies.
• This is highly efficient in predicting the output on the basis of input data.
• It overcomes the drawbacks of Supervised and Unsupervised Learning
algorithms.

Disadvantages of semi-supervised learning

• Iteration results may not be stable and outputs may vary significantly.
• We cannot apply these algorithms to network-level data due to its complexities.
• The accuracy rate for this type of Learning is low.
Criteria Supervised ML Unsupervised ML Semi-supervised ML Reinforcement ML

Learns from a small amount

Trained using unlabelled data of labelled data combined Works on interacting with
Definition Learns by using labelled data
without any guidance with a large amount of the environment
unlabelled data

Mostly unlabelled data + few No predefined data (learns

Type of data Labelled data Unlabelled data
labelled samples from feedback)

Classification, regression,
Type of problems Regression and classification Association and clustering clustering (when labels are Exploitation or exploration
scarce)

Limited supervision (partial No supervision (feedback-

Supervision Extra supervision No supervision
labels) based)

Label Propagation, Semi-

Linear Regression, Logistic
Algorithms K-Means, C-Means, Apriori supervised SVM, Graph- Q-Learning, SARSA
Regression, SVM, KNN etc.
based SSL, Self-training

Improve accuracy when

Aim Calculate outcomes Discover underlying patterns Learn a series of actions
labels are limited

Risk evaluation, Forecast Recommendation systems, Text classification, Medical Self-driving cars, Gaming,
Application
sales Anomaly detection diagnosis, Fraud detection Healthcare
Machine Learning Models and Performance
Measures
Machine Learning models are programs that has been trained to find
patterns within new data and make predictions.
These models are represented as a mathematical function that takes
requests in the form of input data, makes predictions on input data,
and then provides an output in response.
First, these models are trained over a set of data, and then they are
provided an algorithm to reason over data, extract the pattern from
feed data and learn from those data.
Once these models get trained, they can be used to predict the unseen
dataset.
Confusion Matrix

• Confusion Matrix is a performance measurement for the machine learning

classification problems where the output can be two or more classes. It is a table
with combinations of predicted and actual values.
• A confusion matrix is defined as the table that is often used to describe the
performance of a classification model on a set of the test data for which the true
values are known.
• Let’s take an example of a patient who has gone to a doctor with certain
symptoms. Since it’s the season of Covid, let’s assume that he went with fever,
cough, throat ache, and cold. These are symptoms that can occur during any
seasonal changes too. Hence, it is tricky for the doctor to do the right diagnosis.

39
• True Positive (TP):
• Let’s say the patient was actually suffering from Covid and on doing the required
assessment, the doctor classified him as a Covid patient. This is called TP or True Positive.

• This is because the case is positive in real and at the same time the case was classified
correctly. Now, the patient can be given appropriate treatment which means, the decision
made by the doctor will have a positive effect on the patient and society.

• False Positive (FP):

• Let’s say the patient was not suffering from Covid and he was only showing symptoms of
seasonal flu but the doctor diagnosed him with Covid. This is called FP or False Positive.

• This is because the case was actually negative but was falsely classified as positive. Now,
the patient will end up getting admitted to the hospital or home and will be given treatment
for Covid.
• This is an unnecessary inconvenience for him and others as he will get unwanted
treatment and quarantine. This is called Type I Error.
40
• True Negative (TN):
• Let’s say the patient was not suffering from Covid and the doctor also gave him a
clean chit. This is called TN or True Negative. This is because the case was actually
negative and was also classified as negative which is the right thing to do. Now
the patient will get treatment for his actual illness instead of taking Covid treatment.
• False Negative (FN):
• Let’s say the patient was suffering from Covid and the doctor did not diagnose
him with Covid. This is called FN or False Negative as the case was actually
positive but was falsely classified as negative.
• Now the patient will not get the right treatment and also he will spread the disease
to others. This is a highly dangerous situation in this example. This is also
called Type II Error.

41
42
• Precision:
• Precision = TP / (TP + FP)

• Out of all that positive predicted, how many are actually truly positive.

• The precision value lies between 0 and 1.

• Recall or Sensitivity:
• Recall = TP/ (TP + FN)

• Out of all total positive cases, what percentage is predicted as positive.

• F1-Score:
• F1 score = 2* (Precision * Recall) / (Precision + Recall)

• F1- score is a weighted average of precision and recall.

• It is the harmonic mean of precision and recall. It takes both false positive and false negatives into account. Therefore, it performs well on
an imbalanced dataset.

43
AUC - ROC curve
• The ROC(Receiver Operating Characteristic) curve: It is a graphical
representation of the performance of a binary classifier at different
classification thresholds.
• To compute it, you must measure the area under the ROC curve, which
shows the classifier's performance at varying decision thresholds.
• The ROC curve shows the performance of a binary classifier with different decision
thresholds. It plots the True Positive rate (TPR) against the False Positive rate (FPR).
• The ROC AUC score is the area under the ROC curve. It sums up how well a model
can produce relative scores to discriminate between positive or negative instances
across all classification thresholds.
• The ROC AUC score ranges from 0 to 1, where 0.5 indicates random guessing, and 1
indicates perfect performance.

44
• ROC AUC score shows how well the classifier distinguishes positive and negative classes. It can take
values from 0 to 1.

• A higher ROC AUC indicates better performance. A perfect model would have an AUC of 1, while a
random model would have an AUC of 0.5.

45
• True vs. False Positive rates:
• The ROC curve plots the True Positive rate
(TPR) against the False Positive rate (FPR)
at various classification thresholds. You
can derive TPR and FPR from a confusion
matrix.
• TPR (True Positive rate, also known as
recall) shows the share of detected true
positives out of all objects of the Positive
class.
• For example, the share of emails correctly
labeled as spam out of all spam emails in
the dataset.
• FPR (False Positive rate) shows the share
of objects falsely assigned a positive class
out of all objects of the negative class.
• For example, the proportion of legitimate
emails falsely labeled as spam

46
• When you set the threshold higher, you make the model "more conservative." It
assigns the True label when it is "more confident." But as a consequence, you typically
lower recall: you detect fewer examples of the target class overall.

• When you set the threshold lower, you make the model "less strict." It assigns the True
label more often, even when "less confident."
• Consequently, you increase recall: you will detect more examples of the target class.
• However, this may also lead to lower precision, as the model may make more False
Positive predictions.
• Higher threshold → fewer positives predicted → fewer mistakes (false positives) but more
misses (false negatives).
• Lower threshold → more positives predicted → fewer misses but more false alarms.

47
• In the example above, the recall (TPR) decreases as we set the
different decision higher:
• - 0.5 threshold: 800/(800+100)=0.89
• - 0.8 threshold: 600/(600+300)=0.67
• - 0.95 threshold: 200/(200+700)=0.22
• The FPR also goes down:
• - 0.5 threshold: 500/(500+8600)=0.06
• - 0.8 threshold: 100/(100+9000)=0.01
• - 0.95 threshold: 10/(10+9090)=0.001

48
• In the perfect scenario, we measure the square area: ROC AUC is 1. In
the random scenario, it is precisely half: ROC AUC is 0.5.
• The ROC AUC score can range from 0 to 1. A score of 0.5 indicates
random guessing, and a score of 1 indicates perfect performance.
• A score slightly above 0.5 shows that a model has at least "some"
(albeit small) predictive power. This is generally inadequate for any
real applications.

49
• When AUC is approximately 0, the model is actually
reciprocating the classes. It means the model is predicting a
negative class as a positive class and vice versa.

50
Quick analogy with a spam filter:
● Accuracy: How often emails are classified correctly (spam or not).
● Precision: Of the emails marked as spam, how many are truly spam?
● Recall: Of all real spam emails, how many did the filter catch?
● F1: A balance if you want both precision and recall to be good.
● ROC & AUC: A way to see how well the filter can separate spam vs. not
spam when you adjust how strict it is.
What is a feature?
• Generally, all machine learning algorithms take input data to generate the
output.
• The input data remains in a tabular form consisting of rows (instances or
observations) and columns (variable or attributes), and these attributes are
often known as features.
• The input variables that we give to our machine learning models are called
features. Each column in our dataset constitutes a feature.
• Features are nothing but the independent variables in machine learning
models. What is required to be learned in any specific machine learning
problem is a set of these features (independent variables), coefficients of
these features, and parameters for coming up with appropriate functions
or models (also termed hyperparameters).
Example
• A model for predicting the risk of cardiac disease may have features
such as the following:
• Age
• Gender
• Weight
• Whether the person smokes
• Whether the person is suffering from diabetic disease, etc.
What is Feature Engineering?
• Feature engineering is the pre-processing step of machine learning,
which extracts features from raw data.
• Feature engineering refers to a process of selecting and transforming
variables when creating a predictive model using machine learning.
• It helps to represent an underlying problem to predictive models in a better
way, which as a result, improve the accuracy of the model for unseen data.
• The predictive model contains predictor variables and an outcome variable,
and while the feature engineering process selects the most useful predictor
variables for the model.
• Eg: size of the house, number of bedrooms, and location, are the predictor
variables. These are believed to determine the value of the property or
house. Actual prices is the outcome variable.
1. Feature Creation:
• Feature creation is finding the most useful variables to be used in a predictive
model.
• The process is subjective, and it requires human creativity and intervention.
• The new features are created by mixing existing features using addition,
subtraction, and ration, and these new features have great flexibility.
• Types of Feature Creation:
1. Domain-Specific: Creating new features based on domain knowledge, such
as creating features based on business rules or industry standards.
2. Data-Driven: Creating new features by observing patterns in the data, such
as calculating aggregations or creating interaction features.
3. Synthetic: Generating new features by combining existing features or
synthesizing new data points.
2. Transformations:
• The transformation step of feature engineering involves adjusting the predictor variable to improve
the accuracy and performance of the model.
• For example, it ensures that the model is flexible to take input of the variety of data; it ensures that
all the variables are on the same scale, making the model easier to understand.
• It improves the model's accuracy and ensures that all the features are within the acceptable range to
avoid any computational error.
• Types of Feature Transformation:
1. Normalization: Rescaling the features to have a similar range, such as between 0 and 1, to
prevent some features from dominating others.
2. Scaling: Rescaling the features to have a similar scale, such as having a standard deviation of 1,
to make sure the model considers all features equally.
3. Encoding: Transforming categorical features into a numerical representation. Examples are
one-hot encoding and label encoding.
4. Transformation: Transforming the features using mathematical operations to change the
distribution or scale of the features. Examples are logarithmic, square root, and reciprocal
transformations.
3. Feature Extraction:
• Feature extraction is an automated feature engineering process that generates new
variables by extracting them from the raw data.
• The main aim of this step is to reduce the volume of data so that it can be easily used and
managed for data modelling.
• Feature extraction methods include cluster analysis, text analytics, edge detection
algorithms, and principal components analysis (PCA).
• Types of Feature Extraction:
1. Dimensionality Reduction: Reducing the number of features by transforming the data
into a lower-dimensional space while retaining important information. Examples are PCA
and t-SNE.
2. Feature Combination: Combining two or more existing features to create a new one. For
example, the interaction between two features.
3. Feature Aggregation: Aggregating features to create a new one. For example, calculating
the mean, sum, or count of a set of features.
4. Feature Transformation: Transforming existing features into a new representation. For
example, log transformation of a feature with a skewed distribution.
4. Feature Selection:
• While developing the machine learning model, only a few variables in the dataset are
useful for building the model, and the rest features are either redundant or irrelevant.
• If we input the dataset with all these redundant and irrelevant features, it may negatively
impact and reduce the overall performance and accuracy of the model.
• Hence it is very important to identify and select the most appropriate features from the
data and remove the irrelevant or less important features, which is done with the help of
feature selection in machine learning.
• Types of Feature Selection:
1. Filter Method: Based on the statistical measure of the relationship between the
feature and the target variable. Features with a high correlation are selected.
2. Wrapper Method: In wrapper methods, we try to use a subset of features and train a
model using them. Based on the inferences that we draw from the previous model, we
decide to add or remove features from your subset.
3. Embedded Method: Embedded methods combine the qualities’ of filter and wrapper
methods. It’s implemented by algorithms that have their own built-in feature selection
methods.
• 5. Feature Scaling

• Feature Scaling is the process of transforming the features so that they

have a similar scale. This is important in machine learning because the
scale of the features can affect the performance of the model.
• Types of Feature Scaling:
1. Min-Max Scaling: Rescaling the features to a specific range, such as
between 0 and 1, by subtracting the minimum value and dividing by the
range.
2. Standard Scaling: Rescaling the features to have a mean of 0 and a
standard deviation of 1 by subtracting the mean and dividing by the
standard deviation.
3. Robust Scaling: Rescaling the features to be robust to outliers by dividing
them by the interquartile range.
Feature Scaling and Normalization
• Feature scaling transforms the features to be on a similar scale without distorting differences in
the ranges of values.
• Min-Max Scaling: Rescales the feature to a specific range (usually 0 to 1).

• For example, for a dataset, the min and max observable values as 30 and -10. We can then
normalize any value, like 18.8, as follows:
• y = (x – min) / (max – min)
• y = (18.8 – (-10)) / (30 – (-10))
• y = 28.8 / 40
• y = 0.72
Standardization (Z-score Normalization)
• Rescaling the features to have a mean of 0 and a standard deviation
of 1 by subtracting the mean and dividing by the standard deviation.

• Where the mean is calculated as:

• mean = sum(x) / count(x)
• And the standard_deviation is calculated as:
• standard_deviation = sqrt( sum( (x – mean)^2 ) / count(x))
Example
• We can estimate a mean of 10.0 and a standard deviation of about
5.0. Using these values, we can standardize the first value of 20.7 as
follows:
• XI = (X – mean) / standard_deviation
• XI = (20.7 – 10) / 5
• XI = (10.7) / 5
• XI = 2.14
Min-Max Scaling-Example
• A value is normalized as follows:
• y = (x – min) / (max – min)
• Where the minimum and maximum values pertain to the value x being
normalized.
• For example, for a dataset, the min and max observable values as 30 and -
10. We can then normalize any value, like 18.8, as follows:
• y = (x – min) / (max – min)
• y = (18.8 – (-10)) / (30 – (-10))
• y = 28.8 / 40
• y = 0.72
Robust Scaler
• The Robust Scaler is a feature scaling technique in machine learning
that transforms features by removing the median and scaling them
according to the interquartile range (IQR).
• This makes it more robust to outliers compared to standard scaling
methods like Min-Max or StandardScaler, which can be heavily
influenced by extreme values.
• Centers the data by subtracting the median instead of the mean.
• Example:
• Suppose you have a feature with the following values: [10, 20, 30, 40,
1000].
• Here, the value 1000 is an outlier.
• Median of this feature: 30
• IQR:𝑄3=40 (75th percentile)
• 𝑄1=20 (25th percentile)
• So, IQR=40−20=20
• Using Robust Scaler:

• This transformation is less affected by the outlier (1000) compared to Min-

Max or Standard Scaler, as it uses the median and IQR.
Encoding Categorical Variables
• Encoding categorical variables is a crucial preprocessing step in machine learning when
working with datasets that contain non-numeric (categorical) features.
• Many machine learning algorithms require numerical input, so categorical variables
must be converted into a suitable format.
• 1. Label Encoding:
• Assigns a unique integer to each category.
• This method is simple but introduces an implicit ordinal relationship between categories,
which might not always be desired.
• Example:For the categorical feature "Color" with values ["Red", "Blue", "Green"],
• Label Encoding might assign:
• Red → 0
• Blue → 1
• Green → 2
• Best for ordinal variables (i.e., categories with a natural ordering, like ["Low", "Medium",
"High"]).
One-Hot Encoding
• Converts each category into a new binary
column. Each category gets its own column, and
a row will have 1 for the present category and 0
for others.
• This method avoids introducing ordinal
relationships.
• Example:For "Color" with values ["Red", "Blue",
"Green"],
• One-Hot Encoding creates new columns:
• Red → [1, 0, 0]
• Blue → [0, 1, 0]
• Green → [0, 0, 1]
• Suitable for nominal (non-ordinal) categorical
variables, especially in models that are sensitive
to scale (e.g., linear models).
.Binary Encoding

• Encodes categories as binary numbers and then splits them into columns.
• It is more memory-efficient than One-Hot Encoding when dealing with high
cardinality (many categories).
• Step 1: Label Encoding :
• First, assign a unique integer to each category:
• Red → 1
• Blue → 2
• Green → 3
• Yellow → 4
• Purple → 5
• Step 2: Convert to Binary
• Now, convert each integer into binary:
• Red (1) → 001
• Blue (2) → 010
• Green (3) → 011
• Yellow (4) → 100
• Purple (5) → 101
• Step 3: Split into Columns

• Now, instead of 5 columns (as you would get with One-Hot Encoding),
you only need 3 columns to represent 5 categories.
Cross Validation

Cross-validation is a technique for evaluating a machine learning model and

testing its performance.

 CV is commonly used in applied ML tasks.

 It helps to compare and select an appropriate model for the specific
predictive modeling problem.
 CV is easy to understand, and easy to implement.
 All of this makes cross-validation a powerful tool for selecting the best
model for a specific task.
 Cross-validation Algorithm
 Divide the dataset into two parts:
 Hold-out
one for training, the other for
 K-folds
testing
 Leave-one-out
 Train the model on the training
set  Leave-p-out

 Validate the model on the test set  Stratified K-folds

 Repeat 1-3 steps a couple of  Repeated K-folds

times. This number depends on  Nested K-folds
the CV method that you are using  Time series CV
 Inner Working Process of Cross-Validation
 Shuffle the dataset in order to remove any kind of order

 Split the data into K number of folds. K= 5 or 10 will work for most of the cases.

 Now keep one fold for testing and remaining all the folds for training.
 Train(fit) the model on the train set and test(evaluate) it on the test set and note
down the results for that split

 Now repeat this process for all the folds, every time choosing a separate fold as
test data
 So for every iteration our model gets trained and tested on different sets of data

 At the end sum up the scores from each split and get the mean score
Inner Working of Cross Validation
 K-fold cross validation
 In K Fold cross-validation input data is divided into 'K' number of folds,
hence the name K Fold.

 Suppose we have divided data into 5 folds i.e. K=5.

 Now we have 5 sets of data to train and test our model.

 So the model will get trained and tested 5 times, but for every iteration we
will use one fold as test data and rest all as training data.

 Note that for every iteration, data in training and test fold changes which
adds to the effectiveness of this method.
K-fold cross-validation
 Stratified K Fold Cross Validation
Stratified K Fold used when just random shuffling and splitting the data
is not sufficient.
 In case of regression problem folds are selected so that the mean response
value is approximately equal in all the folds.
 In the case of classification problems folds are selected to have the same
proportion of class labels.
 Stratified K Fold is more useful in the case of classification problems,
where it is very important to have the same percentage of labels in every
fold.
Stratified K Fold Cross Validation
 Advantages
 We end up using all the data for training and testing and this is very useful
in case of small datasets

 It covers the variation of input data by validating the performance of the

model on multiple folds

 Multiple folds also helps in case of unbalanced data

 Model performance analysis for every fold gives us more insights to fine
tune the model

 Used for hyperparameter tuning

Leave-One-Out Cross-Validation (LOOCV)

• Each data point is treated as its own fold.

• The model is trained on all data points except one, which is used as
the test set. This process is repeated for each data point.
• Uses almost all data for training, providing the best possible estimate
of model performance.
• Computationally very expensive for large datasets.
• May lead to high variance as each test set contains only one data
point.
Train Test Split
• The train-test split procedure is used to estimate the
performance of machine learning algorithms when they are
used to make predictions on data not used to train the model.
The procedure involves taking a dataset and dividing it into two
subsets.
• Train Dataset: Used to fit the machine learning model.
• Test Dataset: Used to evaluate the fit machine learning model.
• The procedure has one main configuration parameter, which is
the size of the train and test sets. For example, a training set
with the size of 0.67 (67 percent) means that the remainder
percentage 0.33 (33 percent) is assigned to the test set.
• common split percentages include:
• Train: 80%, Test: 20%
• Train: 67%, Test: 33%
• Train: 50%, Test: 50%
• Code
# split into train test sets
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.33)
• Repeatable Train-Test Splits
• Rows are assigned to the train and test sets randomly.
• This is done to ensure that datasets are a representative sample
(e.g. random sample) of the original dataset, which in turn, should be
a representative sample of observations from the problem domain.
• This can be achieved by setting the “random_state” to an integer
value
Code
# split into train test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33,
random_state=1)
1. Imagine a phishing or malware classification model where phishing and
malware websites are in the class labeled 1 (true) and harmless websites
are in the class labeled 0 (false). This model mistakenly classifies a
legitimate website as malware. What is this called?

A. A true positive
B. A false positive
C. A false negative
D. A true negative

Correct answer.
A false positive
A negative example (legitimate site) has been wrongly classified as a positive
example (malware site).
2. In general, what happens to the number of false positives when the classification
threshold increases? What about true positives? Experiment with the slider above.
A. Both true and false positives increase.
B. True positives increase. False positives decrease.
C. Both true and false positives decrease.

Correct answer.

Both true and false positives decrease.

• As the threshold increases, the model will likely predict fewer positives overall,
both true and false.
3. In general, what happens to the number of false negatives when the classification
threshold increases? What about true negatives? Experiment with the slider above.
A. Both true and false negatives decrease.
B. True negatives increase. False negatives decrease.
C. Both true and false negatives increase.
Correct answer.

Both true and false negatives increase.

• As the threshold increases, the model will likely predict more negatives overall,
both true and false. At a very high threshold, almost all emails, both spam and not-
spam, will be classified as not-spam.
4. For the given Confusion Matrix find Accuracy ,Precision and Recall.

Correct answer.
5. A model outputs 5 TP, 6 TN, 3 FP, and 2 FN. Calculate the recall.

Correct answer.
0.714
6. A model outputs 3 TP, 4 TN, 2 FP, and 1 FN. Calculate the precision.

Correct answer.

0.6
7. You're building a binary classifier that checks photos of insect traps for
whether a dangerous invasive species is present. If the model detects the
species, the entomologist (insect scientist) on duty is notified.
Early detection of this insect is critical to preventing an infestation.
A false alarm (false positive) is easy to handle: the entomologist sees that the
photo was misclassified and marks it as such. Assuming an acceptable accuracy
level, which metric should this model be optimized for?

Correct answer.

Recall
In this scenario, false alarms (FP) are low-cost, and false negatives are highly
costly, so it makes sense to maximize recall, or the probability of detection.