UNIT 6: Machine Learning Algorithms
MACHINE LEARNING:
Machine Learning (ML) is a part of artificial intelligence (AI) that focuses on teaching computers to
learn from data and make decisions without being explicitly programmed.
• ML algorithms learn from various types of data, including images, text, sensor readings, and
historical records.
• Some common ML algorithms include decision trees, neural networks, and support vector
machines.
• Artificial intelligence (AI) and machine learning (ML) have significantly impacted various
aspects of our lives. From transportation and finance to healthcare and entertainment, AI
algorithms are pervasive.
TYPES OF MACHINE LEARNING:
Three main types of machine learning: Supervised Learning, Unsupervised Learning, and
Reinforcement Learning.
• Supervised learning involves the model learning from labeled data, where the input data is
accompanied by the correct output. Examples include linear regression, logistic regression,
decision trees, support vector machines, and neural networks.
• Unsupervised learning, on the other hand, deals with unlabelled data, where the algorithm
tries to find hidden patterns or structure without explicit guidance. Examples include k-
means clustering, hierarchical clustering, principal component analysis, and autoencoders.
• Reinforcement learning involves an agent learning to make decisions by interacting with an
environment to maximize cumulative rewards. Through trial and error, the agent learns a
policy or strategy to take actions that lead to the highest cumulative reward over time. These
rewards serve as feedback, guiding the agent towards favorable actions, while penalties
discourage undesirable behavior.
A. SUPERVISED LEARNING:
Within supervised learning, two primary types of algorithms emerge:
1. Regression – works with continuous data
2. Classification – works with discrete data
1. REGRESSION
Understanding Correlation- The Foundation of Regression Analysis. Correlation is a
measure of the strength of a linear relationship between two quantitative variables (e.g.
price, sales). If the change in one variable appears to be accompanied by a change in the
other variable the two variables are said to be correlated and this inter dependence is called
correlation.
Types of Correlation:
• Positive Correlation: In a positive correlation, both variables move in the same
direction. As one variable increases, the other also tends to increase, and vice versa.
• Negative Correlation: Conversely, in a negative correlation, variables move in
opposite directions. An increase in one variable is associated with a decrease in the
other, and vice versa.
• Zero Correlation: When there is no apparent relationship between two variables,
they are said to have zero correlation. Changes in one variable do not predict
changes in the other
Correlation can have a value:
▪ 1 is a perfect positive correlation
▪ 0 is no correlation (the values don't seem linked at all)
▪ -1 is a perfect negative correlation
Causation:
Causation indicates that one event is the result of the occurrence of the other event. Example: Since
there is hot weather, the person will use more sunscreen or eat more ice cream.
Pearson's R correlation coefficient:
Pearson's r measures the strength and direction of the linear relationship between two continuous
variables.
Regression analysis may not be suitable in certain situations:
• No Correlation
• Non-linear Relationships
• Outliers
• Violation of Assumptions
REGRESSION
Regression is a statistical technique used to model the relationship between a dependent variable
and one or more independent variables. Regression analysis is particularly useful when dealing with
continuous data, where variables can take on any value within a certain range. For example, variables
such as height, temperature, salary, and time are all continuous. When we make a distribution in
which there is an involvement of more than one variable, then such an analysis is called Regression
Analysis.
Some of the regression algorithms include Linear Regression, Logistic Regression, Decision Tree
Regression, Random Forest Regression. Let us learn about Linear Regression.
Linear Regression
Linear regression is one of the most basic types of regression in machine learning. The linear
regression model consists of a predictor variable and a dependent variable related linearly to each
other.
Linear regression is further divided into two types:
a) Simple Linear Regression: The dependent variable's value is predicted using a single independent
variable in simple linear regression.
b) Multiple Linear Regression: In multiple linear regression, more than one independent variable is
used to predict the value of the dependent variable.
Applications of Linear Regression:
1. Market Analysis
2. Sales Forecasting
3. Predicting Salary Based on Experience
4. Sports Analysis
5. Medical Research
Advantages of Linear regression
• Simple technique and easy to implement
• Efficient to train the machine on this model
Disadvantages of Linear regression
• Sensitivity to outliers, which can significantly impact the analysis.
• Limited to linear relationships between variables.
2. CLASSIFICATION
Classification is a fundamental concept in artificial intelligence and machine learning that involves
categorizing data into predefined classes or categories. The main objective of classification is to
assign labels to data instances based on their features or attributes.
K- Nearest Neighbour algorithm (KNN)
The K-Nearest Neighbors algorithm, commonly known as KNN or k-NN, is a versatile non-parametric
supervised learning technique used for both classification and regression tasks. It operates based on
the principle of proximity, making predictions or classifications by considering the similarity between
data points.
Applications of KNN:
• Image recognition and classification
• Recommendation systems
• Healthcare diagnostics
• Text mining and sentiment analysis
• Anomaly detection
Advantages of KNN:
• Easy to implement and understand.
• No explicit training phase; the model learns directly from the training data.
• Suitable for both classification and regression tasks.
• Robust to outliers and noisy data.
Limitations of KNN:
• Computationally expensive, especially for large datasets.
• Sensitivity to the choice of distance metric and the number of neighbors (K).
• Requires careful preprocessing and feature scaling.
• Not suitable for high-dimensional data due to the curse of dimensionality.
B. UNSUPERVISED LEARNING
3. CLUSTERING
Clustering, or cluster analysis, is a machine learning technique used to group unlabeled dataset into
clusters or groups based on similarity. It is an unsupervised learning method, hence no supervision is
provided to the algorithm, and it deals with the unlabeled dataset.
K- Means clustering
K-Means Clustering is an unsupervised learning algorithm that is used to solve the clustering
problems in machine learning or data science. The k-means algorithm is one of the most popular
clustering algorithms. It classifies the dataset by dividing the samples into different clusters of equal
variances.
Advantages of K-Means Clustering:
• Easy to implement, making it suitable for users of all levels.
• Handles large datasets with low computational resources.
• Works well with numerous features and data points.
• Are easy to understand, aiding in decision-making.
• Applicable across various domains and data types.
Limitations of K-Means Clustering:
• Results can vary based on initial centroid placement.
• Assumes clusters are spherical, which is not always true.
• Number of clusters must be known beforehand.
• Outliers can distort clusters due to their influence on centroids.
• May converge to suboptimal solutions instead of the global optimum.