UNIT–1: INTRODUCTION TO MACHINE LEARNING

1. Overview of Machine Learning

1.1 Definition of Machine Learning
Machine Learning is a branch of Artificial Intelligence that focuses on developing computational methods that
enable systems to learn patterns from data and use those learned patterns to make predictions, classifications,
decisions, or other forms of inference without being explicitly programmed with a separate rule for every
possible situation.
A conventional computer program generally follows explicitly defined rules: input data are processed according
to a predetermined algorithm to produce an output. In machine learning, the system is provided with examples
or observations and an algorithm learns a model that captures useful relationships within the data.
A commonly used formal perspective is that a machine learning system improves its performance on a task as it
gains experience from data. In practice, this means that the quality of a machine learning model depends not
only on the algorithm but also on the quality, quantity, representativeness, and preparation of the data.

1.2 Traditional Programming versus Machine Learning

Aspect Traditional Programming Machine Learning
Rules Explicitly written by the Learned from data
programmer
Input Data + predefined rules Data + expected outcomes or
structure
Output Computed result Learned model or prediction
Adaptability Requires rule modification Can be retrained using new data
Typical use Deterministic business logic Prediction, classification, pattern
recognition

1.3 History and Evolution of Machine Learning

The development of machine learning is closely connected with the broader development of Artificial
Intelligence, statistics, optimization, and computer science. Although the modern term machine learning became
prominent during the twentieth century, the underlying idea of learning from examples has a much longer
intellectual history.
1.​ 1940s–1950s: Early foundations. The development of digital computers, mathematical models of neurons,
and early work in cybernetics established the conceptual foundation for computational learning.
2.​ 1950s–1960s: Early AI and learning systems. Researchers developed systems such as the perceptron and
explored programs capable of pattern recognition and game playing. These systems demonstrated that
machines could learn or adjust parameters based on examples.
3.​ 1970s–1980s: Knowledge-based AI and statistical approaches. Research expanded into expert systems,
symbolic reasoning, decision trees, neural networks, and statistical pattern recognition. The limitations of
computational resources and available data restricted the practical scale of many approaches.
4.​ 1990s: Statistical machine learning. Algorithms such as support vector machines, ensemble methods,
Bayesian methods, and improved decision-tree techniques became increasingly important. The focus shifted
toward generalization, statistical evaluation, and learning from real datasets.
5.​ 2000s: Data-driven machine learning. Increased storage, internet-scale data, and improved processors
enabled organizations to apply machine learning to recommendation systems, search, fraud detection,
advertising, and other large-scale applications.
6.​ 2010s–present: Deep learning and foundation models. Graphics processing units, large datasets, improved
optimization methods, and neural network architectures enabled major advances in computer vision, speech
recognition, natural language processing, generative AI, and autonomous systems.

1.4 Why Machine Learning is Important

●​ It can identify complex relationships in large and high-dimensional datasets.
●​ It can automate repetitive analytical and decision-support tasks.
●​ It can generate predictions that support planning and resource allocation.
●​ It can adapt to changing patterns when models are regularly retrained with appropriate new data.
●​ It enables applications such as recommendation systems, fraud detection, predictive maintenance, image
recognition, and natural-language processing.

2. Types of Machine Learning

Machine learning is commonly classified according to the type of feedback available during learning. The three
foundational paradigms are supervised learning, unsupervised learning, and reinforcement learning.

2.1 Supervised Learning

In supervised learning, the algorithm learns from a dataset in which the desired output, commonly called the
label or target, is known for each training example. The objective is to learn a mapping from input features to the
corresponding output.
If X represents the input features and y represents the target variable, a supervised learning model attempts to
learn a function of the form:
y = f(X) + ε
where f represents the underlying relationship that the model attempts to estimate and ε represents unexplained
variation or noise.
Supervised learning is mainly divided into classification and regression.
●​ Classification predicts a discrete category, such as spam/not spam, disease/no disease, or survived/not
survived.
●​ Regression predicts a continuous numerical value, such as house price, sales revenue, temperature, or
demand.
Examples include predicting student performance, classifying emails as spam, predicting loan default risk, and
estimating the price of a property.

2.2 Unsupervised Learning

In unsupervised learning, the training dataset does not contain predefined target labels. The algorithm attempts
to discover meaningful structure, relationships, groups, or patterns within the data.
●​ Clustering groups observations that are similar according to selected features.
●​ Dimensionality reduction transforms high-dimensional data into a smaller representation while attempting to
preserve important information.
●​ Association analysis identifies relationships among variables or events.
Examples include customer segmentation, grouping similar documents, anomaly exploration, and exploratory
analysis of biological or financial data.
2.3 Reinforcement Learning
Reinforcement learning is a learning paradigm in which an agent interacts with an environment. The agent
observes a state, performs an action, receives a reward or penalty, and learns a strategy, known as a policy, for
selecting actions that maximize cumulative reward.
The basic components are:
●​ Agent: the decision-making system.
●​ Environment: the external system with which the agent interacts.
●​ State: the current situation observed by the agent.
●​ Action: a decision taken by the agent.
●​ Reward: numerical feedback indicating the desirability of an action or outcome.
●​ Policy: a strategy that determines which actions should be selected.
Applications include robotics, game playing, resource allocation, recommendation strategies, and certain control
problems.

2.4 Comparison of Learning Paradigms

Paradigm Training Feedback Typical Tasks Example
Supervised Known labels/targets Classification, regression Email spam detection
Unsupervised No target labels Clustering, Customer segmentation
dimensionality reduction
Reinforcement Rewards/penalties Sequential Robot navigation
decision-making

3. Key Concepts in Machine Learning

3.1 Data
Data are observations or measurements used by a machine learning system. A dataset may contain numerical
values, categorical variables, text, images, audio, sensor measurements, or combinations of these forms.
A dataset is commonly divided into training, validation, and test sets. The training set is used to learn model
parameters, the validation set can be used for model selection and hyperparameter tuning, and the test set is
reserved for evaluating the final model on previously unseen observations.

3.2 Features
Features are measurable properties or input variables used by a model to make a prediction. For example, in
house-price prediction, area, number of bedrooms, number of bathrooms, location, and age of the property can
serve as features.
Feature quality has a direct influence on model performance. Irrelevant, redundant, highly noisy, or improperly
represented features can reduce predictive performance and increase computational complexity.

3.3 Labels or Target Variables

A label is the expected output associated with an observation in supervised learning. The label is also commonly
called the target variable or dependent variable.
●​ Classification label: a category such as 'Yes' or 'No'.
●​ Regression target: a numerical value such as ₹500,000 or 72.5.
The distinction between features and labels is essential. Features are the information supplied to the model,
while the label represents what the model is expected to predict.

3.4 Model
A model is a mathematical or computational representation of patterns learned from data. During training, the
algorithm estimates parameters that allow the model to map input features to outputs or identify structure in the
dataset.
A model should not merely memorize the training data. Its principal objective is generalization: performing well
on new, unseen observations drawn from the same or a sufficiently similar problem domain.

3.5 Training, Validation, and Testing

Training is the process of estimating model parameters from data. Validation is used to compare configurations
and tune hyperparameters. Testing provides an approximately unbiased estimate of how the final selected model
performs on unseen data.
A common division is 70:15:15 or 80:10:10, although the appropriate split depends on dataset size, problem
type, and methodology. For small datasets, cross-validation may provide a more reliable estimate of
generalization performance.

3.6 Parameters and Hyperparameters

Parameters are values learned by the model from training data. For example, the coefficients of a linear
regression model are parameters. Hyperparameters are settings selected before or during model training, such as
the value of k in k-nearest neighbours, maximum tree depth, or the regularization parameter in an SVM.
Hyperparameter tuning should be performed using training and validation procedures rather than by repeatedly
optimizing against the final test set.

3.7 Overfitting and Underfitting

Overfitting occurs when a model learns the training data too closely, including noise or accidental patterns, and
consequently performs poorly on unseen data. Underfitting occurs when a model is too simple to capture the
important structure of the problem.
●​ Underfitting: high error on training data and high error on unseen data.
●​ Good generalization: relatively low training error and acceptable unseen-data error.
●​ Overfitting: very low training error but substantially higher validation/test error.

4. Applications of Machine Learning

4.1 Healthcare
Machine learning is used in medical image analysis, disease-risk prediction, patient monitoring, drug discovery,
personalized treatment support, and hospital operations. For example, classification models can assist in
identifying patterns in medical images, while regression models can estimate continuous clinical outcomes.
Healthcare applications require special attention to data quality, privacy, fairness, interpretability, clinical
validation, and the consequences of incorrect predictions. A model should generally support qualified
professionals rather than be treated as an unquestionable authority.
4.2 Finance and Banking
Financial institutions use machine learning for fraud detection, credit-risk assessment, customer segmentation,
algorithmic decision support, anti-money-laundering analysis, and forecasting.
Fraud detection is often formulated as a classification or anomaly-detection problem. The model identifies
transactions whose characteristics differ from normal behavior. Because fraudulent behavior evolves over time,
models may require continuous monitoring and retraining.

4.3 Marketing and Customer Analytics

Machine learning supports customer segmentation, churn prediction, recommendation systems, campaign
response prediction, demand forecasting, and customer lifetime value estimation.
For example, a churn model may use customer activity, service usage, transaction history, and interaction
patterns to estimate the probability that a customer will discontinue a service.

4.4 Autonomous Systems

Autonomous vehicles, drones, robots, and intelligent machines use machine learning for perception, object
detection, localization, planning, and decision support. Such systems typically combine machine learning with
sensors, control systems, optimization algorithms, and safety mechanisms.
Autonomous systems are safety-critical in many contexts. Therefore, performance must be evaluated under
diverse environmental conditions, including situations that were not strongly represented in the training data.

4.5 Manufacturing and Predictive Maintenance

Industrial organizations use machine learning to monitor equipment and predict potential failures. Sensor
readings such as vibration, temperature, pressure, and energy consumption can be analyzed to identify abnormal
patterns.
Predictive maintenance can reduce unplanned downtime and support more efficient maintenance scheduling
compared with purely reactive maintenance.

4.6 Education
Machine learning can be used for student performance prediction, early identification of students requiring
academic support, personalized learning recommendations, automated assessment support, and learning
analytics.
Educational applications should consider fairness and privacy because model predictions can influence
important academic decisions.

4.7 Natural Language and Computer Vision

Natural language processing uses machine learning to analyze and generate human language for tasks such as
sentiment analysis, text classification, translation, question answering, and information extraction. Computer
vision applies machine learning to images and videos for tasks such as classification, object detection,
segmentation, and recognition.
5. Case Studies and Industry Examples
5.1 Recommendation Systems
E-commerce and streaming platforms use recommendation models to estimate which products, films, songs, or
other items may be relevant to a user. Models can use historical interactions, item characteristics, and similarities
among users or items.
The business objective is not simply prediction accuracy. A practical recommendation system may also consider
diversity, novelty, user satisfaction, latency, business constraints, and feedback loops.

5.2 Credit Risk Assessment

A financial institution may develop a supervised classification model to estimate whether an applicant is likely
to default. Features can include financial history, repayment behavior, income-related variables, and other
legally permissible information.
The system can support risk assessment, but organizations must evaluate explainability, fairness, data quality,
regulatory requirements, and the impact of incorrect decisions.

5.3 Predictive Maintenance in Manufacturing

Suppose a manufacturing plant collects vibration and temperature measurements from industrial machines.
Historical records can be associated with failure events. A classification model can learn patterns associated
with impending failure, while anomaly-detection methods can identify unusual operating behavior.
The resulting system can generate maintenance alerts before a severe failure occurs, potentially reducing
downtime and maintenance costs.

5.4 Medical Image Classification

Medical imaging systems can use machine learning to identify patterns associated with particular conditions.
Image features may be learned automatically using modern deep learning methods, while traditional approaches
may use manually engineered features.
A clinically useful system requires evaluation on representative external data and must account for false
positives, false negatives, population differences, and the clinical consequences of errors.

6. Key Machine Learning Algorithms

This section introduces four foundational algorithms. They illustrate different approaches to learning
relationships from data and are useful for understanding broader machine learning concepts.

6.1 Linear Regression

Linear regression is a supervised learning algorithm primarily used for predicting a continuous numerical target.
It assumes that the expected value of the target can be approximated as a linear combination of the input
features.
For a single feature, the model can be expressed as:
ŷ = β₀ + β₁x
For multiple features:
ŷ = β₀ + β₁x₁ + β₂x₂ + … + β x
Here, ŷ is the predicted value, β₀ is the intercept, β₁ through β are model coefficients, and x₁ through x are
input features.
The ordinary least squares method estimates the coefficients by minimizing the sum of squared residuals
between observed and predicted values:
SSE = Σ(yᵢ − ŷᵢ)²
Advantages:
●​ Simple and computationally efficient.
●​ Easy to interpret when the assumptions are reasonably satisfied.
●​ Useful as a baseline model.
Limitations:
●​ Cannot naturally represent strongly nonlinear relationships without feature transformations.
●​ Sensitive to outliers.
●​ Performance can be affected by multicollinearity among predictors.
Typical applications include house-price prediction, sales forecasting, demand estimation, and estimation of
continuous physical measurements.

6.2 Decision Trees

A decision tree is a supervised learning algorithm that represents decisions as a tree-like structure. Internal nodes
correspond to feature-based tests, branches represent outcomes of those tests, and leaf nodes contain the final
prediction.
For classification, a tree can select splits using measures such as Gini impurity or entropy. Gini impurity for a
node can be expressed as:
Gini = 1 − Σ pₖ²
where pₖ is the proportion of observations belonging to class k.
Entropy can be expressed as:
Entropy = −Σ pₖ log₂(pₖ)
For regression, decision trees commonly choose splits that reduce prediction error, often measured through
variance or squared error.
Advantages:
●​ Can model nonlinear relationships.
●​ Can handle numerical and categorical features with suitable preprocessing.
●​ Easy to visualize and explain for small trees.
●​ Requires relatively little feature scaling.
Limitations:
●​ Deep trees can overfit the training data.
●​ Small changes in training data can sometimes produce substantially different trees.
●​ A single tree may be less accurate than ensemble methods such as random forests or gradient boosting.
6.3 k-Nearest Neighbours (k-NN)
k-Nearest Neighbours is a supervised, instance-based learning algorithm. To make a prediction for a new
observation, the algorithm identifies the k training observations that are closest to it according to a selected
distance measure.
For numerical features, Euclidean distance between two observations x and z can be written as:
d(x,z) = √Σ(xⱼ − zⱼ)²
For classification, the class with the majority representation among the k nearest neighbours is typically
selected. For regression, the target values of the neighbours can be averaged or combined using a
distance-weighted method.
The choice of k is important. A very small k can make the model sensitive to noise, while a very large k can
oversmooth local patterns.
Advantages:
●​ Conceptually simple.
●​ No explicit parametric training model is required.
●​ Can model nonlinear decision boundaries.
Limitations:
●​ Prediction can be computationally expensive for large datasets.
●​ Sensitive to feature scaling.
●​ Performance can degrade in high-dimensional spaces because distance measures become less informative.
Applications include pattern recognition, recommendation-related similarity tasks, and classification of
observations based on local similarity.

6.4 Support Vector Machines (SVM)

Support Vector Machine is a supervised learning algorithm that can be used for classification and, in modified
forms, regression. For binary classification, an SVM seeks a decision boundary that separates classes while
maximizing the margin between the boundary and the closest training observations.
The observations that determine the position of the optimal boundary are called support vectors.
For linearly separable data, a hyperplane can be represented as:
wᵀx + b = 0
The margin is related to the distance between the separating hyperplane and the closest observations.
Maximizing this margin is a central principle of SVM learning.
SVMs can use kernel functions to represent nonlinear decision boundaries. Common kernels include linear,
polynomial, and radial basis function (RBF) kernels.
Advantages:
●​ Effective for high-dimensional feature spaces.
●​ Can produce strong classification performance when an appropriate kernel and hyperparameters are
selected.
●​ The margin-based formulation provides a principled approach to generalization.
Limitations:
●​ Training and hyperparameter tuning can become computationally demanding for very large datasets.
●​ Feature scaling is generally important.
●​ Kernel and parameter selection can significantly affect performance.
●​ The resulting model can be less directly interpretable than a small decision tree.

7. Comparative Summary of the Four Algorithms

Algorithm Learning Typical Task Main Strength Main Limitation Scaling Need
Linear Supervised Regression Simple and Limited nonlinear Often useful
Regression interpretable modeling
Decision Tree Supervised Classification/Regression Nonlinear rules Can overfit Usually not
and required
interpretability
k-NN Supervised Classification/Regression Simple local Slow prediction; Important
similarity high-dimensional
issues
SVM Supervised Classification/Regression Strong Parameter/kernel Important
margin-based sensitivity
generalization

8. Important Terminology
Term Meaning
Artificial Intelligence The broader field concerned with building systems capable of
tasks associated with intelligent behavior.
Machine Learning A field of AI that develops methods for learning patterns from
data.
Feature An input variable used by a model.
Label/Target The expected output in supervised learning.
Model A learned representation used to make predictions or identify
structure.
Training The process of learning model parameters from data.
Inference Using a trained model to produce predictions or decisions for
new data.
Classification Predicting a discrete class or category.
Regression Predicting a continuous numerical value.
Clustering Grouping observations based on similarity without predefined
labels.
Overfitting Learning training-specific patterns that do not generalize well.
Underfitting Using a model that is too simple to capture important patterns.
Generalization The ability of a model to perform well on unseen data.
Hyperparameter A configuration selected outside the ordinary
parameter-learning process.