Introduction

To Machine
learning
CHAPTER ONE

Introduction
 What Is Machine Learning?
 Learning-to gain knowledge, or understanding of, or skill in, by study, instruction, or experience,

and modification of a behavioral tendency by experience.

E.g A child touches a hot stove and gets burned. Before the experience, the child’s tendency might be to touch

interesting objects freely. After the painful experience, the child avoids touching hot surfaces. That change in behavior is

modification of a behavioral tendency by experience.

 A machine learns whenever it changes its structure, program, or data (based on its inputs or in

response to external information) in such a manner that its expected future performance improves.

 for example, when the performance of a speech-recognition machine improves after hearing

several samples of a person’s speech, we feel quite justified in that case to say that the machine has

learned.

 In machine learning, the term “machine” simply refers to a computer system or algorithm that

can
 What Is Machine Learning?

 Machine learning usually refers to the changes in systems that perform tasks associated with

artificial intelligence (AI). Such tasks involve recognition, diagnosis, planning, robot control,

prediction, etc.

 Machine learning is the design and study of software artifacts that use past experience to

inform future decisions.

 Machine learning is the study of programs that learn from data.

 Machine Learning is the science (and art) of programming computers so they can learn from data.
 What Is Machine Learning?

More general definition:

 Machine Learning is the field of study that gives computers the ability to learn without being

explicitly programmed.

And a more engineering-oriented one:

 A computer program is said to learn from experience E with respect to some task T and some

performance measure P, if its performance on T, as measured by P, improves with experience

E.
 Why Should Machines Have To Learn?
 Some tasks cannot be defined well except by example-we might be able to specify input/output

pairs but not a concise relationship between inputs and desired outputs.

 Machines would be able to adjust their internal structure to produce correct outputs for a large number

of sample inputs.

Example: Handwriting recognition.

Input: Images of handwritten digits (0–9).

Output: The correct digit.

Problem: Writing explicit rules for every possible handwriting style is impossible.

 Instead, we show the system thousands of examples so it learns the mapping.

 Why Should Machines Have To Learn?

 It is possible that hidden among large piles of data are important relationships and correla-

tions.

Machine learning methods can often be used to extract these relationships (data mining).

Example: Medical data analysis.

From thousands of patient health records, ML can uncover correlations (e.g., “smoking +

certain genetic markers → higher cancer risk”).

 Doctors may not detect these hidden patterns, but ML can highlight them for prevention or

diagnosis.
 Why Should Machines Have To Learn?

 The amount of knowledge available about certain tasks might be too large for explicit encoding

by humans. Machines that learn this knowledge gradually might be able to capture more of it than

humans would want to write down. Some domains have so much knowledge that humans cannot

hand-code all the rules.

Example: Language translation.

 Human languages have millions of grammar rules, exceptions, idioms, and vocabulary.

 It’s impossible for humans to explicitly program all of these into a computer.

 Instead, ML models (like Google Translate) learn from billions of bilingual text examples.
 Why Should Machines Have To Learn?

 Environments change over time- Machines that can adapt to a changing environment

would reduce the need for constant redesign.

Machines need to adapt without humans constantly reprogramming them.

Example: Spam email filtering.

• Spammers constantly invent new tricks to bypass filters.

• A hard-coded set of rules would quickly become outdated.

 ML-based spam filters learn from new data (examples of spam/non-spam) and

update themselves to adapt to changing strategies.

 Why Should Machines Have To Learn?

To summarize, Machine Learning is great for:

 Problems for which existing solutions require a lot of hand tuning or long lists of rules:

one Machine Learning algorithm can often simplify code and perform better.

 Complex problems for which there is no good solution at all using a traditional ap-

proach: the best Machine Learning techniques can find a solution.

 Fluctuating environments: a Machine Learning system can adapt to new data.

 Getting insights about complex problems and large amounts of data.

 History And Relationships To Other Fields
1940s–1950s: Foundations of Artificial Intelligence

 1943 – McCulloch & Pitts: Created the first mathematical model of a neuron (the artificial

neuron). This was the inspiration for modern neural networks.

 1950 – Alan Turing: Published Computing Machinery and Intelligence, proposing the Turing

Test to evaluate machine intelligence.

 1952 – Arthur Samuel: Developed a checkers-playing program that improved its perfor-
mance

the more it played. He coined the term “machine learning.”

 1957 – Frank Rosenblatt: Introduced the Perceptron, the first trainable neural network for

classification problems.
 History And Relationships To Other Fields

1960s: Early Algorithms

 1960 – ADALINE (Adaptive Linear Neuron) by Bernard Widrow & Marcian Hoff, used

for signal processing and adaptive filtering.

 1967 – Nearest Neighbor algorithm introduced, which laid the foundation for K-Near-
est

Neighbors (KNN).

 AI research was popular, but computers were very limited in memory and speed, so ML

progress was slow.

 History And Relationships To Other Fields

1970s: First Setbacks (AI Winter I)

 Researchers discovered that Perceptrons could not solve non-linear problems (e.g., XOR

problem).

 Marvin Minsky & Seymour Papert’s book Perceptrons (1969) showed these limitations →

funding for neural network research decreased.

 Focus shifted to symbolic AI (rule-based systems), where machines followed explicit

rules instead of Learning.

 History And Relationships To Other Fields

1980s: Revival with Better Mathematics

 Backpropagation algorithm (1986) rediscovered/popularized by Rumelhart, Hinton, and

Williams → allowed multi-layer neural networks (MLPs) to be trained effectively.

 Decision trees (like ID3 algorithm by Quinlan, 1986) became widely used for classifica-
tion tasks.

 Expert systems dominated AI, but machine learning gained traction in pattern recognition

and speech recognition.

 History And Relationships To Other Fields

1990s: Statistical Machine Learning

 Shift from symbolic AI to probabilistic and statistical models.

 Key developments:

Support Vector Machines (SVMs, 1995) → powerful for classification problems.

Naïve Bayes classifiers became popular for text classification.

Reinforcement Learning (Q-learning, 1989 by Watkins, applied widely in 1990s).

 Data and computational power increased, making ML more practical.

 History And Relationships To Other Fields

2000s: Big Data & Ensemble Learning

 Growth of the internet led to huge datasets → ML became essential.

 Ensemble methods like Random Forests (Breiman, 2001) and Boosting (AdaBoost, 1995
→ improved in 2000s) increased predictive accuracy.

 Kernel methods expanded the power of SVMs (Support Vector Machine).

 ML applied in spam detection, recommendation systems, and web search (Google, Ama-
zon,

Netflix).
 History And Relationships To Other Fields

2010s: Deep Learning Revolution

 2012 – ImageNet competition: Geoffrey Hinton’s team used deep convolutional neural

networks (CNNs) to dramatically outperform traditional methods. This sparked the deep

learning boom.

 Advances in GPUs (faster computing) and big data fueled progress.

 Applications exploded: computer vision, speech recognition (Siri, Alexa), natural language

processing (translation, chatbots), and autonomous vehicles.

 Notable architectures: CNNs, Recurrent Neural Networks (RNNs), LSTMs (long short-term

memory networks), and GANs (Generative Adversarial Networks, 2014).

 History And Relationships To Other Fields

2020s: Modern ML & Generative AI

 1. Transformers revolutionized NLP (2017 onward):Transformers are a type of neural

network architecture introduced in 2017.

 Key idea: attention mechanisms allow the model to focus on important words or tokens in

a sentence, instead of processing words sequentially like RNNs.

 Examples: BERT, GPT, T5.

 Impact: Transformers drastically improved natural language understanding and generation

tasks (like translation, summarization, Q&A).

 History And Relationships To Other Fields

2020s: Modern ML & Generative AI

 Large Language Models (LLMs) dominate AI research.

 LLMs are models trained on massive amounts of text data to understand and generate

human-like language.

Examples: GPT-3, GPT-4, and beyond.

Applications:

o Chatbots (e.g., ChatGPT)

o Text generation, summarization, translation

o Coding assistance, content creation

 History And Relationships To Other Fields

2020s: Modern ML & Generative AI

Reinforcement Learning + Deep Learning

 Combines trial-and-error learning (RL) with deep neural networks (DL).

Examples:

• AlphaGo (2016): AI defeated human Go champions.

• AlphaZero: Learned chess, Go, and Shogi from scratch without human examples.

• Robotics: Helps robots learn complex tasks (grasping, navigation) by experiment-

ing in simulations.
 History And Relationships To Other Fields
 Machine Learning is naturally a subfield of computer science.

 As an interdisciplinary field, machine learning shares common threads with the mathematical fields of

statistics, information theory, game theory, and optimization.

 In a sense, machine learning can be viewed as a branch of AI (Artificial Intelligence), since, after

all, the ability to turn experience into expertise or to detect meaningful patterns in complex sensory

data is a cornerstone of human (and animal) intelligence.

 However, in contrast with traditional AI, machine learning is not trying to build automated imitation

of intelligent behavior, but rather to use the strengths and special abilities of computers to

complement human intelligence, often performing tasks that fall way beyond human capabilities.
 History And Relationships To Other Fields
 In machine learning, the concept of experience or training usually refers to data, often generated

in a random manner. The learner’s role is to analyze these examples and draw conclusions that

generalize to the larger environment from which the data originates. This approach highlights the

strong connection between machine learning and statistics.

 However, there is a key distinction: in statistics, it is common to assume specific data models in

advance (for example, assuming that data follows a normal distribution or that relationships are

linear). In contrast, machine learning typically works in a distribution-free setting.

Instead of making strong assumptions about the data, the learning algorithm is designed to discover

which models best capture the underlying data-generating process.

 Essential Math And Statistics For Machine Learning
1. Linear Algebra

What it is: The study of vectors, matrices, and linear transformations.

Why it matters in ML:

o Data representation: Features of a dataset are often stored as vectors or matrices.

o Model representation: Images, audio, or text embeddings are expressed in high-dimensional

vectors.

o Core algorithms: Linear regression, Principal Component Analysis (PCA), Support Vector

Machines (SVMs), and Neural Networks use linear algebra heavily.

o Matrix operations: Multiplications, eigenvalues, and eigenvectors are used in dimensionality

reduction and optimization.
 Eigenvalues are special numbers that tell us how a linear transformation (or a square matrix) stretches or shrinks vectors

along specific directions called eigenvectors.

 Essential Math And Statistics For Machine Learning
2. Calculus (Differential and Integral)

What it is: The study of continuous change (derivatives, integrals, gradients).

Why it matters in ML:

o Optimization: Training ML models involves minimizing a loss function. Calculus

(gradients & partial derivatives) powers gradient descent and backpropagation.

o Understanding learning dynamics: Derivatives show how small changes in

parameters affect model performance.

o Integrals: used to describe continuous change, probabilities, expectations, and

optimization.
 Essential Math And Statistics For Machine Learning

3. Probability

What it is: The mathematics of uncertainty and randomness.

Why it matters in ML:

o Modeling uncertainty: Probabilistic models (Naïve Bayes, Hidden Markov Model,

Bayesian Networks).

o Hypothesis testing: Evaluating whether patterns in data are significant.

o Likelihood functions: Used in maximum likelihood estimation (MLE) to fit models

o Bayesian methods: Incorporate prior knowledge into learning (Bayesian inference)

 Essential Math And Statistics For Machine Learning
4. Statistics

What it is: Collecting, analyzing, and interpreting data.

Why it matters in ML:

o Data understanding: Mean, variance, standard deviation summarize data.

o Feature selection: Correlation, hypothesis testing.

o Evaluation metrics: Precision, recall, F1 score, ROC(Receiver Operating Characteris-

tic) curves are rooted in statistics.

o Statistical inference: Helps generalize from a sample (training data) to a population

(real-world).
 Essential Math And Statistics For Machine Learning

5. Linear Optimization / Convex Optimization

What it is: Finding the best solution (minimum or maximum) given constraints.

Why it matters in ML:

o Loss function minimization: Training models relies on optimization methods.

o Convex functions: Many ML models (e.g., linear regression, logistic regression) rely on
convex optimization for efficient training.
 Essential Math And Statistics For Machine Learning

6. Discrete Mathematics

What it is: Study of finite structures (logic, sets, combinatorics, graphs).

Why it matters in ML:

o Logic: Used in decision trees, Boolean networks.

o Graph theory: Used in social networks, recommendation systems, Graph Neural

Networks (GNNs).

o Combinatorics: Helps analyze algorithm complexity and probability outcomes.

 Applications Of Machine Learning

Facial Recognition Product Recommendations

Email Automation & Spam Filtering Financial Accuracy

Social Media Optimization Healthcare Advancement

Mobile Voice & Predictive Text Predictive Analytics

 Applications Of Machine Learning

1. Facial Recognition

 Uses ML to identify and verify individuals through facial patterns.

 Applications: photo tagging, healthcare diagnostics, security, and combating

exploitation.
 Applications Of Machine Learning

2. Product Recommendations

 Retailers like Amazon use ML to analyze buying habits and demographics to predict

customer needs.

 Improves targeted marketing and customer experience.

 Applications Of Machine Learning

3. Email Automation & Spam Filtering

 ML filters spam by recognizing patterns in email content, domains, IPs, and user

feedback.

 Supports efficient inbox management and automation.

 Applications Of Machine Learning

4. Financial Accuracy

 ML detects fraudulent transactions, enables mobile check deposits, credit scoring,

and enhances digital banking with AI-powered risk analysis.

 Applications Of Machine Learning

5. Social Media Optimization

 Platforms use ML for content suggestions, ad targeting, and fighting inappropriate

content or cyberbullying.

 Strengthens user engagement and platform safety.

 Applications Of Machine Learning

6. Healthcare Advancement

 ML improves diagnoses, lowers treatment costs, supports preventive care, and

integrates with wearables for real-time monitoring.

 Used in radiology, cardiology, and pathology.

 Applications Of Machine Learning

7. Mobile Voice & Predictive Text

 Voice assistants (Siri, Cortana) use ML to convert speech to text.

 Predictive text learns user-specific vocabulary for better typing suggestions.

 Applications Of Machine Learning

8. Predictive Analytics

 ML predicts future trends by analyzing historical and real-time data.

 Helps organizations identify risks, opportunities, and improve decision-making.

Types of Machine Learning Techniques

Classification Criteria
 Based on supervision
 Based on learning mode
 Based on generalization
 Classification Criteria

Based on supervision:
 Supervised

 Unsupervised

 Semi-supervised

 Reinforcement Learning

Based on learning mode:

 Online vs Batch Learning

Based on generalization:
 Instance-based vs Model-based Learning
 Based on supervision: Supervised Learning

 Training data includes labels

 Tasks: Classification, Regression

 Examples: Spam filtering, price prediction

 Algorithms: k-NN, Linear/Logistic Regression, SVMs, Decision Trees, Random

Forests, Neural Networks

 Based on supervision: Unsupervised Learning

 Data is unlabeled; system finds patterns

 Algorithms: Clustering (K-Means, DBSCAN, HCA), PCA, t-SNE

 Applications: Visitor grouping, anomaly/novelty detection, dimensionality reduction,

association rules
 Based on supervision: Semi-supervised Learning

 Mix of labeled and unlabeled data

 Example: Google Photos face recognition

 Algorithms: Deep Belief Networks (DBNs), Restricted Boltzmann Machines

(RBMs)

 Combines clustering with minimal labels

 Based on supervision: Reinforcement Learning

 Agent learns by interacting with environment

 Gets rewards or penalties (trial-and-error)

 Applications: Robotics (walking), AlphaGo game strategy

 Learns optimal policy to maximize long-term reward

 Based on learning mode: Batch Learning

 Learns from all available data at once

 Offline training, resource-intensive

 Requires retraining for new data

 Automated pipelines update models periodically

 Based on learning mode: Online Learning

 Learns incrementally from streaming data

 Fast, adaptive, suitable for large datasets

 Useful for stock prediction, autonomous systems

 Learning rate controls adaptation speed

 Based on generalization: Instance-Based Learning

 Memorizes training examples

 Generalizes using similarity measures

 Example: Spam filter using word similarity

 Simple but limited in generalization ability

 Based on generalization: Model-Based Learning

 Builds predictive models from data

 Example: Linear Regression for GDP vs Life Satisfaction

 Involves model selection, training, and evaluation

 Goal: fit data and predict unseen cases

 Fitting the data means the model learns the pattern from the given dataset (training data).
Thank you!