Machine Learning
Dr. Sanga Chaki
Assistant Professor
Department of Computer Science and Engineering
IIIT Pune
Introduction
Contents
1. Course related General Information
2. Introduction to ML
3. Brief history of ML
4. Applications of ML
5. Different types of Learning
6. What is learning?
7. Learning Algos as Black Box
8. Steps to build a learner
9. Hypothesis space
10. Supervised Learning Concepts and Terminology
Course-Related Information
Course Evaluation
1. 3 credit theory course – 100 marks
2. Google classroom –
3. End Term – 50 marks
4. Mid term – 30
5. Continuous evaluation – 20
a) Attendance – 5 marks
b) Quiz – 15 marks
Course overview – Syllabus - Resources
1. Introduction: Basic principles, Applications, Challenges
2. Supervised learning:
• Linear Regression (with one variable and multiple variables),
• Classification, (Logistic Regression, Overfitting, Regularization, Support Vector
Machines),
• Gradient Descent,
• Decision Trees
• K Nearest Neighbours
• Bayesian Learning
• Artificial Neural Networks etc
3. Unsupervised learning:
• Clustering (K-means, Hierarchical),
• Dimensionality reduction,
• Principal Component Analysis, etc
Course overview – Syllabus - Resources
4. Theory of Generalization:
• In-sample and out-of-sample error,
• VC inequality, VC analysis
• Bias and Variance analysis
5. Introduction to Deep Learning
6. Introduction to Reinforcement Learning
7. Reference Books:
• Pattern Recognition and Machine Learning by Christopher M. Bishop.
• Machine Learning by Tom Mitchell.
• Richard O. Duda, Peter E. Hart, David G. Stork. Pattern Classification
Introduction to ML
Introduction to ML
1. ML is one of the key enabler or AI – which is making great strides today
2. It is about making computers act without explicitly programming them
3. ML algorithms
• Figure out how to perform tasks based on generalizing from data/examples
• Learn to improve themselves from past experience
4. All pervasive today
• Business
• Industry
• Science
• Government
• World problems
• Financial
• Medical apps
Introduction to ML
1. Some key contributions till date
1. Self driving cars
2. Automatic Speech recognition system
3. Effective web search systems
4. Vastly improved understanding of human genome etc
Brief History of ML
1. Machine learning - the term was coined by Arthur Samuel in 1959
2. It described a computer program that could learn to play checkers without
explicit programming.
3. His work built on the concept of AI - which was introduced by Alan Turing in
the 1950s with his proposal of the Turing Test for machines.
Brief History of ML
Brief History of ML
Brief History of ML
1. Recent popularity reasons:
a) New software/algorithms
• Neural networks
• Deep learning
b) New hardware – GPUs
c) Cloud enables
d) Availability of Big Data
Introduction to ML: Domains & Applications
Introduction to ML
Introduction to ML
Introduction to ML
Introduction to ML
1. In Business analytics
Introduction to ML
1. Miscellaneous
Different Types of ML
1. Types of Machine Learning
a. Supervised
b. Unsupervised
c. Reinforcement
d. Semi supervised
Learning in ML
Learning in ML
1. Learning is the ability to improve one’s behaviour
with experience
2. Machine learning is about building computer
systems that improve behaviour with experience
3. ML explores algorithms that
a. Learn from data/Build models from data
b. Models which can be used for
• prediction or
• decision making or
• solving different tasks
ML: Formal Definition
1. Tom M. Mitchell defined machine learning as:
"A computer program is said to learn from experience E with respect to some
class of tasks T and performance measure P, if its performance on T, as
measured by P, improves with experience E".
2. In simpler terms, a machine learning model
• improves its ability to perform a specific task (T)
• as it gains more experience (E),
• and this improvement is measured by a performance metric (P).
Introduction to ML
1. Components of a leaning algorithm:
a. Behaviour or Task T – ML algos seeks to improve this behaviour
a. Can be prediction
b. Classification
c. Acting on an environment etc
b. Data or Experience E – used for improving at task
c. Measure of improvement – P
a. Example: Increase accuracy in classification etc
Introduction to ML
Introduction to ML
Y = f(X)
Introduction to ML
1. Concept of Best-fit model to
given training data.
2. Or fitting the model to the data
3. What is that formula f(x) which
Y = f(X) can generate Y, given some X?
4. Is it linear?
5. Is it quadratic?
6. Is it third-degree or even higher
order polynomial?
fitting the model to the data vs fitting the data to the model?
Introduction to ML
Now that we know what a learner is, how do we build/create one?
1. Choose the training experience/ training data
2. Choose target function – how we want to represent the model – this is what
needs to be learnt
• Example: For chess model – target function would be, given a board position, what is the
next move to take for best possible outcome
3. Choose how to represent the target function – is it linear function or non-
linear function, richer or not
4. Choose a learning algorithm to infer the target function from hypothesis
space
Introduction to ML: Hypothesis Space
1. Hypothesis: A supposition or proposed
explanation made on the basis of limited
evidence as a starting point for further
investigation.
2. The hypothesis space in machine learning
refers to the set of all possible functions or
models that a learning algorithm can
potentially select from to solve a given
problem.
3. It represents the range of hypotheses that
the algorithm considers during the training
process to find the best fit for the data.
Introduction to ML: Hypothesis Space
1. Which of these is better?
2. The red curve fits the given data perfectly.
3. So, should we choose this formula (model)?
4. What can be issues?
a. Overfitting
b. Increased Computational Cost
c. Reduced Interpretability
d. Risk of Bias Amplification etc
5. What to do? – Find a target function from the hypothesis space
• rich enough to represent the patterns of the data well,
• at the same time it should be able to generalize to newer inputs well too.
Supervised Learning Concepts
Supervised Learning
1. Supervised learning is a subcategory of
machine learning where an algorithm
learns to map input data to a specific
output based on example input-output
pairs.
2. This method requires labeled datasets for
training - each input data point is
associated with a known, correct output
label.
3. Given data X, Y, what is best value of Y for
an unknown datapoint X
Supervised Learning - Schematic
Supervised Learning
X1 X2 X3 X4 X5 Xn Y
a1 a2 a3 a4 a5 … an y1
b1 b2 .. .. .. bn y2
.. .. .. .. .. .. ..
o1 o2 .. .. .. on yn
z1 z2 z3 .. .. zn ??
Supervised Learning
Features
Target
X1 X2 X3 X4 X5 Xn Y
a1 a2 a3 a4 a5 … an y1
b1 b2 .. .. .. bn y2
.. .. .. .. .. .. ..
o1 o2 .. .. .. on y15
z1 z2 z3 .. .. zn ??
Training Data Test Data/Input Required Output from
Trained Model: Unknown
Features
1. Individual data points/observations are analysed into a set of quantifiable
properties which are called features.
2. Should have relevance to target – feature selection for ML tasks is an
important study area
3. Features may be of different types
a. Categorical: Eg. Blood types
b. Integer values: Number of wheels in a car/words in a text
c. Real values: Readings of a meter etc
Supervised Learning
Supervised Learning: Classification
1. Example: Credit
Scoring
2. Differentiating
between low risk
and high risk
customers from
their income and
savings
3. Binary
classification
Supervised Learning: Classification
1. Example: Credit Scoring
2. If we want to have
three classes:
• Low risk
• Medium risk
• High risk
3. How many
discriminants do we
need?
4. Multiclass
classification
Medium Risk Customers
Supervised Learning: Classification
1. Example: Credit Scoring
2. If we want to have
three classes:
• Low risk
• Medium risk
• High risk
3. How many
discriminants do we
need?
4. Multiclass
classification –
Increased complexity
of learning task? Medium Risk Customers
Supervised Learning: Regression
1. Example: Price of a used car
2. Simple Linear Regression:
• One dependent variable (y)
• one independent variable (x)
Supervised Learning: Regression
1. Example: Price of a used car (Y)
2. Dependent on:
• Horsepower (X1)
• Car weight (X2)
3. Multiple Linear Regression:
• One/more dependent
variable (y)
• More than one independent
variable (x1, x2, … , xn)
4. Complexity of learning
increased?
5. There are other kinds of
regression as well
Supervised Learning: Representations
Supervised Learning: Representations
Supervised Learning: Representations
Supervised Learning: Terminology Summary
1. Features
2. Feature vector
3. Example (x, y): Instance x with label y = f(x)
4. Instance space
5. Target function
6. Hypothesis space
7. Training data
Thank You