L T P C
IT23903 FUNDAMENTALS OF MACHINE LEARNING
3 0 0 3
COURSE OBJECTIVES:
• To understand the basic concepts of machine learning and probability theory.
• To appreciate supervised learning and their applications.
• To understand unsupervised learning like clustering and EM algorithms
• To understand the theoretical and practical aspects of probabilistic graphical
models
• To learn other learning aspects such as reinforcement learning,
representation learning, deep learning, neural networks and other
technologies.
UNIT I INTRODUCTION 9
Basic Concepts in Machine Learning – Types of Machine Learning – Supervised,
Unsupervised, Semi-supervised and Reinforcement Learning - Applications of Machine
Learning - Basics of Learning Theory – Concept Learning – Challenges of Machine Learning
– Feature Engineering - Linear Regression – Single and Multiple Variable Regression –
Polynomial Regression – Bias and variance - Logistic regression
Suggested Activities:
• Implement Find-S algorithm and Candidate Elimination Algorithm.
• Tutorial on Model selection and Validation
• External Learning - Overfitting and Underfitting
• Practical - Installing Python and exploring the packages required for machine
learning
Suggested Evaluation Methods:
• Quiz on machine learning concepts and data.
• Seminar on Version spaces.
• Quiz of Python tools available for implementing machine learning
applications.
UNIT II SUPERVISED LEARNING - I 9
Linear Regression – Multiple variable regression – Logistic regression – Regularization
techniques - LASSO, Ridge, and Elastic Net Regression - Decision Tree Learning- ID3 -
C4.5 – CART - Instance based Learning - K-Nearest Neighbor Algorithm - Neural
Networks – Perceptron - Feed-Forward Networks for binary and multi-class classification
- Multi Layer Perceptron - Back Propagation.
Suggested Activities:
• External Learning - Regularization
• Practical - Develop an application that makes predictions from data using Linear
Regression, Logistic Regression.
• Practical – Implement ID3 algorithm.
• Practical – Implement a Perceptron and Multi-Layer Perceptron model
Suggested Evaluation Methods:
• Quiz on Regression models
• Group discussion on basics of classification and regression.
• Evaluation of the practical implementations of neural network models using
the appropriate test dataset
SUPERVISED LEARNING II AND UNSUPERVISED
UNIT III 9
LEARNING
Basics of Neural Networks – Biological and Artificial Neurons - Perceptron – Perceptron
Rule - Feedforward networks – backpropagation Algorithms – Classification using Neural
networks – Challenges in ANN - Support Vector Machine – Optimal Hyperplane – hard
and Soft margin SVM – Non-Linear SVM – Kernels – Support Vector Regression
Suggested Activities:
• Practical – Develop an SVM model for a two-class problem, whose training points
are distributed in a 2D plane and improve the performance of the model by applying
kernel methods.
• Practical – Implement a bagging and boosting approach for some case studies.
• Implement K- means algorithm for a data set.
Suggested Evaluation Methods:
• Quiz on SVM and Kernel methods.
• Group discussion on Ensemble methods.
• Quiz on Clustering Methods, Dimensionality reduction
UNIT IV PROBABILISTIC GRAPHICAL MODELS 9
Probability-based learning – Classification using Bayes Model - Naive Bayes Algorithm –
– Gibbs Algorithm - Bayes Classifier for continuous variables - Probabilistic Graphic
models – Bayesian Belief Network – Construction of Bayesian Network – Bayesian
Inference - Markov Chain – Markov Models - Hidden Markov Models – Applications of
HMM
Suggested Activities:
• Assignment on solving numerical problems using HMM.
• Practical - Classification using Naive Bayes algorithm.
• Group Discussion on Markov Random Fields (MRF) and Conditional Random
Fields (CRF)
Suggested Evaluation Methods:
• Group discussion on Graphical models.
• Seminar on Parameterization of MRFs.
• Quiz on CRF and MRF
UNIT V ADVANCED LEARNING 9
Introduction to Clustering - Hierarchical Clustering – Single Linkage – Complete Linkage
– Average Linkage – Partitional Clustering Algorithms – K-means - Expectation
Maximization Algorithm – Linear Discriminant Analysis – Principal Component Analysis -
Gaussian Mixture Models – Latest Trends – Overview and Scope of Reinforcement
Learning – Components of reinforcement Learning – Model-based and Model-free
models – Q-Learning Algorithm
Suggested Activities:
• Assignment on SARSA Learning
• Practical - Implement CNN, LSTM
Suggested Evaluation Methods:
• Quiz on Reinforcement Learning
• Group Discussion on Deep Neural Networks.
• Evaluation of the practical implementation of CNN, LSTM
TOTAL: 45 = 45 PERIODS
COURSE OUTCOMES:
Upon successful completion of the course, the student will be able to:
Disseminate the key elements of machine learning and the basics of
CO 1.
concept learning.
Apply regression analysis, decision tree models and neural networks for
CO 2.
regression and classification problems.
CO 3. Implement SVM, ensembling methods for an appropriate application
CO 4. Apply clustering methods for learning with unsupervised data.
Design and implement a BBN, HMM for a sequence model type of
CO 5. application and implement a PGM for any real time application using an
open-source tool.
Describe Reinforcement learning and use a tool to implement Deep
CO6
learning algorithms.
TEXTBOOKS:
1. Christopher Bishop, “Pattern Recognition and Machine Learning”, First Edition,
Springer, 2006.
2. Tom Mitchell, “Machine Learning”, McGraw-Hill, 1997.
3. Sridhar S, Vijayalakshmi M, “Machine Learning”, First Edition, Oxford University
Press, 2022.
REFERENCES:
1. Kevin P. Murphy, “Machine Learning: A Probabilistic Perspective”, MIT Press, 2012.
2. EthemAlpaydin, “Introduction to Machine Learning”, Third Edition, Prentice Hall of
India, 2005.
3. T. Hastie, R. Tibshirani, J. Friedman, “The Elements of Statistical Learning”, Second
Edition, Springer, 2008.
4. Stephen Marsland, “Machine Learning – An Algorithmic Perspective”, CRC Press,
2009.
5. T. V. Geetha, S. Sendhilkumar, “Machine Learning: Concepts, Techniques and
Applications” Chapman & Hall/CRC Press, 2023.