Prof.

Dhananjai Rao

Machine learning Notes 22MBABA403

Module 1: Introduction to Machine Learning (6 Hours)

Key Points
Creativity and Motivation in Machine Learning
Computer Hardware Architecture for ML
Basic Programming for Machine Learning
Machine Learning Terminology
Debugging and Troubleshooting ML Models

Creativity and Motivation in Machine Learning: Machine learning requires innovative

approaches to solving complex problems like image recognition, speech analysis, etc. The
motivation stems from automating decision-making and predicting outcomes more accurately
using algorithms.
Computer Hardware Architecture for ML: ML algorithms need computational resources
for processing data. Understanding the architecture (CPU, GPU, RAM) helps in choosing the
right hardware for training models. Deep learning models, for example, require powerful
GPUs for faster training.
Basic Programming for Machine Learning: Python is the most common language used for
ML due to its readability and libraries (NumPy, pandas, scikit-learn). Key programming
concepts for ML include:
Data structures like arrays, lists, and dictionaries.
Functions for modularizing code.
Loops to iterate over datasets.
Libraries like matplotlib for plotting and pandas for data manipulation.
Machine Learning Terminology:
Training Data: The dataset used to train the algorithm.
Testing Data: The dataset used to evaluate the trained model’s accuracy.
Model: The mathematical representation of the learned algorithm.
Algorithm: The set of rules or steps used to learn from data.
Debugging and Troubleshooting ML Models: Debugging in ML involves identifying errors
in code, but also diagnosing issues such as underfitting or overfitting in models. Debugging
also means tuning hyperparameters like learning rate, regularization, and more to enhance
model performance.
 Prof. Dhananjai Rao

Likely Questions
3 Marks:
1. What is machine learning? Explain its motivation.
2. What are the common programming languages used in ML?
3. Define training data and testing data.

7 Marks:
1. Discuss the importance of creativity in machine learning with examples.
2. Explain the role of hardware architecture in training machine learning models.
3. Define the key terminology used in machine learning and explain with examples.
10 Marks:
1. Explain the basic programming concepts necessary for machine learning. Include
Python libraries for data processing.
2. Describe the common challenges faced in debugging machine learning models and
how to address them.

Module 2: Supervised and Unsupervised Learning (6 Hours)

Supervised Learning: Regression and Classification Models
Decision Tree Models
Neural Networks and Support Vector Machines (SVM)
Unsupervised Learning: Clustering and K-means
Supervised Learning: Regression and Classification Models: In supervised learning, the
model is trained on labeled data.
Regression: Used to predict continuous values (e.g., house prices).
Classification: Used for categorizing data into distinct classes (e.g., spam vs. non-spam
emails).
Decision Tree Models: A tree-like structure used for both regression and classification tasks.
Each node represents a decision based on a feature, and the leaves represent the class labels
or predicted values.
Information Gain: The metric used to decide the best feature to split the data at each node.
Neural Networks and SVM: Neural networks (especially MLP) consist of multiple layers of
neurons that learn complex patterns. Support Vector Machines are used for classification
tasks, using kernel functions to map data to higher-dimensional spaces for better separation.
 Prof. Dhananjai Rao

Unsupervised Learning: Clustering: Unsupervised learning does not use labeled data.
Clustering algorithms like K-means classify data into groups (clusters) based on similarity.

Likely Questions

3 Marks:
1. What is supervised learning? Name its types.
2. Explain what decision trees are and how they are used in machine learning.
7 Marks:
1. Explain the difference between regression and classification in supervised learning
with examples.
2. How does a decision tree algorithm work? Explain the concept of information gain.
10 Marks:
1. Describe the working of neural networks. Explain how support vector machines are
used for classification.
2. What is unsupervised learning? Explain clustering with the K-means algorithm.

Module 3: Decision Trees and Genetic Algorithms (7 Hours)

Basic Decision Tree Algorithm
Pruning and Information Gain
Genetic Algorithms and Their Working
Selection and Crossover in Genetic Algorithms
Basic Decision Tree Algorithm: A decision tree recursively splits the dataset into subsets
based on feature values, aiming to improve the purity of each subset. The leaf nodes represent
the final prediction.
Pruning and Information Gain: After constructing a decision tree, pruning removes
branches that do not add much value in improving the model's accuracy. Information gain
measures the reduction in entropy when a feature is used for splitting.
Genetic Algorithms: A search heuristic inspired by the process of natural evolution. It works
by evolving a population of solutions using operations like:
• Selection: Choosing the fittest individuals.
• Crossover: Combining solutions to form offspring.
• Mutation: Randomly altering a solution to introduce variability.
 Prof. Dhananjai Rao

Likely Questions
3 Marks:
1. What is pruning in decision trees? Why is it important?
2. Define genetic algorithms and give an example of their application.
7 Marks:
1. Explain the decision tree algorithm. How is information gain used to construct
decision trees?
2. Describe how genetic algorithms work, focusing on the selection and crossover
process.
10 Marks:
1. Discuss decision tree learning in detail. Include information gain and pruning
techniques.
2. Explain genetic algorithms, including steps like selection, crossover, and mutation.

Module 4: Ensemble and Probabilistic Learning (7 Hours)

Ensemble Methods: Bagging, Boosting, Stacking
Random Forest Algorithm
Gaussian Mixture Models
Nearest Neighbour Methods (KNN)
Ensemble Methods: These methods combine predictions from multiple models to improve
accuracy:
• Bagging (Random Forest): Involves training multiple models on different subsets of
the data and averaging their predictions.
• Boosting (AdaBoost): Sequentially builds models where each new model corrects the
errors of the previous ones.
• Stacking: Combining the predictions from several models using a meta-model.

Gaussian Mixture Models (GMM): A probabilistic model that assumes data is generated
from a mixture of several Gaussian distributions. The Expectation-Maximization (EM)
algorithm is used to estimate the parameters of the GMM.
Nearest Neighbour Methods (KNN): A classification algorithm where the class of a sample
is determined by the majority class of its nearest neighbors.

Likely Questions
3 Marks:
1. What is ensemble learning? Name its types.
2. Define nearest neighbor methods.
 Prof. Dhananjai Rao

7 Marks:
1. Explain the concept of bagging and boosting in ensemble learning with examples.
2. Describe how Gaussian Mixture Models work and the role of the EM algorithm.

10 Marks:
1. Discuss ensemble methods (bagging, boosting, stacking) and their applications.
2. Explain nearest neighbor methods and how they are used for classification tasks.

Module 5: Reinforcement Learning and Evaluating Hypotheses (7 Hours)

Reinforcement Learning: Q-Learning and Temporal Difference Learning
Exploration vs. Exploitation in RL
Evaluating Models and Estimating Errors
Q-Learning: A model-free reinforcement learning algorithm where an agent learns the value
of actions in a given state. The goal is to maximize cumulative reward.
Temporal Difference Learning: A learning method that combines the benefits of Monte
Carlo methods and dynamic programming. It updates predictions based on the difference
between expected and actual rewards.
Exploration vs. Exploitation: Balancing between trying new actions (exploration) and
exploiting known actions that yield higher rewards (exploitation).
Evaluating Models and Estimating Errors: Techniques like cross-validation and bias-
variance analysis help assess how well a model generalizes to unseen data.

Likely Questions
3 Marks:
1. Define Q-learning in reinforcement learning.
2. What is the exploration-exploitation trade-off?
7 Marks:
1. Explain temporal difference learning in reinforcement learning.
2. How do you evaluate a machine learning model? Explain with error estimation
techniques.

10 Marks:
1. Describe reinforcement learning and its components, focusing on Q-learning and
temporal difference learning.
2. Discuss various methods for evaluating machine learning models and estimating
errors.
 Prof. Dhananjai Rao

Module 6: Introduction to Virtual Reality (VR) and Augmented Reality (AR) (7 Hours)
Virtual Reality (VR) Basics
Augmented Reality (AR) Technology and Features
AR vs. VR
Challenges in AR Implementation
Virtual Reality (VR): VR creates an immersive, simulated environment. It is used in
applications such as flight simulations and gaming. VR requires high computational power
and real-time rendering.
Augmented Reality (AR): AR overlays digital content on the real world, enhancing the
user's experience. It is used in fields like education, marketing, and healthcare.
AR vs. VR: The key difference is that VR replaces reality, whereas AR enhances it by adding
digital elements.
Challenges in AR: Some challenges include device limitations (e.g., battery life, processing
power), alignment of digital elements with the physical world, and ensuring a seamless user
experience.

Likely Questions
3 Marks:
1. Define VR and AR. Explain the difference between them.
2. What are the challenges faced in AR implementation?

7 Marks:
1. Explain virtual reality and its applications in real-world scenarios.
2. Describe the technology behind augmented reality and its features.

10 Marks:
1. Compare and contrast AR and VR. Discuss their technologies and applications in
various industries.
2. Discuss the challenges faced while implementing AR systems.
 Prof. Dhananjai Rao

Guide on how to answer

Module 1: Introduction to Machine Learning

3 Marks Questions
1. What is machine learning? Explain its motivation.
Machine learning is a subset of artificial intelligence that allows systems to learn from data
and improve over time without explicit programming. The motivation for ML is to automate
decision-making, recognize patterns in data, and make predictions, which is useful in tasks
like fraud detection, recommendations, and autonomous vehicles.
2. What are the common programming languages used in ML?
The primary language, Python, because of its extensive libraries like pandas, scikit-learn, and
TensorFlow, making it the go-to choice for ML. Other languages like R and Julia are also
used, with R being good for statistical analysis and Julia known for speed.
3. Define training data and testing data.
• Training Data: This is the dataset used to train the machine learning model. It
includes both inputs (features) and outputs (labels).
• Testing Data: This dataset is used to evaluate the performance of the model after
training, ensuring that the model can generalize well to new, unseen data.
7 Marks Questions
1. Discuss the importance of creativity in machine learning with examples.
Creativity is essential in ML because solving problems often requires unique ways to
interpret and process data. For example, in image recognition, creative approaches are needed
to enhance images or deal with data augmentation techniques like rotating images to improve
model robustness. In recommendation systems, creative algorithms might be used to predict
user preferences.
2. Explain the role of hardware architecture in training machine learning models.
Machine learning, especially deep learning, requires significant computational power. GPUs
are essential for training large neural networks efficiently, while CPUs handle simpler
computations. RAM ensures that data can be processed efficiently. When training on large
datasets, powerful hardware like GPUs can accelerate model training by handling parallel
computations.
 Prof. Dhananjai Rao

3. Define the key terminology used in machine learning and explain with examples.
Define terms such as:
• Model: The learned representation (e.g., a decision tree or a neural network).
• Algorithm: The method used to train the model (e.g., gradient descent for neural
networks).
• Overfitting: When the model learns too much from the training data, including noise,
and performs poorly on new data.
• Training and Testing Data: Explained earlier.

10 Marks Questions
1. Explain the basic programming concepts necessary for machine learning.
Include Python libraries for data processing.
Focus on concepts like:
• Data Structures: Lists, arrays, and data frames (using libraries like pandas).
• Functions and Loops: To process and iterate over data.
• Libraries: Mention key libraries:
• pandas for data manipulation.
• NumPy for numerical operations.
• matplotlib for visualization.
• scikit-learn for ML algorithms.
• Code Snippet (just for context, not necessary to memorize code):
• import pandas as pd to load and manipulate datasets.
• import [Link] as plt for plotting data.

2. Describe the common challenges faced in debugging machine learning models

and how to address them.
Discuss challenges like:
• Overfitting: The model performs well on training data but fails on test data. Solution:
Use regularization, more data, or simpler models.
• Underfitting: The model is too simple to capture patterns in the data. Solution: Use
more complex models or improve features.
• Data Preprocessing Issues: Missing or noisy data. Solution: Handle missing values,
remove outliers, or normalize data.
 Prof. Dhananjai Rao

Module 2: Supervised and Unsupervised Learning

3 Marks Questions
1. What is supervised learning? Name its types.
Supervised learning is a type of machine learning where the model is trained on labeled data.
The main types are:
• Regression: Predicts continuous values (e.g., predicting house prices).
• Classification: Predicts discrete classes (e.g., spam detection).

2. Explain what decision trees are and how they are used in machine learning.
A decision tree is a flowchart-like structure where each internal node represents a decision
based on an attribute, and leaf nodes represent class labels or predicted values. It is used in
both regression and classification tasks to model data hierarchically and easily interpret
results.

7 Marks Questions
1. Explain the difference between regression and classification in supervised
learning with examples.
• Regression: The goal is to predict a continuous output (e.g., predicting a student's
exam score based on hours studied).
• Classification: The goal is to predict a categorical output (e.g., classifying emails as
spam or not).
• Provide real-world examples for clarity.

2. How does a decision tree algorithm work? Explain the concept of information
gain.
• Working: The decision tree splits the dataset based on features that maximize
information gain.
• Information Gain: A measure used to decide which feature to split on, calculated
based on the reduction in entropy (uncertainty) when a feature is selected.
Example: If splitting by age in a dataset of customer purchases reduces uncertainty about
whether someone will buy a product, it’s a good choice.
10 Marks Questions
1. Describe the working of neural networks. Explain how support vector machines
are used for classification.
• Neural Networks: Composed of layers of neurons, each connected with weighted
links. Each neuron applies an activation function to decide whether it should pass
information to the next layer.
• Support Vector Machines (SVM): SVMs are supervised models used for
classification tasks. They find the optimal hyperplane that maximizes the margin
 Prof. Dhananjai Rao

between classes. Kernel functions help SVMs handle non-linear problems by

transforming data into higher dimensions.
2. What is unsupervised learning? Explain clustering with the K-means algorithm.
• Unsupervised Learning: In unsupervised learning, the model learns patterns from
data without labeled outputs.
• K-means Clustering: A technique where data points are grouped into k clusters. The
algorithm iterates to minimize the variance within each cluster. It starts by randomly
selecting cluster centers and assigns data points to the nearest cluster

Module 3: Decision Trees and Genetic Algorithms

Genetic Algorithms
3 Marks Questions
1. Define genetic algorithms and give an example of their application.
Approach: Genetic algorithms (GAs) are optimization techniques inspired by natural
evolution. They iteratively evolve a population of solutions to find the best possible solution.
Example: They are used in problems like optimization of neural network configurations
or tuning hyperparameters for ML models. They are also used for scheduling problems,
like job scheduling in manufacturing.
7 Marks Questions
1. Describe how genetic algorithms work, focusing on the selection and crossover
process.
• Selection: The process of choosing the fittest individuals based on a fitness function.
The fittest solutions are more likely to be selected for reproduction.
• Crossover: It combines two parent solutions to create offspring, mimicking the
genetic process of recombination. The crossover operator combines parts of two
solutions, typically by swapping sections of the genetic strings.
10 Marks Questions
1. Explain genetic algorithms, including steps like selection, crossover, and
mutation.
• Selection: Evaluate solutions based on how well they perform (fitness function) and
select the best-performing solutions for reproduction.
• Crossover: The chosen individuals (parents) are combined to produce offspring that
may carry characteristics from both parents. This can be done using one-point, two-
point, or uniform crossover methods.
• Mutation: Introduces randomness to the offspring by randomly altering parts of the
genetic code. This prevents the algorithm from getting stuck in local optima by
maintaining diversity in the population.
 Prof. Dhananjai Rao

Module 4: Ensemble and Probabilistic Learning

Ensemble Methods
3 Marks Questions
1. What is ensemble learning? Name its types.
Ensemble learning combines multiple models to produce a better predictive model. The key
types of ensemble methods are:
• Bagging: Models are trained independently and combined (e.g., Random Forest).
• Boosting: Models are trained sequentially, with each new model trying to correct the
errors of the previous model (e.g., AdaBoost).
• Stacking: Multiple models are trained, and their predictions are combined using
another model (meta-model).
•
7 Marks Questions
1. Explain the concept of bagging and boosting in ensemble learning with examples.
• Bagging (Bootstrap Aggregating): Involves training multiple models independently
on different subsets of data (bootstrapped datasets). The predictions are then averaged
for regression or voted on for classification. Random Forest is an example of a
bagging algorithm.
• Boosting: A sequential ensemble method where models are trained one after another.
Each model tries to correct the mistakes made by the previous one. AdaBoost is an
example of boosting, where weak learners (models slightly better than random) are
combined to form a strong model.
10 Marks Questions
1. Discuss ensemble methods (bagging, boosting, stacking) and their applications.
• Bagging: Useful for reducing variance and overfitting. Example: Random Forest,
which can be used for classification, regression, and feature selection tasks.
• Boosting: Useful for reducing bias by focusing on difficult-to-predict instances.
Example: AdaBoost can be used for tasks like spam detection and classification.
• Stacking: Combines the predictions of multiple different models (e.g., decision trees,
neural networks). It is powerful when combining heterogeneous models.

Probabilistic Learning
3 Marks Questions
1. What is a Gaussian mixture model?
A Gaussian Mixture Model (GMM) is a probabilistic model that assumes data is generated
from a mixture of several Gaussian distributions. Each data point is considered a mixture of
probabilities drawn from these Gaussian distributions.
 Prof. Dhananjai Rao

7 Marks Questions
1. Explain how Gaussian Mixture Models work and the role of the Expectation-
Maximization (EM) algorithm.
• Gaussian Mixture Models (GMMs): GMMs use a mixture of Gaussian distributions
to model data with multiple underlying sub-populations. The mixture components
(Gaussians) represent clusters in the data.
• Expectation-Maximization (EM): EM is an iterative method for estimating the
parameters of the GMM. In the E-step, it assigns each data point a probability of
belonging to each Gaussian component. In the M-step, it updates the parameters of
the Gaussians (mean, variance) to maximize the likelihood of the observed data.

10 Marks Questions
1. Describe probabilistic learning and the role of Gaussian mixture models.
• Probabilistic Learning: A form of learning where models predict the probability of
an event or class. GMMs use this approach by modeling the probability distribution of
data as a mixture of several Gaussian distributions.
• GMMs: Explain the working of GMMs in more detail, including how the EM
algorithm helps fit GMMs to data. Mention applications like image segmentation,
speech recognition, and anomaly detection.

Module 5: Reinforcement Learning

3 Marks Questions
1. What is Q-learning?
Q-learning is a model-free reinforcement learning algorithm used for finding the optimal
action-selection policy for an agent. It uses a Q-table to store the value of action-state pairs
and updates these values using a Bellman equation based on rewards received.
7 Marks Questions
1. Explain temporal difference learning and its relationship to dynamic
programming.
• Temporal Difference (TD) Learning: TD learning combines the benefits of
dynamic programming and Monte Carlo methods. TD learning updates its
estimates based on both current estimates and future estimates, reducing the need for a
complete model of the environment.
• Relationship to Dynamic Programming: Like dynamic programming, TD learning
involves updates based on previous state-action pairs, but it does not require a model
of the environment.
10 Marks Questions
1. Describe reinforcement learning and its components, focusing on Q-learning and
temporal difference learning.
 Prof. Dhananjai Rao

• Reinforcement Learning (RL): RL involves an agent interacting with an

environment to maximize cumulative rewards. The agent learns an optimal policy by
exploring and exploiting actions based on feedback (rewards).
• Q-learning: Define Q-learning and explain how it helps the agent learn the best
actions in different states.
• Temporal Difference (TD) Learning: Discuss TD learning as a way for the agent to
learn from incomplete information without needing to know the entire future ahead of
time.

Module 6
3 Marks Questions
1. Define Virtual Reality (VR) and Augmented Reality (AR). Explain the difference
between them.
Define both VR and AR. VR creates an immersive, entirely simulated environment, while AR
overlays digital content on the real world. Highlight the key differences such as immersion
and hardware requirements.
2. What are the challenges in implementing AR systems?
Discuss challenges like hardware limitations (battery life, processing power), accuracy and
alignment (difficulty in accurately overlaying content), and user interface design.

3. What is the difference between marker-based and markerless AR?

Define both methods. Marker-based AR relies on physical markers (QR codes), while
markerless AR uses sensors like GPS and accelerometers to detect real-world context and
overlay digital content.

7 Marks Questions
1. Explain the working of Virtual Reality and its applications in real-world
scenarios.
Define VR, and explain how it simulates environments using specialized hardware. Mention
applications like flight simulators, medical training, and gaming, where VR creates safe,
immersive experiences for practice or entertainment.

2. Describe AR technology and its features with real-world examples.

Define AR and explain how it overlays digital content onto the real world using devices like
smartphones or smart glasses. Include examples such as Pokémon GO and Google Lens, and
mention the technologies involved (e.g., GPS, cameras).
 Prof. Dhananjai Rao

10 Marks Questions
1. Compare and contrast AR and VR. Discuss their technologies, applications, and
limitations.
Provide a detailed comparison of AR and VR, explaining their differences in terms of
immersion, hardware, and usage. Discuss their applications in areas like education,
marketing, and training, and explain the limitations like hardware constraints and usability
issues.
2. Discuss the challenges in implementing AR systems. How can these challenges be
overcome?
Detail challenges such as hardware limitations, accuracy, alignment issues, and user
experience design. Then, discuss possible solutions, like improving hardware capabilities,
using better sensors, and focusing on intuitive UI/UX design for AR apps.

Previous question paper Answer Guide.

1(a) Machine Learning – Definition & Examples
Machine Learning (ML) is a branch of Artificial Intelligence (AI) that enables computers to
learn and make predictions or decisions from data without explicit programming.
Examples:
Spam Detection: Classifies emails as spam or not spam.
House Price Prediction: Predicts property prices based on features like size and location.

1(b) Computer Hardware Architecture for ML

Main Components:
1. CPU (Central Processing Unit): Controls all operations and executes instructions.
2. GPU (Graphics Processing Unit): Accelerates training of ML models (esp. deep
learning).
3. Memory (RAM): Temporarily stores data for quick access during computation.
4. Storage (SSD/HDD): Stores datasets, trained models, and logs.
5. Networking Units: Enable data transfer and distributed computing.
6. Input/Output Devices: For data entry and visualization.

1(c) Programming Languages for ML

 Prof. Dhananjai Rao

Most Popular: Python

• Why: Easy syntax, extensive libraries, strong community.
• Libraries: TensorFlow, PyTorch, Scikit-learn, Keras, NumPy, Pandas,
Matplotlib.
• Uses: Data preprocessing, model training, deployment.

2(a) Supervised vs Unsupervised Learning

Supervised Learning Unsupervised Learning

Learns from labeled data (input-output Learns from unlabeled data.
pairs).
Predict outcomes/classify data. Find hidden patterns or structure
Eg..Classification, Regression Eg.. Clustering, PCA, Dimensionality Reduction
Algorithms: Linear/Logistic Algorithms:
Regression, Decision Trees K-means, Hierarchical Clustering, DBSCAN

2(b) Regression Models

I. Linear Regression
Models linear relationship between dependent and independent variables.
In is use for House price prediction, stock analysis, sales forecasting.
II. Decision Tree Regression
Splits data into subsets to minimize variance.
Applications: Energy use prediction, medical cost estimation, crop yield prediction.

2(c) Classification Models Comparison

Logistic Regression Random Forest
Linear Model Ensemble of Decision Trees
Interpretability is High Interpretability is Low
Handles Non-linearity Poorly Handles Non-linearity Very Well
Overfitting Less (with regularization) Overfitting Possible (if trees are deep)
Faster to use Slower (training)
Sensitive to outliers Robust due to averaging
Used for Spam detection, churn prediction Used for Fraud detection, medical diagnosis
 Prof. Dhananjai Rao

3(a) Decision Tree

A supervised algorithm for classification/regression using a tree-like model.
Structure:
• Nodes: Feature-based decisions
• Branches: Outcomes of decisions
• Leaves: Final class/value
Example: Predicting purchase decision based on age and income.
3(b) Entropy
• Definition: Measure of impurity or uncertainty in a dataset.
𝐸𝑛𝑡𝑟𝑜𝑝𝑦 = −∑𝑝𝑖 log⁡2 (𝑝𝑖 )

• Example: from ppt module 3

3(c) ID3 Algorithm

Steps:
1. Calculate entropy for the dataset.
2. Compute Information Gain (IG) for each feature.
3. Choose the feature with highest IG for the root node.
4. Repeat for sub-nodes until all samples are classified or max depth is reached.
Example: Predicting product purchase using age and income.

4(a) Ensemble Models

• Combine multiple base models to improve accuracy and reduce overfitting.
• Example: Random Forest → averages predictions from many decision trees.

4(b) Bagging vs Boosting

Bagging: Trains multiple models in parallel, the data is Random subsets with replacement the
goal is to Reduce variance example: Random Forest.
Boosting: Trains models sequentially it Focuses on errors of previous model,it Reduces bias.
Example: AdaBoost, XGBoost
 Prof. Dhananjai Rao

4(c) Stacking
Combines predictions from multiple base models using a meta-model.
Example:
Base models: Decision Tree, SVM, Linear Regression
Meta-model: Logistic Regression
Predicting house prices using outputs of multiple models.

5(a) Reinforcement Learning (RL)

• Concept: Agent learns by interacting with the environment to maximize rewards.
• Examples:
➢ Robotics (navigation, manipulation)
➢ Game AI (e.g., AlphaGo)
➢ Self-driving cars

5(b) Q-Learning
Model-free RL algorithm
Learn optimal policy using Q-values
Core Equation:
Q(s,a)=R+γmaxQ(s′,a′)
Where :State (S), Action (A), Reward (R), Learning Rate (α), Discount Factor (γ)

Example: A robot learning shortest paths to a goal.

5(c) RL vs Dynamic Programming vs Active RL

• Reinforcement Learning Learns from interaction,it gets data from Trial and error,it
Maximize cumulative reward
• Dynamic Programming Requires full model,it gets Full knowledge as data. It
Computes optimal policy
• Active RL Learns with selective sampling, Chooses most informative samples,it
Balances exploration and exploitation
 Prof. Dhananjai Rao

6(a) Definition & Example

Immersive technology simulating a 3D environment using headsets/controllers.
Example: Oculus Rift or Meta Quest for 360° gaming.
6(b) Flight Simulation vs Traditional Training
Advantages of Simulation:
• Safer, cost-efficient, repeatable, allows complex scenarios.

Traditional Training:
• Provides real-world experience and hands-on control.

Other Uses: Military training, air traffic control, aircraft testing, R&D, public safety.

6(c) Virtual Environment Requirements

Hardware: CPU, GPU, RAM, SSD, VR headsets, motion sensors
Software: Unity, Unreal Engine, Blender, Oculus SDK
Network: High bandwidth, low latency
Environment: Physical movement space, lighting conditions
User Experience: Comfort, 3D audio
Performance: 90+ FPS, high resolution, optimized rendering

7(a) Definition & Example

Technology overlaying digital content onto the real world.
• Example: Microsoft HoloLens used by surgeons to visualize organs.

7(b) Motivation and Sampling in RL

• Motivation: Encourage exploration for learning optimal policies.
• Sampling Theory: Select experiences (samples) from interactions for efficient
learning
 Prof. Dhananjai Rao

7(c) Visualization Techniques in AR

• Techniques:
1. Object overlay
2. 3D modeling
3. Motion tracking
4. Spatial mapping
5. Gesture-based interaction
• Applications: Gaming, education, retail, healthcare.

Case study

The Fresh Mart case study demonstrates how different machine learning and advanced
technologies can be applied to solve business challenges effectively.
To predict customer purchase behavior, the Decision Tree algorithm is used. This algorithm
analyzes customer data such as age, shopping frequency, and product preferences to identify
patterns and determine the likelihood of purchasing baked products like bread, cakes, or
pastries.

To optimize promotional strategies, Reinforcement Learning is applied. In this approach, the

system experiments with different promotional offers and observes the resulting sales
performance. Based on the feedback,whether sales increase or decrease,the algorithm learns
which promotions are most effective and retains only those that lead to higher sales, ensuring
continuous improvement in marketing efforts.
Finally, to enhance the customer shopping experience, Augmented Reality (AR) technology
is utilized. Through AR tools integrated into the company’s app, customers can virtually
visualize bakery products,such as how a cake or pastry would appear on their dining table or
serving area,before making a purchase. This interactive and immersive experience not only
increases engagement but also helps customers make confident buying decisions, ultimately
improving satisfaction and boosting sales.

………………………………………………………………………………………………….