Introduction to Machine Learning

1. What is Machine Learning and how is it different from traditional programming?

 Machine Learning (ML) is a technology where computers learn patterns from data and make decisions
without being directly programmed.
 In traditional programming, you give the computer clear rules, and based on those rules, it gives an
output. For example, if you program a calculator, you tell it exactly how to add or subtract. But in
Machine Learning, you give the computer a lot of data, and it automatically learns the rules from that
data.
 For example, instead of writing rules to detect spam emails, an ML model studies many emails and
learns what spam looks like. ML is useful when rules are too many or too complex to write manually.
 It focuses on pattern recognition and predictions, while traditional programming focuses on fixed
instructions.

2. What does “learning from data” mean in Machine Learning?

 “Learning from data” means a computer studies past information and builds its own understanding to
make decisions for new situations.
 In Machine Learning, you give the computer many examples.
o From these examples, it finds patterns and relationships.
o For example, if you want the model to predict house prices, you provide data like house size,
location, and past prices.
 The model learns how these factors relate. When it sees a new house, it uses what it learned to predict
the price. Learning happens through algorithms that adjust internal settings to reduce errors.
 The more data the model sees, the better it becomes at identifying patterns. This process is similar to
how humans learn from experience.

3. What are the main types of Machine Learning, and how do they differ?

 Machine Learning mainly has three types: Supervised Learning, Unsupervised Learning, and
Reinforcement Learning. In Supervised Learning, the model is trained using labelled data, meaning
the input and the correct output are already known.
 It is used for prediction tasks. In Unsupervised Learning, the data has no labels, and the model tries
to find patterns or group the data on its own. It is used for discovering hidden structures.
 In Reinforcement Learning, the model learns by interacting with an environment and receiving
rewards or penalties based on actions. It focuses on learning the best actions over time.
 The key difference between the three is the type of data and the way learning happens.

4. What is Supervised Learning and what are some of its applications?

 Supervised Learning is a type of Machine Learning where the model learns from labelled data. This
means the input and the correct output are already known.
 The model studies examples and learns how to predict the output for new data. There are two main
types: Regression (predicting numbers) and Classification (predicting categories).
 Some common applications include email spam detection, predicting house prices, diagnosing
diseases, and loan approval systems. In spam detection, the model learns from examples of spam and
non-spam emails.
 In medical diagnosis, models learn from patient data and help identify diseases. Supervised learning is
widely used because it gives accurate and reliable results when labelled data is available.

5. What is Unsupervised Learning and how is it used in real-world problems?

 Unsupervised Learning is a Machine Learning method where the data has no labels. The model tries to
find patterns, groups, or structures within the data on its own.
 One main task is clustering, where similar items are grouped together. Another task is association,
where relationships between different items are found. Unsupervised learning is used in many real-
world areas.
 For example, companies use it to group customers based on buying behaviour so they can plan
marketing strategies. It is also used in identifying unusual transactions in banking, which helps in fraud
detection.
 In social media, it helps group similar posts or users. It is useful when you have data but do not know
the correct output in advance.

6. What is Reinforcement Learning and what makes it unique compared to other types?

 Reinforcement Learning (RL) is a type of Machine Learning where a model learns by interacting with an
environment. Instead of learning from labelled data, the model learns through rewards and
penalties.
  When it takes a good action, it earns reward points; when the action is wrong, it loses points. Over
time, it learns the best actions to achieve long-term success. RL is unique because learning happens
through trial and error, just like humans learn to ride a bicycle.
 It is used in robotics, self-driving cars, game-playing (like chess), and automated control systems.
Unlike supervised learning, RL does not need labelled data. Unlike unsupervised learning, RL focuses
on learning actions rather than finding patterns.

7. What is Regression in Machine Learning and when is it used?

 Regression is a type of supervised learning used to predict numerical values. It finds the relationship
between input factors and a continuous output.
 For example, predicting house prices based on size, location, and age is a regression problem. It is also
used for forecasting sales, predicting temperatures, estimating demand, and analysing trends.
 Regression models try to fit a line or curve to the data that best explains the relationship. It is useful
when the result is a number and you want to understand how different factors affect that number.
 Regression helps make informed business decisions and future planning.

8. What is Classification and how does it differ from Regression?

 Classification is a supervised learning method used to predict categories or classes. The output is not a
number but a label, such as “spam or not spam,” “disease or no disease,” or “approved or rejected.”
 It differs from regression because regression predicts continuous numbers, while classification predicts
fixed categories. For example, predicting someone’s age is regression, but predicting whether they are
a minor or adult is classification.
 Classification models learn from labelled examples and then decide which class new data belongs to. It
is used in medical diagnosis, email filtering, security systems, and image recognition.
 Both regression and classification use supervised learning but solve different types of problems.

9. What is Clustering and what are some practical applications of clustering algorithms?

 Clustering is an unsupervised learning technique where data is grouped into clusters based on
similarity. Items in the same cluster are alike, while items in different clusters are different.
 Clustering is used in many real applications. In marketing, companies use it to group customers based
on shopping habits. In healthcare, clustering helps group patients with similar symptoms.
 In banking, it helps detect unusual spending patterns. In social media, clustering groups similar images
or posts. It is also used in organizing large datasets and reducing complexity.
 Clustering is powerful when you have unlabelled data and want to discover meaningful groups.

10. What are some real-world applications of Machine Learning across different industries?

 Machine Learning is used in almost every industry today. In healthcare, it helps in disease prediction,
medical imaging, and personalised treatment plans.
 In finance, ML is used for fraud detection, stock market prediction, and credit scoring. In retail, it helps
with product recommendations, customer segmentation, and demand forecasting.
 In transportation, it powers self-driving cars, route optimisation, and traffic prediction. In
manufacturing, ML is used for quality control, predictive maintenance, and automation.
 In entertainment, platforms like Netflix and YouTube use ML to recommend videos. In agriculture, ML
helps analyse soil, predict crop yield, and detect plant diseases. ML is becoming essential for faster
decisions, automation, and improving accuracy across industries.

Unit: 2 Linear Regression

11. What is Linear Regression and why is it important in Machine Learning?

 Linear Regression is a supervised learning method.

 It predicts a numerical value based on input data.

 It fits a straight line between input and output values.

 The line shows the relationship between variables.

 Important because it is simple and easy to interpret.

 Used for forecasting, trend analysis, and decision-making.

 Helps understand how one variable affects another.

12. How is a scatter diagram useful in understanding the relationship between variables?

 Scatter diagram shows data points on a graph.

 Shows whether relationship is positive, negative, or none.

 Helps decide if Linear Regression is suitable.

 Good for spotting outliers or unusual points.

13. What is the cost function in Linear Regression and why is it needed?

 Cost function measures error between actual and predicted values.

 Common cost: Mean Squared Error (MSE).

 Lower cost = better model.

 Needed to check how well the regression line fits data.

 Used to guide optimization methods like gradient descent.

14. How does the Least Squares Method work in fitting a regression line?

 Calculates the best-fit line by minimizing errors.

 Error = difference between actual and predicted values.

 Squares all errors to avoid negative values.

 Chooses the line with minimum total squared error.

 Simple and widely used for Linear Regression.

15. What are Normal Equations and how are they applied in Linear Regression?

 Normal Equations give a direct formula to compute model parameters.

 No need for iterative steps like gradient descent.

 Solve equation: (XᵀX)θ = Xᵀy

 Used when dataset is small or medium-sized.

 Gives exact solution for linear regression weights.

16. How does the Gradient Descent method optimize the regression model?

 Gradient Descent reduces the cost function step by step.

 Starts with random values for model parameters.

 Moves in the direction where cost decreases.

 Each step updates parameters to reduce error.

 Continues until minimum error is reached.

17. What are the key assumptions of Linear Regression?

 Linear relationship exists between variables.

 Errors are normally distributed.

 Constant variance of errors (no heteroscedasticity).

 No strong multicollinearity among inputs.

 Errors are independent of each other.

 No extreme outliers.

18. What is the R² score and what does it indicate about model performance?

 R² is a measure of how well the model explains the data.

 Higher R² = better model fit.

 R² = 1 means perfect prediction.

 R² = 0 means model predicts nothing useful.

 Helps compare different regression models.

19. What are overfitting and underfitting in regression models, and how can they be identified?

 Overfitting: Model learns noise instead of pattern.

o High accuracy on training data, low on test data.

 Underfitting: Model is too simple.

o Low accuracy on both training and test data.

 Identified using training vs testing performance.

20. What are Regularization techniques in Linear Regression and why are they used?

 Regularization reduces model complexity.

 Adds penalty to large coefficients.

 Helps avoid overfitting.

 Two main types: L1 (Lasso) and L2 (Ridge).

 Used when data has many features or noise.

 Makes model more stable and general.

Unit: 3 Logistic Regression

21. Why is Linear Regression not suitable for classification tasks?

 Linear Regression predicts continuous numerical values, not categories.

 Classification needs outputs like “yes/no” or “0/1.”

 Linear Regression can give values beyond 0 and 1, which are not valid probabilities.

 It does not handle non-linear class boundaries properly.

 It is sensitive to outliers, which affects classification accuracy.

22. What is the Sigmoid function and why is it important in Logistic Regression?

 Sigmoid function converts any number into a value between 0 and 1.

 Formula: 1 / (1 + e⁻ˣ).

 It transforms linear output into probability.

 Helps decide class membership (e.g., above 0.5 = class 1).

 Makes Logistic Regression suitable for binary classification.

23. What is the Logit function and how is it related to Logistic Regression?

 Logit function is the logarithm of odds.

 Formula: log(p / (1 - p)).

 Logistic Regression models this logit value as a linear combination of inputs.

 It converts probability to a linear scale for easier modelling.

 Probability: Likelihood of an event occurring (0 to 1).

 Odds: Ratio of event happening to not happening.

 Logistic Regression uses odds to model relationships.

 Higher odds mean higher chance of the event occurring.

25. What is a decision boundary in Logistic Regression and how is it determined?

 Decision boundary separates two classes.

 It is created using the predicted probability threshold (usually 0.5).

 If predicted probability > 0.5 → class 1.

 If predicted probability < 0.5 → class 0.

 Boundary is determined by model coefficients and input features.

26. What is Maximum Likelihood Estimation (MLE) and how is it used in Logistic Regression?

 MLE finds model parameters that maximize the probability of observing given data.

 It chooses coefficients that make predictions closest to actual labels.

 Used instead of least squares because Logistic Regression deals with probabilities.

 Ensures best possible fit for classification tasks.

27. How does Logistic Regression differ from Linear Regression?

 Linear Regression predicts numbers; Logistic Regression predicts categories.

 Linear regression uses a straight line; Logistic Regression uses the sigmoid curve.

 Linear output is unlimited; logistic output is between 0 and 1.

 Logistic Regression uses MLE, not least squares.

28. What is a Confusion Matrix and how is it used?

 A confusion matrix shows the performance of a classification model.

 Contains: True Positive, True Negative, False Positive, False Negative.

 Helps understand correct and incorrect predictions.

 Used to calculate precision, recall, accuracy, and specificity.

29. Performance metrics used in Logistic Regression

 Accuracy: Correct predictions out of total.

 Precision: Correct positive predictions.

 Recall (Sensitivity): Ability to detect actual positives.

 Specificity: Ability to detect actual negatives.

 F1 Score: Balance of precision and recall.

 AUC-ROC: Overall model performance measure.

30. Advantages and limitations of Logistic Regression

Advantages:

 Simple, fast, and easy to interpret.

 Works well with small and clean datasets.

 Requires fewer computations.

Limitations:

 Works only for linear decision boundaries.

 Performs poorly with complex or non-linear data.

 Sensitive to outliers.

 Less accurate than advanced models like SVM or Random Forest.

Unit 4: Supervised Learning Techniques

31. What are classification problems and what role do decision boundaries play?

 Classification problems involve predicting categories (e.g., yes/no, spam/not spam).

 They use labelled data in supervised learning.

 The model assigns each input to one of the predefined classes.

 A decision boundary separates different classes in the feature space.

 Points on one side belong to class 1, and on the other side to class 0.

 Good decision boundaries improve classification accuracy.

32. How does the K-Nearest Neighbors (K-NN) algorithm work and what are its key parameters?

 K-NN classifies a point based on its nearest neighbours.

 It checks the “K” closest data points using a distance measure.

 The majority class among neighbours becomes the prediction.

 Key parameters:

o K value (number of neighbours).

o Distance metric (Euclidean, Manhattan).

o Weighting (all neighbours equal or closer ones weighted more).

33. What are linear classifiers and how are they different from non-linear classifiers?

 Linear classifiers use a straight-line decision boundary.

 They work well when data is linearly separable.

 Examples: Logistic Regression, Linear SVM.

 Non-linear classifiers capture curved or complex boundaries.

 Examples: Kernel SVM, Decision Trees, K-NN.

 Non-linear models handle complex relationships better.

34. What is Bayes’ Rule and how is it applied in Naive Bayes?

 Bayes’ Rule calculates the probability of an event based on prior knowledge.

 Formula: P(A|B) = [P(B|A) × P(A)] / P(B).

 Naive Bayes applies Bayes’ Rule with the assumption that features are independent.

 It calculates the probability for each class and selects the class with the highest probability.

35. What are the advantages and limitations of Naive Bayes?

Advantages:
  Very fast and simple to implement.

 Works well with high-dimensional data.

 Performs well even with small training datasets.

Limitations:

 Assumes all features are independent (not true in real data).

 Performs poorly when feature correlation is high.

 Less accurate than complex models for difficult datasets.

36. What is a Support Vector Machine (SVM) and what are support vectors?

 SVM is a powerful classification algorithm.

 It finds the best separating boundary (hyperplane) between classes.

 Support vectors are the data points closest to the boundary.

 They determine the position and direction of the hyperplane.

 SVM aims to maximize the margin between support vectors.

37. Difference between Hard Margin SVM and Soft Margin SVM

 Hard Margin SVM:

o No misclassification allowed.

o Only works if data is perfectly separable.

o Very sensitive to noise and outliers.

 Soft Margin SVM:

o Allows some misclassification.

o Uses penalty term to balance accuracy and margin size.

o Works better in real-world noisy data.

38. Role of kernel methods in SVM and why they are important

 Kernel methods transform data into a higher-dimensional space.

 Help SVM separate data that is not linearly separable.

 Common kernels: Polynomial, RBF (Gaussian), Sigmoid.

 Kernels allow complex decision boundaries without heavy computation.

 Make SVM suitable for non-linear relationships.

39. What are ensemble methods and how do Bagging, Random Forest, and Boosting differ?

 Ensemble methods combine multiple models to improve accuracy.

Bagging:

 Builds several independent models on different subsets of data.

 Reduces variance. Example: Random Forest.

Random Forest:

 Extension of bagging using many decision trees.

 Uses random feature selection and averaging.

 Builds models sequentially.

 Each new model focuses on correcting previous errors.

 Improves accuracy but may overfit.

40. Advantages and limitations of Decision Trees

Advantages:

 Easy to understand and interpret.

 Requires little data preprocessing.

 Handles numerical and categorical data.

 Works well for both classification and regression.

Limitations:

 Prone to overfitting.

 Small changes in data can change the tree structure.

 Not always accurate compared to advanced models.

Unit 5: Unsupervised Learning

41. What is cluster analysis and how is it applied in real scenarios?

 Cluster analysis groups similar data points into clusters.

 It finds natural patterns without using labels.

 Used for customer segmentation in marketing.

 Helps detect fraud or unusual behaviour in banking.

 Used in image processing to group similar pixels.

 Helps hospitals group patients by symptoms or conditions.

42. How does classification differ from clustering?

 Classification uses labelled data and predicts predefined classes.

 Clustering uses unlabelled data and discovers groups on its own.

 Classification is supervised; clustering is unsupervised.

 Classification predicts; clustering explores patterns.

43. Main types of clusters

 Well-separated clusters: Clear distance between groups.

 Overlapping clusters: Groups mix or overlap.

 Density-based clusters: Formed around high-density regions.

 Hierarchical clusters: Based on tree-like structure.

 Prototype-based clusters: Represented by a central point (e.g., K-Means).

44. What are distance and proximity measures used for in clustering?

 Used to find how similar or different two data points are.

 Help assign points to clusters based on closeness.

 Common measures include Euclidean, Manhattan, and Cosine similarity.

45. Distance measures for different variable types

 Continuous variables: Use Euclidean or Manhattan distance.

 Discrete/categorical variables: Use Hamming distance or simple matching.

 Mixed variables: Use Gower distance to handle both types together.

46. What is the K-Means algorithm and what criteria does it use?

 K-Means divides data into K clusters.

 Each cluster has a centroid (center point).

 Points are assigned to the nearest centroid.

 Criteria used: minimize within-cluster sum of squares (WCSS).

 Iteratively updates centroids until stable.

47. What is fuzzy clustering and how does it differ from K-Means?

 Fuzzy clustering allows one point to belong to multiple clusters with different probabilities.

 K-Means assigns each point to exactly one cluster.

 Fuzzy clustering handles overlapping data better.

 Useful when boundaries are not clear.

48. What is hierarchical clustering? Differences between agglomerative and divisive

 Hierarchical clustering builds a tree of clusters (dendrogram).

Agglomerative:

 Bottom-up approach.

 Each point starts as its own cluster and merges step-by-step.

Divisive:

 Top-down approach.

 Starts with one big cluster and splits it gradually.

49. What are cluster validation techniques and why are they necessary?

 Techniques used to check how good and meaningful the clusters are.

 Ensure clusters are not random or incorrect.

 Include silhouette score, Davies–Bouldin index, Dunn index.

 Necessary to confirm reliability and accuracy of clustering.

50. Internal and external criteria for evaluating clustering

 Internal criteria:

o Evaluate clusters using internal data only.

o Metrics: Silhouette score, cohesion, separation.

 External criteria:

o Compare clusters with known true labels (if available).

o Metrics: Rand Index, Adjusted Rand Index, Mutual Information.