Course Name: Machine Learning Using Python
Home Assignment 1 - Answers
CO1
Question 1: Explain the role of probability and Bayesian statistics in machine learning. Provide a brief overview of
key concepts from these areas that are foundational to machine learning algorithms.
Answer:
1 1. Probability theory provides a mathematical foundation for modeling uncertainty and randomness in data.
Machine learning models often rely on probabilistic assumptions to make predictions and estimate
confidence levels.
2 2. Bayesian statistics extend probability by allowing beliefs (priors) to be updated as new evidence (data)
becomes available, forming the posterior probability. This makes Bayesian methods ideal for iterative
learning and dynamic environments.
3 3. Key concepts include: (a) Random variables — represent uncertain quantities; (b) Probability
distributions — describe the likelihood of possible outcomes; (c) Conditional probability — models
dependency between variables; (d) Bayes’ theorem — mathematically updates prior beliefs with new
evidence; (e) Likelihood and posterior — quantify the model’s fit and updated beliefs.
4 4. Applications include Bayesian networks for decision making, Naïve Bayes classifiers for text
classification, and probabilistic graphical models for structured data representation.
Question 2: Outline the typical framework for developing machine learning models. Describe the key stages
involved, from problem definition to deployment and maintenance.
Answer:
1 1. Problem Definition — Clearly define the business or research objective and the type of prediction or
classification task required.
2 2. Data Collection — Gather sufficient, high-quality, and relevant data from available sources such as
databases, APIs, or sensors.
3 3. Data Preprocessing — Clean, handle missing values, normalize or standardize features, and encode
categorical variables to make data model-ready.
4 4. Exploratory Data Analysis (EDA) — Visualize patterns, distributions, and correlations to identify key
variables and detect anomalies or biases.
5 5. Feature Engineering — Create, transform, or select the most informative features that improve model
performance and interpretability.
6 6. Model Selection — Choose an appropriate algorithm based on problem type (e.g., regression,
classification, clustering) and dataset characteristics.
7 7. Model Training — Fit the model to training data, optimize hyperparameters, and use cross-validation to
avoid overfitting.
8 8. Model Evaluation — Assess using performance metrics such as accuracy, precision, recall, F1-score,
RMSE, or AUC depending on problem type.
9 9. Deployment — Integrate the trained model into production systems for real-time or batch predictions,
ensuring scalability and efficiency.
10 10. Maintenance — Continuously monitor model performance, retrain with new data, and address model
drift to maintain accuracy and reliability.
CO2
Question 1: Explain the goal of feature reduction. Discuss two common feature reduction techniques (e.g.,
Principal Component Analysis (PCA), feature selection methods) and their underlying principles.
Answer:
1 1. Goal of feature reduction — To simplify models by reducing the number of input variables while retaining
as much relevant information as possible. This improves computational efficiency, reduces overfitting, and
enhances interpretability.
2 2. Principal Component Analysis (PCA): A linear transformation technique that converts correlated
variables into uncorrelated components (principal components) ordered by explained variance. The first
few components often capture most of the dataset’s variability, allowing dimensionality reduction without
significant information loss.
3 3. Feature Selection Methods: Techniques like filter, wrapper, and embedded methods select the most
relevant features. Filter methods use statistical metrics (e.g., correlation, chi-square); wrapper methods
evaluate subsets through model performance; embedded methods (like LASSO) perform selection during
model training by penalizing less important coefficients.
Question 2: Describe the process of feature construction. Provide two examples of how new features can be
derived from existing ones to potentially improve model performance.
Answer:
1 1. Feature construction is the process of creating new, informative variables from existing data to reveal
hidden patterns and improve model accuracy.
2 2. Steps involve domain understanding, exploring relationships between variables, and applying
transformations such as arithmetic combinations, ratios, or aggregations.
3 3. Example 1 — Interaction Features: Creating new features that capture interaction effects between two
variables (e.g., Age × Income to capture spending capacity).
4 4. Example 2 — Aggregated Features: Summarizing multiple observations into meaningful metrics (e.g.,
average transaction value, total purchase frequency) to represent customer behavior over time.
5 5. Effective feature construction can significantly enhance predictive power and often leads to better
performance than simply switching algorithms.