Advanced Machine Learning – Week 1: Introduction & Core ML Review
Lecture Objectives
By the end of this lecture, students should be able to:
Understand the scope and structure of the course
Revisit key concepts in supervised and unsupervised machine learning
Understand the bias-variance tradeoff and its implications
Be familiar with the types of datasets and evaluation metrics used in ML
Set up their ML development environment for labs and assignments
Lecture Outline
Part 1: Course Introduction
Overview of course objectives
Grading policy, assignments, and project timeline
Recommended tools and platforms
Expectations and academic integrity
Part 2: Review of Machine Learning Fundamentals
Types of Machine Learning
o Supervised Learning
Regression vs Classification
o Unsupervised Learning
Clustering, Dimensionality Reduction
o Semi-supervised & Reinforcement Learning (brief intro)
Core Concepts
o Model, hypothesis space, loss function
o Training vs testing vs validation
o Overfitting and underfitting
Bias-Variance Tradeoff
o Definitions
o Visualization and practical intuition
o Generalization error decomposition
Common Algorithms Recap
o Linear regression
o Logistic regression
o Decision trees
o K-NN and Naive Bayes (brief overview)
Part 3: Evaluation Metrics & Model Selection (30 minutes)
Classification metrics: Accuracy, Precision, Recall, F1-Score, ROC-AUC
Regression metrics: MSE, RMSE, MAE, R²
Cross-validation techniques: k-fold, stratified k-fold, LOOCV
Lab Component (60 minutes)
Lab Title: ML Review and Mini Project Setup
Objective: To ensure students are familiar with Scikit-learn and basic model training pipelines.
Activities:
Load a dataset (Iris or Boston Housing)
Preprocess the data (normalization, train-test split)
Train and evaluate a logistic regression and decision tree model
Plot ROC curves and confusion matrix
Tools:
Python, Jupyter Notebook
Libraries: scikit-learn, matplotlib, seaborn, pandas, numpy
Assignment 1 (To be submitted in Week 2)
Title: Regularized Linear Regression on Real-world Data
Task:
Download a real-world regression dataset (e.g., California Housing)
Implement and compare:
o Linear Regression
o Ridge Regression
o Lasso Regression
Evaluate using RMSE, MAE
Plot learning curves and comment on overfitting/underfitting
Deliverables:
Code
Short report (1–2 pages PDF)
Datasets
Recommended for Week 1 & 2:
Iris Dataset (Classification)
California Housing Dataset (Regression)
Discussion Topics
Why is generalization critical in ML?
What makes a model "simple" vs "complex"?
How does the dataset size impact the bias-variance tradeoff?
Discuss real-world examples where ML has failed due to overfitting or poor evaluation
Key Terms
Generalization
Bias-variance tradeoff
Overfitting / Underfitting
Loss function
Cross-validation
ROC-AUC / Confusion Matrix
Regularization