Introduction to Machine
Learning
Session 1: Foundations & Workflow
Prof. Soufiane Hourri
What You'll Learn Today
AI Fundamentals ML Workflow
What is AI and where do we encounter it every day? The complete process from data to predictions
Train/Test Split Common Pitfalls
Creating fair tests for your models Data leakage, overfitting, and how to avoid them
Choosing Metrics Hands-On Practice
Picking the right measure of success Load → split → train → evaluate in action
No heavy math or proofs today — we focus on intuition, visuals, and practical code you can use immediately
Course Outcomes
By the end of this course, you'll confidently be able to:
01 02 03
Identify ML Opportunities Prepare Data Build & Compare Models
Recognize everyday problems where Clean datasets and transform them Train, test, and evaluate models using
machine learning can provide value into meaningful features best practices
04 05
Communicate Results Deploy Solutions
Explain your findings with clear, accessible metrics Create demo applications and document your models
professionally
What is Artificial Intelligence?
Simple definition: Getting computers to perform tasks that typically require human intelligence
AI Powers Your Daily Life
• Face unlock & voice assistants on your phone
• Email spam filters & predictive text
• Personalized recommendations on Netflix, Spotify, YouTube
• Navigation apps predicting traffic patterns
Machine Learning is the "learning" part of AI — systems that improve from data rather than explicit
programming
AI, ML, and Deep Learning
Understanding the relationship
Machine Learning
Learning patterns from data without
explicit programming — our focus
Artificial Intelligence
The big goal — creating systems
with smart, human-like behavior
Deep Learning
ML using large neural networks —
we'll explore this later in the course
Key insight: All deep learning is ML, and all ML is part of AI — they're nested concepts, not separate fields
Essential ML Vocabulary
Terms you'll use throughout this course
Feature
An input column or attribute the model uses to make predictions
Label/Target
The output we're trying to predict — our goal
Train/Test
Train teaches the model patterns; test evaluates performance on new data
Overfitting
When a model memorizes training data too well and fails on new examples
Pipeline
Your complete recipe: clean data → encode features → train model
AI's Strengths & Limitations
What AI Excels At Current Limitations
• Pattern recognition at scale — images, text, audio • Data-dependent — generalizes poorly without sufficient training data
• Prediction & ranking — forecasting, recommendations • Bias inheritance — reflects biases present in training data
• Content generation — text, images with learned styles • Reasoning gaps — lacks true common sense understanding
• Automation of repetitive tasks requiring pattern matching • Out-of-distribution challenges — struggles with truly novel situations
This course focuses on data-driven ML methods and how to evaluate and deploy them responsibly
Machine Learning Defined
Learning from data to make predictions
Working definition: Algorithms that discover patterns in data to make predictions or decisions — without being explicitly programmed for
every rule
Inputs Learning Algorithm Outputs
Raw data & features Pattern discovery Predictions & decisions
Real-World ML Applications
• Email spam filtering & threat detection • Medical image classification & diagnosis
• Movie, music, and product recommendations • Speech recognition & language translation
• Credit scoring & fraud detection systems • Autonomous vehicle navigation
When to Use ML (and When Not To)
✓ Use ML When ✗ Avoid ML When
Patterns are too complex for Rules are simple and stable
manual rules
Logic can be clearly defined and
Thousands of interacting factors doesn't change often
make rule-writing impractical
No reliable data or targets
Historical data exists with available
clear targets
Insufficient training examples or
You have labeled examples to unclear objectives
learn from
Decisions require verifiable
Some error tolerance is logic
acceptable
High-stakes situations demanding
Predictions don't need to be explainable reasoning
perfect every time
Types of ML Problems
Supervised Learning
Learning from labeled examples — most common in practice
Regression
Predicting numbers: house prices, temperature, sales revenue
Classification
Predicting categories: spam/not spam, disease types, customer segments
Unsupervised Learning
Finding patterns without labels — exploratory analysis
Clustering
Grouping similar items: customer segments, document topics
Dimensionality Reduction
Compressing data while preserving patterns: visualization, noise reduction (PCA)
Coming later: Semi-supervised learning, self-supervised learning, and reinforcement learning
The Complete ML Workflow
From question to deployed solution
01 02 03
Define the Question Collect & Inspect Data Split Data Safely
What exactly are you trying to predict or Gather data and verify columns are clean Set aside a fair test set before any analysis
understand? and useful
04 05 06
Prepare Features Start with a Baseline Evaluate Performance
Handle missing values, scale numbers, Build the simplest reasonable model first Test on unseen validation data — does it
encode categories work?
07 08 09
Improve Carefully Final Test Set Check Deploy & Document
Try different approaches systematically, One-time evaluation on completely Save model, create demo, write clear
track results untouched data documentation
Train/Test Split
The foundation of honest evaluation
Training Set (80%)
Where your model learns patterns and relationships in the data
Test Set (20%)
Kept completely hidden until final evaluation — your "honest exam"
Think of it like studying for an exam: if you memorize the exact test
questions, your score doesn't reflect true understanding. Same principle
applies to ML models.
Beginner guideline: An 80/20 train/test split works well for most small to
medium projects
Data Leakage
Critical Mistake to Avoid
Definition: Using information during training that wouldn't be available at prediction time in the real world
Example 1: Improper Missing Value Example 2: Future Information Example 3: Target Variable Proxies
Handling Leaking In
Using columns that are essentially the
Filling missing values using statistics Including features created after the target in disguise
from the entire dataset including test data event you're trying to predict
✓ Check each feature: "Would I know this
✓ Calculate fill values using only the ✓ Only use information that existed before before making a prediction?"
training set prediction time
Prevention Strategy
• Use pipelines that apply all transformations inside the training process
• Keep test data completely untouched until final evaluation
• Document when each feature was created relative to the target
Underfitting vs Overfitting
Finding the right model complexity
Underfitting Just Right ✓ Overfitting
Too simple — model misses important patterns Balanced — captures real signal without Too complex — memorizes training data including
memorizing noise noise
• Like drawing a straight line through curved data
• Poor performance on both training and test • Good training performance • Excellent training performance
data • Similar performance on test data • Much worse test performance
• Model lacks capacity to learn the relationships • Generalizes well to new examples • Fails on new, unseen examples
Fix: Try a more flexible model or add relevant Goal: This is what we're aiming for! Fix: Simplify model, add regularization, or collect
features more data
Recognizing Overfitting in Practice
1 2
Warning Sign: Performance Gap Common Cause: Too Many Features
Large difference between training accuracy (95%) and validation Hundreds of features with only dozens of training examples
accuracy (65%)
3 4
Common Cause: Excessive Model Complexity Common Cause: Validation Set Peeking
Very deep decision trees or too many model parameters Repeatedly tuning based on validation set performance without proper
cross-validation
Solutions to Try
Simplify Regularize More Data
Reduce model complexity or feature count Add penalties for model complexity Collect additional training examples
Choosing the Right Metric
How do you measure success?
Regression Metrics
For predicting numbers
MAE (Mean Absolute Error)Average absolute difference — easy to interpret in original units
RMSE (Root Mean Squared Error)Like MAE but penalizes large mistakes more heavily
Classification Metrics
For predicting categories
AccuracyPercentage correct — but misleading with imbalanced classes
PrecisionOf predicted positives, how many were actually positive?
RecallOf actual positives, how many did we catch?
F1 ScoreHarmonic mean balancing precision and recall
Choose metrics that align with business goals — sometimes catching all positives (recall) matters more than precision, or vice
versa
Always Start with a Baseline
Why baselines matter Simple Baseline Strategies
Reality Check
Regression
A simple baseline tells you if your complex model is
actually adding value Predict the mean of training targets
Early Warning System
Classification
If sophisticated models barely beat baseline, you likely
Always predict the majority class
have data quality issues
Set Expectations Time Series
Provides a realistic bar for improvement and guides Use last known value (persistence)
feature engineering
If your tuned model only improves by 2% over baseline,
rethink your features and problem framing
Reproducibility Matters
Making your work trustworthy and shareable
Fix Random Seeds Track Data Versions
Set seeds for train/test splits and model initialization so Document which dataset version and preprocessing steps
results stay consistent across runs produced each result
Record Experiments Save Environment
Log model parameters, feature choices, and evaluation Keep requirements files (pip freeze) and environment
metrics systematically specifications for exact recreation
Tools for tracking: Jupyter notebooks for exploration, MLflow or Weights & Biases for experiment management (we'll preview these later)
Your Toolkit for This Course
Python 3.x NumPy Pandas Matplotlib
Our primary programming language Numerical computing and array Data manipulation and analysis Creating visualizations and plots
operations
scikit-learn
ML models, pipelines, and evaluation
Later sessions will introduce Keras and PyTorch for neural networks. A starter notebook and environment file are available in the course repository.
Mini-Case Study: House Price Prediction
Problem Setup Our Approach
Goal: Predict sale price of residential properties
Split Data
Input Features:
Train / validation / test sets
• Number of bedrooms and bathrooms
• Square footage / area
• Neighborhood location
Preprocess
• Year built Impute missing, scale numeric, one-hot encode categoricals
• Lot size
Target Variable: Sale price in dollars Baseline Model
Start with linear regression
Compare Models
Try Ridge, Lasso, Random Forest
Final Evaluation
RMSE on validation, then test
Pipelines: Your Recipe for Success
Using scikit-learn pipelines to prevent mistakes
Think of a pipeline as a complete recipe that applies all transformations in the correct order: Key Benefits
Select Columns Prevent Leakage
All preprocessing happens inside training automatically
Choose relevant features
Clean Data One Object
Impute missing values Single pipeline to train, evaluate, and save
Transform Reproducible
Scale numbers, encode categories Exact same steps apply consistently
Model
Train your algorithm
Looking Ahead: Cross-Validation
Today's approach
We'll use a simple train/test split to get started — it's straightforward and
teaches core concepts clearly
Coming in next sessions
Cross-validation provides even more reliable performance estimates by:
• Using multiple train/validation splits
• Averaging results across folds
• Reducing dependence on any single split
• Better detecting overfitting
For now, focus on mastering the single split — cross-validation builds on
these same principles
The Beginner's Rule: Change One Thing at a Time
Systematic experimentation beats random changes
Establish Baseline 1
Record your starting point performance with the simplest reasonable model
2 Make Single Change
Try one modification — different model type, add/remove feature, adjust
parameter
Evaluate Impact 3
Did performance improve, stay same, or get worse? By how much?
4 Document Results
Write down what changed and the outcome — build your intuition over time
Keep or Revert 5
If better, keep the change. If worse, go back to previous best
Advanced techniques like grid search and hyperparameter tuning come later — for now, build intuition through careful, documented experiments
Ethics & Responsible ML
Building models that are fair, transparent, and trustworthy
Dataset Representativeness
Does your training data reflect the real-world population? Missing groups can lead to poor
predictions for those groups
Bias & Fairness
Historical biases in data get learned by models. Actively check performance across demographic
groups
Privacy & Consent
Was data collected ethically? Do people know how their data is being used? Can sensitive
information be inferred?
Transparency of Limitations
Be honest about what your model can and cannot do. Document assumptions, edge cases, and
failure modes
Course requirement: You'll create a simple Model Card documenting intended use, performance, and
limitations for each project
Hands-On Mini-Lab
Build your first ML model with a safe pipeline (35-40 minutes)
What You'll Do
01
Load a small CSV dataset into Pandas (file provided)
02
Split data into train/test sets (80/20)
03
Build a preprocessing pipeline: impute → scale/encode
04
Fit Linear Regression or Logistic Regression
05
Print evaluation metric (MAE or Accuracy/F1)
06 A starter notebook with ready-to-run cells is provided — you'll fill in a few gaps
and observe the complete workflow in action
Interpret: Is the result good? Why or why not?
Stretch challenge: Swap in a Decision Tree and compare performance
Key Takeaways
ML is a Workflow Prevent Leakage Early Metrics Reflect Goals
Not just picking a model — it's the Use pipelines, keep test data Choose evaluation measures that
complete process from data to sacred, think about information align with real-world objectives
deployment timing
Baselines First Reproducibility Counts
Start simple, establish expectations, then iterate with Document everything from day one — seeds, versions,
cross-validation parameters, decisions
Next session: Exploratory Data Analysis (EDA) and in-depth feature preprocessing techniques