WEEK : 2

Introduction:
Machine learning projects follow a pipeline of repeatable steps, from
understanding the problem to deploying the model. As one ML guide puts it, “a
machine learning pipeline is a set of repeatable, linked, and often automated
steps…to engineer, train, and deploy ML models”.
In practice we typically: (1) define the problem and success metric, (2) gather the
data, (3) explore and visualize the data, (4) prepare/clean the data, (5) choose and
train a model, (6) evaluate its performance, (7) fine-tune hyperparameters, and
finally (8) present/deploy the solution and monitor it in production. For example,
we might predict wine quality from its features.
Data Gathering:
First we load the data. In many cases data comes from files, databases, or external
sources. For illustration, we’ll use the UCI Wine Quality dataset (red wine) which
is provided as a CSV file online. We can download it directly using pandas. For
example:
import pandas as pd
# URL of the dataset (CSV with “;” delimiter)
url = "[Link]
[Link]"
# Read into a DataFrame; note sep=';' because this file uses semicolons
df = pd.read_csv(url, sep=';')
print([Link]())

This uses pandas.read_csv, which reads a CSV file from a local path or URL into a
table-like DataFrame.
The head() call shows the first 5 rows, helping us see the data’s structure. The first
row lists column names (features), e.g. fixed acidity, volatile acidity, …, and the
last column is quality (the wine rating).
It’s good practice to inspect the data further:
• DataFrame info: [Link]() shows number of rows, columns, and data types.
• Summary stats: [Link]() computes count, mean, std, min, max, and
quartiles for each numeric column. For example, it will tell us the range of
“fixed acidity” or the median value of “alcohol”.
• Value counts: For a classification target (or discrete values),
df['quality'].value_counts() shows how many samples of each quality exist.
This histogram of counts reveals if some classes dominate or if data is
imbalanced. (In the wine data, quality 5 and 6 are most common.)
By exploring these basics, we gain familiarity with the data’s size, types, and target
distribution before moving on.
Data Visualization:
After loading the data, visualization helps us spot patterns or problems. We can
plot relationships between features, or a feature vs. the label. Common plots
include scatter plots, histograms, and heatmaps of correlations. For instance:
import [Link] as plt
import seaborn as sns

# Example: Scatter plot of two features, color-coded by quality

[Link](x="fixed acidity", y="density", hue="quality", data=df)
[Link]("Fixed Acidity vs Density (colored by wine quality)")
[Link]()

# Histogram of the target (wine quality)

df['quality'].hist(bins=6)
[Link]("Wine Quality")
[Link]("Number of samples")
[Link]("Distribution of Wine Quality")
[Link]()

# Heatmap of feature correlations

corr_matrix = [Link]()
[Link](figsize=(6,5))
[Link](corr_matrix, annot=True, cmap='coolwarm')
[Link]("Feature Correlation Matrix")
[Link]()

• Scatter plots: These plot two numeric features against each other; coloring
by the label (using hue) highlights how target values vary. They help reveal
trends or clusters. Seaborn’s scatterplot function (as above) makes it easy to
draw them.
• Histograms: Plotting a feature or target’s distribution (e.g.
df['quality'].hist()) shows how values are spread or skewed. For example,
we might see that most wines have quality around 5–6.
• Correlation heatmap: Computing [Link]() gives a matrix of Pearson
correlations (range –1 to +1) between every pair of numeric features. We
can plot this as a heatmap to quickly see which features move together.
Strong correlations might suggest redundant features or linear relationships.
These plots give a visual intuition. For instance, we might observe that higher
alcohol content tends to correspond to higher quality, or notice that some quality
classes have many more samples. Visualizing data is key to understanding the
problem before modeling.
Data Preparation (Cleaning & Feature Engineering):
Real-world data often needs cleaning and transformation before modeling. Key
steps include: separating features/labels, handling missing values, encoding
categorical data, and scaling features.
• Separate features and label: We split the DataFrame into input X and target
y. For example:
X = [Link]("quality", axis=1) # all columns except the label
 y = df["quality"].copy() # the target column

• Missing values: Check for missing entries in each column using

[Link]().sum(). If any feature has missing values, we must decide to drop
those rows or fill them in (impute). In ML projects, data is often limited, so
dropping many rows can hurt performance. Instead, we usually impute.
Scikit-learn offers SimpleImputer to fill missing values. For example, to
replace missing entries by the column median:
from [Link] import SimpleImputer
imputer = SimpleImputer(strategy="median") # can also use "mean",
"most_frequent", etc.​:contentReference[oaicite:6]{index=6}
X_num = imputer.fit_transform(X) # returns a NumPy array
X = [Link](X_num, columns=[Link]) # convert back to
DataFrame (optional)

The fit step computes the median of each feature, and transform fills missing
entries. (You can check imputer.statistics_ to see the values learned.)
• Encoding categorical features: If any features are text or categories (e.g.
“type” of item), we convert them to numeric form. A common method is one-
hot encoding: create a new binary column for each category. Scikit-learn’s
OneHotEncoder can do this. For example, if X had a column "color" with
values “red”, “blue”, “green”, one-hot encoding would create three
columns (color_red, color_blue, color_green) with 0/1 values. (For ordinal
categories with a natural order, simpler integer encoding can work, but in
general one-hot avoids implying order). If there are many categories, one-
hot can create many features, and sometimes other techniques (like
embedding, target encoding, or feature hashing) are used, but those are
advanced.
• Feature scaling: ML algorithms often perform better when all features are
on a similar scale. For example, if one feature ranges 0–1 and another 0–
10,000, the latter can dominate. A standard approach is to standardize
features by removing the mean and scaling to unit variance. Scikit-learn’s
StandardScaler implements this: it transforms each feature x to z = (x –
mean) / std. For example:
from [Link] import StandardScaler
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X) # fit learns mean/std, transform scales

After scaling, most algorithms (like linear regression, SVM, KNN, etc.) work
more reliably.
• Preprocessing pipelines: It’s convenient to chain these steps so they’re
applied consistently. Scikit-learn’s Pipeline (for a sequence of transforms)
and ColumnTransformer (to apply different transforms to different
columns) make this easy. For instance, we might create a numeric pipeline
that imputes then scales:
 from [Link] import Pipeline
from [Link] import StandardScaler
num_pipeline = Pipeline([
('imputer', SimpleImputer(strategy="median")),
('scaler', StandardScaler())
])
X_prepared = num_pipeline.fit_transform(X)

This applies the median imputer then scaler in order. For a dataset with both
numeric and categorical columns, we can use ColumnTransformer to apply
num_pipeline to numeric columns and one-hot encoding to categorical
columns. For example:
from [Link] import ColumnTransformer
from [Link] import OneHotEncoder
num_attribs = ["fixed acidity", "volatile acidity", ..., "alcohol"] # numeric columns
cat_attribs = ["grape_type"] # example category column
full_pipeline = ColumnTransformer([
("num", num_pipeline, num_attribs),
("cat", OneHotEncoder(), cat_attribs),
], remainder='drop')
X_ready = full_pipeline.fit_transform(df)

This produces a single transformed feature matrix combining all processed

features. In summary, data preparation ensures our features are clean,
numeric, and on similar scales so that learning algorithms work effectively.
Model Selection and Training:
With the prepared data, we now choose a model and train it. But first we split the
data into a training set and a test set. This helps us evaluate the model on unseen
data and avoid data leakage or snooping bias. In code:
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
X_ready, y, test_size=0.2, random_state=42, shuffle=True)

By default, train_test_split randomly shuffles and splits the data, keeping 20% for
testing. The random_state ensures reproducibility. If the target label is
imbalanced, we can stratify the split to preserve proportions: train_test_split(...,
stratify=y).
Now select a model. For regression problems (predicting a continuous label like
wine quality), common choices include linear regression, decision trees, or
ensemble methods like random forests. For classification (discrete categories), we
might use logistic regression, SVM, or random forests. Here, assume wine quality
(0–10 integer) as a regression target. As a first try, we could use Random Forest
Regressor. Example code:
from [Link] import RandomForestRegressor
from [Link] import mean_squared_error
model = RandomForestRegressor(random_state=42)
[Link](X_train, y_train) # train the model
preds = [Link](X_test) # make predictions on test set
mse = mean_squared_error(y_test, preds) # compute test MSE
print("Test MSE:", mse)

The mean_squared_error above computes average squared error on the test set.
(The lower the MSE, the better the predictions.) We should also look at other
metrics. For regression we often use Mean Squared Error (MSE) or Mean Absolute
Error (MAE); for classification we use accuracy, precision, recall, F1-score, etc.
(Scikit-learn provides functions like r2_score, accuracy_score, precision_score in
the metrics module.) It’s important to report performance only on the test set (or
via cross-validation) to avoid optimistic bias.
If the initial model’s performance is poor or the data is complex, we might try other
algorithms (e.g. linear regression, gradient boosting, SVM) and compare their
validation errors. But we always keep the final test set aside until the very end.
Model Evaluation:
Evaluating the model involves looking at the chosen metrics. For example, a low
MSE or high R² indicates good fit. We should also inspect for overfitting: if
training error is much lower than test error, the model may have learned noise.
Plotting predictions vs. true values or looking at residuals can also help.
Classification vs. Regression: If instead our task were classification, we would use
metrics like accuracy (fraction correct), precision, recall, and F1-score. We might
also use a confusion matrix to see how classes are misclassified. Always pick a
metric that makes sense for the problem (e.g. in imbalanced classification,
accuracy may be misleading, so F1 or AUC-ROC might be better).
In any case, we should verify that the performance on the test set is acceptable for
the application. It’s also good practice to use cross-validation during training (e.g.
cross_val_score in scikit-learn) to get a more reliable estimate of performance.
Model Fine-Tuning (Hyperparameter Tuning):
Most learning algorithms have hyperparameters that control model complexity
(e.g. number of trees, learning rate, maximum depth). Choosing good
hyperparameters can greatly improve performance. Two common approaches are
Grid Search and Randomized Search, both built into scikit-learn.
• Grid Search (GridSearchCV): we specify a grid of hyperparameter values
and exhaustively try all combinations with cross-validation. Scikit-learn’s
GridSearchCV automates this. For example, to tune a random forest:
from sklearn.model_selection import GridSearchCV
param_grid = {
'n_estimators': [50, 100, 200],
'max_features': [6, 8, 10]
}
 grid_search = GridSearchCV(
RandomForestRegressor(random_state=42),
param_grid, cv=5,
scoring='neg_mean_squared_error'
)
grid_search.fit(X_train, y_train)
print("Best params:", grid_search.best_params_)

GridSearchCV performs an “exhaustive search over specified parameter

values for an estimator”, using cross-validation folds. The best_params_
attribute tells us which combination gave the lowest error (highest score). We
can also inspect grid_search.best_estimator_ to get the model with tuned
hyperparameters.
• Randomized Search (RandomizedSearchCV): When there are many
hyperparameters, GridSearch can be slow. RandomizedSearchCV samples a
fixed number of random combinations from parameter distributions. It’s
more efficient when the search space is large. For example:
from sklearn.model_selection import RandomizedSearchCV
from [Link] import randint
param_dist = {
'n_estimators': randint(50, 200),
'max_features': randint(1, X_train.shape[1]+1)
}
rand_search = RandomizedSearchCV(
RandomForestRegressor(random_state=42),
param_distributions=param_dist,
n_iter=10, cv=5,
scoring='neg_mean_squared_error',
random_state=42
)
rand_search.fit(X_train, y_train)
print("Best params (random search):", rand_search.best_params_)

This tries 10 random combinations from the specified ranges. Both grid and
random search use cross-validation internally, so they give more robust
estimates of which parameters work best on unseen data.
After tuning, we re-evaluate the final model on the test set to see the real-world
performance. Often, hyperparameter tuning can significantly improve test
accuracy or reduce error.
Presenting, Deployment, and Monitoring:
Once we have a well-trained, tuned model, we “present” our solution by
explaining the results (e.g. showing performance metrics, selected features, or
sample predictions) to stakeholders. In practice, the final model is often saved (for
example, using Python’s pickle) so it can be loaded later for predictions. For
deployment, one might expose the model via an API or a web service.
Finally, monitoring is crucial. In production, the model will see new incoming
data. We should continuously check that its performance stays acceptable. For
example, monitor the error metric on recent data and watch for “drift” in input
distributions. If performance drops (e.g. new data is very different), we may need
to retrain the model with fresh data. In short, ML systems require ongoing
maintenance: retraining periodically and handling any system issues or outages.
Proper monitoring and a plan to update the model as needed ensure the solution
continues to add value.
Introduction to Scikit-Learn:
Scikit-learn (often imported as sklearn) is an open-source Python library that
provides a consistent interface to a wide range of supervised and unsupervised
learning algorithms. It is built on top of SciPy and works seamlessly with NumPy
arrays and pandas DataFrames. In scikit-learn, every ML estimator follows a
standard pattern:
• Transformers vs. Estimators vs. Predictors: Many scikit-learn classes are
transformers (for preprocessing) or estimators (for modeling). Transformers
(e.g. StandardScaler, PCA) implement fit() and transform() methods to
prepare or modify data, while estimators (e.g. LogisticRegression,
KNeighborsClassifier) implement fit(X, y) to learn from data. Predictors (a
type of estimator) also implement predict(X) to make predictions and
score(X, y) to evaluate performance. This uniform API means that any
estimator can be trained with fit and used with predict (or transform)
interchangeably.
• Pipelines: Scikit-learn provides the Pipeline class to chain several steps
(transformers followed by a final estimator) into one object. For example,
one can create a pipeline of scaling followed by logistic regression. This is
useful for creating concise workflows and for tuning parameters across
multiple steps (with GridSearchCV).
• Model Selection & Evaluation: The sklearn.model_selection module
contains tools like train_test_split, cross-validation (cross_val_score), and
GridSearchCV or RandomizedSearchCV for hyperparameter tuning. These
allow you to systematically evaluate and compare models.
Some common scikit-learn modules include:
• [Link] (feature scaling, normalization, encoding)
• sklearn.feature_extraction (converting text or images into numeric features)
• [Link] (handling missing values)
• sklearn.linear_model, sklearn.naive_bayes, etc. (families of ML algorithms)
• [Link] (functions like accuracy, mean squared error, confusion
matrix)
• sklearn.model_selection (cross-validation, parameter search)
For example, using the uniform estimator API might look like:
from sklearn.linear_model import LogisticRegression
model = LogisticRegression()
[Link](X_train, y_train) # train the model
predictions = [Link](X_test) # make predictions

Each estimator implements the same .fit() and .predict() pattern, making it easy to
swap algorithms without changing the surrounding code. This consistency is a
hallmark of scikit-learn’s design.
Data Loading:
Machine learning workflows begin with loading and preparing data. Data often
comes in common formats such as CSV, Excel, JSON, or SQL databases. In
Python, the Pandas library provides convenient functions for each:
• CSV files: Use pd.read_csv(filepath) to read comma-separated values into a
DataFrame. The filepath can be a local path or even a URL (Pandas
supports HTTP/FTP URLs).
• Excel files: Use pd.read_excel(filepath, sheet_name=…) to read Excel
spreadsheets.
• JSON files: Use pd.read_json(filepath) for JSON data.
• SQL databases: Use pd.read_sql(query, connection) to execute a SQL query
and load the result.
Scikit-Learn Dataset API:
Scikit-learn provides built-in utilities to access common datasets. These fall into
three categories:
• Toy datasets (load_*): Small standard datasets included in the library (no
download needed). Examples are the Iris classification dataset and the
Diabetes regression dataset. They can be loaded with functions like
load_iris() or load_diabetes(). Calling iris = load_iris() returns a “Bunch”
object (similar to a dictionary) containing the data and metadata. For
instance:
from [Link] import load_iris
iris = load_iris()
print(iris.feature_names) # e.g. ['sepal length (cm)', 'sepal width (cm)', ...]
print(iris.target_names) # e.g. ['setosa', 'versicolor', 'virginica']
X, y = [Link], [Link] # feature matrix and label vector

The .data field is an (n_samples × n_features) array, and .target holds the
labels. These toy datasets are handy for learning and testing, although they
are often small. (As documentation notes, scikit-learn’s toy datasets “can be
loaded using” functions like load_iris and load_diabetes.) Optionally, you
can call load_iris(return_X_y=True) to get (X, y) directly without the Bunch
object.
• Fetching larger datasets: Scikit-learn also has fetchers for larger, real-
world datasets (often via the OpenML repository). For example,
fetch_openml(name='dataset_name') can download datasets by name or
 ID. It also includes specific fetchers like fetch_california_housing() (U.S.
housing prices) or fetch_20newsgroups() (text data). These functions
download the data to disk on first use and return a Bunch similar to load_*.
(In coursework, one often sees the fetch_openml function mentioned for
datasets like MNIST or Titanic.)
• Synthetic dataset generators (make_*): For experimenting with models,
sklearn can generate random data with specified properties. Functions like
make_regression(), make_classification(),
make_multilabel_classification(), and make_blobs() create feature matrices
and label vectors that follow certain patterns. For example:
from [Link] import make_blobs
X, y = make_blobs(n_samples=100, n_features=2, centers=3)

The code above generates 100 samples (X) in 2 features, grouped into 3
clusters (y labels 0/1/2). In fact, scikit-learn’s dataset generators “make it
easy to create custom datasets”
Together, these scikit-learn dataset tools make it easy to access and create data for
learning and testing. Toy datasets and synthetic generators come right with the
library (no downloads needed), while fetchers can pull larger datasets from the
internet. Once the data are loaded into (X, y) arrays or DataFrames, you can
proceed to fit scikit-learn models as before.