Build, Train and
Evaluate ML
Models using
Azure ML
By Mudathir M. Salahudeen
Table of Content
Preparing and Summarizing
Poisson Regression
Data for Machine Learning
Evaluate and Compare Different
Develop Training Pipelines
ML Models
Create, Deploy and Consume a
Linear Regression
Batch Inference Pipeline
Boosted Decision tree
Decision Forest Regression
Preparing and
Summarizing Data:
Cleaning
Missing Data
Cleaning Missing Data
Handling missing data involves identifying and addressing gaps in
datasets to improve model performance and data quality
Methods to Replace Missing Data in Azure ML
Custom Replace Replace Replace Remove Remove
Value with Mean with Median with Mode entire Row Entire
Replace-m Column
ent
Cleaning Missing Data
Drag Missing Data
Component to the canvas
Connect the dataset to the
Component
Configure the Component
Cleaning Missing Data
4. Click the Edit Column to
select Columns to be
Cleaned from the dataset.
Cleaning Missing Data
5. Select the columns to be
cleaned using rules or by
typing the column name,
then save.
Cleaning Missing Data
6. Set Minimum and
Maximum Missing Value
Ratios (between 0 and 1) to
control when operations
apply based on the
proportion of missing data
in each column.
Cleaning Missing Data
7. Choose Cleaning Mode.
8. Configure and Submit.
Preparing and
Summarizing Data:
Split Datasets
Split Data
1. Add the Split Data Component
Drag the Split Data module
into your pipeline.
Connect it to your dataset.
Split Data
2. Choose Splitting Mode
Regular Expression: Split rows
based on a pattern in a
specified column.
Relative Expression: Use a
condition (e.g., column values
greater than a threshold).
Randomized Split: Randomly
distribute rows based on the
specified fraction.
Split Data
3. Set Split Parameters
Fraction of Rows in the First Output
Dataset: Specify the proportion of data
to go into the first output
(e.g., 0.7 for 70%).
Random Seed: Set a seed value for
reproducibility of random splits.
4. Run the Pipeline
Execute the pipeline to create the split
datasets.
Use these splits for training and testing
models
Preparing and
Summarizing Data:
Normalize
Datasets
Normalize Data
Add the Normalize Data Component
Drag the Normalize Data module
into your pipeline.
Connect it to your dataset.
Normalize Data
2. Choose a Normalization Method
Z-Score Normalization: Scales data to have a
mean of 0 and standard deviation of 1.
Min-Max Normalization: Scales data to a
specified range (e.g., 0 to 1).
Log Normalization: Applies a log
transformation to compress large values.
Logistics : Transforms data into a range
between 0 and 1 using a sigmoid function.
Commonly used for binary classification
problems.
Tanh Normalization: Applies a hyperbolic
tangent (tanh) transformation to scale values
between -1 and 1, often used to prepare data for
neural networks.
Normalize Data
3. Select Columns to Normalize
Use the column selector to
choose specific columns for
normalization
Typically applied to numerical
columns
Preparing and
Summarizing Data:
Summarize
Datasets
Summarize Data
1. Add the Summarize Data Component
Drag the Summarize Data
module into your pipeline
Connect it to your dataset.
2. Run the Pipeline
Execute the pipeline to
generate a summary of the
selected columns.
View summary statistics like
mean, median, mode,
standard deviation, and more.
Data Export
Export Data
Steps to Use Data Export
Add the Data Export Component
• Drag the Data Export module into your
pipeline.
• Connect it to the dataset you want to export
Choose Output Format
• Select the desired file format for export:
– CSV
– Parquet
Configure Output Location
• Choose the location to save the exported
data, either in Azure storage or a local path
Run the Pipeline
• Execute the pipeline to export the dataset in the
selected format.
• Review the exported data in the chosen
location.
Building ML
Pipelines
Pipelines in Azure ML
A pipeline is a sequence of data processing and machine learning tasks
that are organized in a workflow, allowing you to automate and streamline
the end-to-end process of building, training, and deploying machine
learning models.
Azure Pipeline Architecture
Problem Statement: A Case study of Millennial Group
Millennial Group, a budding real estate company, is
preparing to launch its operations. As part of their business
strategy, they aim to develop a system capable of
predicting house prices accurately.
To achieve this, they have collected a rich dataset
containing information on house prices from 1950 to 2022.
Recognizing the importance of leveraging this data, the
company has hired a skilled Machine Learning Engineer to
build a predictive model that can analyze trends and
provide valuable insights into future pricing
This model will empower Millennial Group to make
data-driven decisions and stay competitive in the market
As we progress, you’ll step into the role of the Machine
Learning Engineer tasked with building this model using
Azure Machine Learning
Scenario
Objective: Predict house prices based on historical data.
Explore Dataset
Features
Square_
Lot_Size: size
Footage: total
Num_ Num_ of the land on Neighbor-hoo
area of the Year_Built Garage_Size
Bedrooms Bathrooms which the d_Quality
house in
house is built
square feet
Target variable : House_Price
Machine Learning Types
[Link]
Approaching a Machine Learning Problem
For this problem, we will be
implementing
Linear Regression
Boosted Decision tree
Decision Forest Regression
Poisson Regression
We will then evaluate each model
performance and deploy the best
performing model to be consumed by
Millennial Group
Approaching a Machine Learning Problem
To solve this problem and predict house prices for Millennial
Group, We will be using Azure Machine Learning
Steps
Define the Problem Train the Models
Evaluate each Model
Collect and Explore the Dataset
performance
Store the Dataset in a suitable location
Select the best
(Azure Blob Storage, Azure SQL
performing model
Database, e.t.c)
Prepare the Dataset (Check for missing
Deploy the Model
data, incorrect data types, e.t.c)
Split the Data (Train and Test Sets) Monitor the Model
Select different Models
Linear Regression
Store the Dataset in a suitable location
To use data in an Azure Machine Learning pipeline, it
needs to be stored in a suitable location, such as Azure
Blob Storage, Azure SQL Database, or any other Azure
database. After storing the data, a datastore is created
in Azure Machine Learning to link this data storage to
your ML workspace.
Steps to Store Data and Create a Datastore
1. Store Data in Azure:
Upload your dataset to Azure Blob Storage or
Azure SQL Database. You can do this via the
Azure portal - [Link]
For Blob Storage, create a container to hold your
data files (e.g., CSV, Parquet, etc.).
For Azure SQL Database, ensure your data is
properly loaded into a table.
Store the Dataset in a suitable location
Steps to Store Data and Create a Datastore
2. Create a Datastore in Azure ML:
• Go to the Azure ML workspace in the Azure
portal.
• Under Datasets, click on Datastores.
• Click + New datastore and select the type of
datastore (e.g., Blob Storage, SQL Database,
etc.).
• Enter the necessary connection information
(e.g., account name, container name for
Blob Storage, or connection string for SQL
Database).
• Click Create to link the storage to your Azure
ML workspace
3. Access the Datastore in Azure ML: By creating a datastore, you ensure that your ML
• Once the datastore is created, you can pipeline can securely access and work with the
reference it in your ML pipeline to read or data stored in Azure, enabling seamless data
write data. integration in your model training and inference
• Use the Data module in Azure ML to import workflows.
the data from the datastore into the
pipeline for processing.
Creating a Compute Cluster in Azure ML
To run machine learning models on Azure, you need a
compute cluster. The Standard_DS3_v2 is
recommended for small datasets as it provides good
performance for classical ML model training, with a cost
of $0.23 per hour.
Steps to Create a Compute Cluster
1. Open the Azure Machine Learning workspace in
the Azure portal.
2. In the left-hand menu, click on Compute under
the Manage section. 8. Choose Auto-scaling if you want the cluster to
3. Click on + New to create a new compute cluster. scale up or down automatically depending on
4. Under Compute target type, select Compute demand.
cluster. 9. Set the location where the compute cluster will
5. In the VM Size dropdown, select be created (preferably in the same region as your
Standard_DS3_v2. workspace).
6. Provide a name for your compute cluster (e.g., 10. Click Create to deploy the compute cluster.
"ML-Compute-Cluster"). 11. Once the cluster is created, monitor its status in
7. Set the minimum nodes (e.g., 1) and maximum the Compute section, which may take a few
nodes (e.g., 4) based on your expected workload minutes.
Linear Regression
Linear Regression is a fundamental statistical
and machine learning method used for
modeling the relationship between a
dependent variable (target) and one or more
independent variables (predictors).
The goal of linear regression is to find the
best-fitting linear equation that minimizes the
difference between the predicted and actual
values of the target variable
Linear regression is a good choice when you
want a simple model for a basic predictive
task
Linear regression also tends to work well on
high-dimensional, sparse data sets lacking
complexity
Linear Regression
Linear Regression is a fundamental statistical
and machine learning method used for
modeling the relationship between a
dependent variable (target) and one or more
independent variables (predictors).
The goal of linear regression is to find the
best-fitting linear equation that minimizes the
difference between the predicted and actual
values of the target variable
Linear regression is a good choice when you
want a simple model for a basic predictive
task
Linear regression also tends to work well on
high-dimensional, sparse data sets lacking
complexity
Linear Regression Pipeline
Steps to Log in to Azure Portal and Access
Azure ML Workspace
• Go to
[Link]
• Log in with your Azure account
credentials.
• Use the Search bar and type Azure
Machine Learning Workspaces.
• Click on your desired Azure ML
Workspace from the list.
Done! You're now in your Azure ML
Workspace.
Pipelines in Azure ML
Launch Azure ML Studio
Linear Regression Pipeline
In the Azure ML Studio interface,
navigate to the left-hand menu.
Click on "Designer" under the
Authoring section
Click on "Create New Pipeline"
to start building a new
machine learning pipeline.
Linear Regression Pipeline – Import Data
Steps to Import Data in Azure ML Designer
Open Designer in Azure ML Studio
Drag the "Import Data" component onto
the canvas
Set Data Source Type to Azure Blob
Storage.
Select your Datastore from the dropdown
Enter the file path to your data in the
storage account
Choose the correct file format (e.g., CSV).
Done! You're ready to use the data in your pipeline.
Linear Regression Pipeline – Clean Data
Steps to Add and Configure the Clean
Missing Data Component
Drag the "Clean Missing Data" component
onto the canvas in Designer
Connect it to the output of the Import Data
component
Click the "Clean Missing Data" component
to open the configuration pane
Select the columns you want to clean
under the Columns to be Cleaned setting
Choose a Cleaning Mode (e.g., remove
rows, replace with mean/median/mode)
Specify any additional parameters like
Minimum/Maximum Missing Value Ratios,
if needed
Linear Regression Pipeline – Split data
Steps to Add and Configure the Split Data
Component
Drag the "Split Data" component onto the
canvas in Designer
Connect it to the output of the Clean
Missing Data component
Click the "Split Data" component to open
the configuration pane
Set the Fraction of rows in the first output
dataset (e.g., 0.8 for 80% training data).
Choose the Splitting Mode (e.g., Split Rows
or Split on a column’s value)
Configure any additional parameters
based on your dataset and requirements
Linear Regression Pipeline – Train Model
Steps to Add and Configure Train Model
with Linear Regression
Add Linear Regression:
Drag the Linear Regression component onto
the canvas
Connect Linear Regression to Train Model
Connect the Linear Regression component to
the Train Model component (left port)
Connect Data to Train Model:
Connect the training dataset (first output of
Split Data) to the Train Model component
(right port)
Configure Train Model:
Click Train Model and select the target column
(the column you want to predict).
Now your model is set to train with Linear Regression!
Linear Regression Pipeline – Score Model
This component evaluates the
performance of the trained model by
applying it to unseen data (the test set).
Drag the "Score Model" component onto
the canvas.
Connect the trained model (output of
Train Model) to the Score Model
component (left port).
Connect the test dataset (second output
of Split Data) to the Score Model
component (right port).
No additional configuration is needed; the
Score Model will score the test data using
the trained model.
Linear Regression Pipeline – Evaluate Model
Drag the "Evaluate Model"
component onto the canvas
It compares predicted results to
actual values, giving
performance metrics
Connect the Score Model output
(predictions) to the Evaluate
Model component
Linear Regression Pipeline
Configure and Submit the pipeline
Linear Regression Pipeline – Submit pipeline
Choose to create a new experiment
or select an existing one to track the
pipeline results.
Linear Regression Pipeline
Select the compute cluster you’ve
created for ML training.
Linear Regression Pipeline
Click Submit to run the pipeline on
the selected compute cluster.
Linear Regression Pipeline – Monitor Pipeline Run
After submitting the pipeline, go to the Jobs
section to monitor its progress
The pipeline will take several minutes to
complete, depending on the data size and
compute resources used
In the Jobs section, you can check the status,
view logs, and see detailed metrics for each
step of the pipeline
Once the job completes, you’ll see a
Completed status, and you can access the
outputs for further analysis or model
deployment
If needed, you can also rerun the job or view
historical runs for comparison
Model Evaluation
In the Evaluate Model component, you can view several performance evaluation metrics to
assess the model’s accuracy
R² (Coefficient of Determination): Indicates the proportion
of variance in the target variable explained by the model. A
value close to 1 suggests a better model fit.
RMSE (Root Mean Squared Error): Measures the average
error between predicted and actual values
MAE (Mean Absolute Error): The average of the absolute
differences between predicted and actual values
Use R² (Coefficient of Determination) to select the best model. A higher R² value indicates a
better fit, showing that the model explains more of the variation in the data.
Model Evaluation
Coefficient of Determination - R²
The proportion of variance in
the dependent variable that the
model explains
Indicates how well the model
fits the data, with 1 being a
perfect fit
Higher is better – means the
model captures more variability
in the data
Model Evaluation
Model Coefficient of Determination
Linear Regression - Ordinary Least Squares 0.998319
Linear Regression - Online Gradient Descent 0.997938
*Coefficient of Determination: Indicates how well the model fits the data, with 1 being a perfect fit.
Poisson
Regression
Poisson Regression
Poisson Regression is a
statistical model used to model
count data. It assumes that the
target variable follows a Poisson
distribution, which describes the
number of events occurring in a
fixed interval of time or space.
The distribution is characterized
by a single parameter, λ
(lambda), which represents the
average rate of occurrence of
events.
Poisson Regression
In the designer, replace the Linear Regression
component with the Poisson Regression
component
Connect the Poisson Regression component
to the Train Model component
Configure the Poisson Regression component:
• Set the target column (the variable you're
trying to predict) and features (predictor
variables).
Connect the output from the Split Data
component (the training dataset) to the Train
Model component
After configuration, click Submit to run the
pipeline with Poisson Regression instead of
Linear Regression
Monitor the job status in the Jobs section until
it completes
Model Evaluation
Model Coefficient of Determination
Linear Regression - Ordinary Least Squares 0.998319
Linear Regression - Online Gradient Descent 0.997938
Poisson Regression 0.960831
*Coefficient of Determination: Indicates how well the model fits the data, with 1 being a perfect fit.
Decision Forest
Regression and
Boosted
Decision Tree
Decision Forest Regression
Decision Forest Regression is an ensemble learning method that
uses multiple decision trees to predict continuous target values. It
is similar to random forests but focuses on regression tasks rather
than classification. In this model, each decision tree independently
predicts a value, and the final prediction is obtained by averaging
the results from all the trees.
A decision forest typically consists of many decision trees, each
trained on a random subset of the training data. The ensemble
approach helps reduce overfitting by averaging the predictions,
making the model more robust and accurate.
In decision forest regression:
1. Each tree splits the data based on feature values to
minimize variance in the target.
2. The model works well with both linear and non-linear
relationships in data.
3. The output is the average of the predictions made by all
the trees in the forest, which helps in reducing the bias
and variance compared to a single decision tree.
Boosted Decision Tree
Boosting is a well-known method for constructing ensemble
models, alongside techniques like bagging and random
forests
In Azure Machine Learning, boosted decision trees utilize an
optimized version of the MART gradient boosting algorithm.
Gradient boosting is a technique designed for regression
tasks, where each regression tree is built incrementally
At each step, a loss function is used to evaluate the error,
and the model adjusts to correct it in the next step
As a result, the prediction model is an ensemble of simpler
models, each contributing to the final prediction
In regression problems, boosting builds a series of trees in a
step-wise fashion, and then selects the optimal tree using
an arbitrary differentiable loss function
For additional information, see these article:
[Link]
e_boosting
DFR Vs BDTR Pipeline
Drag the Decision Forest Regression component onto the designer
canvas and connect it to the Train Model component.
Drag the Boosted Decision Tree Regression component onto the
canvas and connect it to another Train Model component.
Connect the outputs of the Split Data component to both Train Model
components
Configure the Train Model component:
• Set the target column (the variable to predict) and features
(predictor variables).
• Do this for both the Decision Forest and Boosted Decision Tree
models
Once both models are configured, run the pipeline to train both
models
Use the Evaluate Model component to compare the performance of
both models.
Compare performance metrics such as R² and RMSE to determine
which model performs better
Follow the image by the side for further guidance on how to connect and
configure the components.
DFR Vs BDTR Pipeline
Pipeline Completed
The pipeline has successfully
run, and the models have
been trained and evaluated.
You can now review the
results and compare the
performance metrics of both
models. The next steps
involve interpreting the
evaluation metrics and
selecting the
best-performing model for
your predictive task.
Model Evaluation
Model Coefficient of Determination
Linear Regression - Ordinary Least Squares 0.998319
Linear Regression - Online Gradient Descent 0.997938
Poisson Regression 0.960831
Decision Forest Regression 0.991248
Boosted Decision tree 0.99014
After evaluating both the Decision Forest Regression and Boosted Decision Tree Regression models, along
with the Linear Regression model, we compared their R² values.
The Linear Regression model showed the highest R² value, indicating it explained the most variance in the
data. Based on this, Linear Regression was selected as the best model for the task.
This model will be used for further predictions and analysis.
*Coefficient of Determination: Indicates how well the model fits the data, with 1 being a perfect fit.
ML Pipeline with
Normalization
ML Pipeline with Normalization
The image shows how normalization is performed
in an ML pipeline using the Normalize Data
component and the Apply Transformation
component
The Normalize Data component scales the
features of the dataset to a specified
range or standardizes them, ensuring that
the data is on a consistent scale for the
model.
The Apply Transformation component is
used to apply the normalization
transformation to new data, ensuring that
the same preprocessing steps are applied
consistently across different datasets
This process helps improve model performance,
especially when features have different units or
scales, allowing the algorithm to learn effectively
Real-time and
Batch Inference
Pipelines
Inference Pipelines in Azure ML
Inference pipelines enable you to
operationalize your trained
machine learning model for
predictions
They provide a seamless way to
deploy and utilize your models in
production environments
Once deployed, you receive a
unique URL that can be integrated
into applications, websites, or
other systems, enabling real-time
or batch predictions with ease.
Inference Pipelines in Azure ML
Once a pipeline run is complete, it can be deployed
as an Inference Pipeline for making predictions on
new data. There are two types of inference:
real-time inference and batch inference
Real-time inference provides an endpoint
(URL) that can be used to send new data
and get predictions instantly via the web
Batch inference allows for processing large
datasets in bulk, returning predictions for
multiple data points at once
The deployed inference pipeline enables you to use
your trained model to make predictions on future
data and integrate it into applications or workflows
Differences: Batch vs. Real-Time Inference
Aspect Batch Inference Real-Time Inference
Large datasets processed Single data points or small
Data Input
in bulk batches
Higher latency Low latency
Latency
(hours/minutes) (milliseconds/seconds)
Periodic, large-scale Immediate responses for
Use Case
processing online apps
Cost-effective for bulk Higher cost for maintaining
Cost
operations endpoints
Output Predictions saved for offline
Predictions available instantly
Availability use
Conclusion
Conclusion
Explored data preparation and summarization for machine learning (Handled Missing
Data, Split Data, Normalize Data, Summarise Data).
Built training pipelines and studied various regression techniques:
• Linear Regression
• Boosted Decision Tree Regression
• Decision Forest Regression
• Poisson Regression
Evaluated and compared different ML models, selecting Linear Regression as the
best-performing model
Introduced inference pipelines to deploy models for predictions
Learned to create and differentiate between batch inference and real-time inference
pipelines
Assignment
Assignment
Scenario:
Angel Real Estate Company aims to predict house prices
(house_price_of_unit_area) using transaction data collected
over time. The dataset includes features such as transaction
date, house age, distance to the nearest bus station, number of
convenience stores nearby, latitude, and longitude.
Tasks
• Store the dataset on Azure Blob Storage.
• Build pipelines using:
1. Linear Regression (OLS and Online Gradient Descent)
2. Boosted Decision Tree Regression
3. Decision Forest Regression
• Evaluate each model, export metrics (e.g., RMSE), and
identify the best-performing model.
Note: Handle missing data by replacing missing values with the mean.
Assignment
Submission Guide
• For each model, the following should be uploaded to your
portal:
1. Screenshot the completed pipeline.
2. Performance Metrics.
3. Identify and justify your best-performing model
Ensure all files are well-named and organized.