UNIT V MACHINE LEARNING IN PRACTICE AND APPLICATIONS

Performance Measurement, Azure Machine Learning. Applications: Image

Recognition – Email spam and Malware Filtering – Online fraud detection- Medical
Diagnosis.

Performance Metrics in Machine Learning

Evaluating the performance of a Machine learning model is one of the important steps while
building an effective ML model. To evaluate the performance or quality of the model, different
metrics are used, and these metrics are known as performance metrics or evaluation
metrics. These performance metrics help us understand how well our model has performed for
the given data. In this way, we can improve the model's performance by tuning the hyper-
parameters. Each ML model aims to generalize well on unseen/new data, and performance
metrics help determine how well the model generalizes on the new dataset.

In machine learning, each task or problem is divided into classification and Regression. Not all
metrics can be used for all types of problems; hence, it is important to know and understand
which metrics should be used. Different evaluation metrics are used for both Regression and
Classification tasks. In this topic, we will discuss metrics used for classification and regression
tasks.

1. Performance Metrics for Classification

In a classification problem, the category or classes of data is identified based on training
data. The model learns from the given dataset and then classifies the new data into
classes or groups based on the training. It predicts class labels as the output, such
as Yes or No, 0 or 1, Spam or Not Spam, etc. To evaluate the performance of a
classification model, different metrics are used, and some of them are as follows:

o Accuracy
o Confusion Matrix
 o Precision
o Recall
o F-Score
o AUC(Area Under the Curve)-ROC

I. Accuracy
The accuracy metric is one of the simplest Classification metrics to implement, and it can
be determined as the number of correct predictions to the total number of predictions.

It can be formulated as:

To implement an accuracy metric, we can compare ground truth and predicted values in
a loop, or we can also use the scikit-learn module for this.

Firstly, we need to import the accuracy_score function of the scikit-learn library as

follows:

from [Link] import accuracy_score

Here, metrics is a class of sklearn.

Then we need to pass the ground truth and predicted values in the function to calcul
ate the accuracy.
1.
2. print(f'Accuracy Score is {accuracy_score(y_test,y_hat)}')
Although it is simple to use and implement, it is suitable only for cases where an equal
number of samples belong to each class.

When to Use Accuracy?

It is good to use the Accuracy metric when the target variable classes in data are
approximately balanced. For example, if 60% of classes in a fruit image dataset are of
Apple, 40% are Mango. In this case, if the model is asked to predict whether the image
is of Apple or Mango, it will give a prediction with 97% of accuracy.

When not to use Accuracy?

It is recommended not to use the Accuracy measure when the target variable majorly
belongs to one class. For example, Suppose there is a model for a disease prediction in
which, out of 100 people, only five people have a disease, and 95 people don't have
one. In this case, if our model predicts every person with no disease (which means a
bad prediction), the Accuracy measure will be 95%, which is not correct.
II. Confusion Matrix
A confusion matrix is a tabular representation of prediction outcomes of any binary
classifier, which is used to describe the performance of the classification model on a set
of test data when true values are known.

The confusion matrix is simple to implement, but the terminologies used in this matrix
might be confusing for beginners.

A typical confusion matrix for a binary classifier looks like the below image(However, it
can be extended to use for classifiers with more than two classes).

We can determine the following from the above matrix:

o In the matrix, columns are for the prediction values, and rows specify the
Actual values. Here Actual and prediction give two possible classes, Yes or
No. So, if we are predicting the presence of a disease in a patient, the
Prediction column with Yes means, Patient has the disease, and for NO, the
Patient doesn't have the disease.
o In this example, the total number of predictions are 165, out of which 110 time
predicted yes, whereas 55 times predicted No.
o However, in reality, 60 cases in which patients don't have the disease,
whereas 105 cases in which patients have the disease.

In general, the table is divided into four terminologies, which are as follows:

1. True Positive(TP): In this case, the prediction outcome is true, and it is true in
reality, also.
2. True Negative(TN): in this case, the prediction outcome is false, and it is false in
reality, also.
3. False Positive(FP): In this case, prediction outcomes are true, but they are false
in actuality.
4. False Negative(FN): In this case, predictions are false, and they are true in
actuality.
III. Precision
The precision metric is used to overcome the limitation of Accuracy. The precision
determines the proportion of positive prediction that was actually correct. It can be
calculated as the True Positive or predictions that are actually true to the total positive
predictions (True Positive and False Positive).

IV. Recall or Sensitivity

It is also similar to the Precision metric; however, it aims to calculate the proportion of
actual positive that was identified incorrectly. It can be calculated as True Positive or
predictions that are actually true to the total number of positives, either correctly
predicted as positive or incorrectly predicted as negative (true Positive and false
negative).

The formula for calculating Recall is given below:

When to use Precision and Recall?

From the above definitions of Precision and Recall, we can say that recall determines
the performance of a classifier with respect to a false negative, whereas precision gives
information about the performance of a classifier with respect to a false positive.

So, if we want to minimize the false negative, then, Recall should be as near to 100%,
and if we want to minimize the false positive, then precision should be close to 100% as
possible.

In simple words, if we maximize precision, it will minimize the FP errors, and if we

maximize recall, it will minimize the FN error.

V. F-Scores
F-score or F1 Score is a metric to evaluate a binary classification model on the basis of
predictions that are made for the positive class. It is calculated with the help of Precision
and Recall. It is a type of single score that represents both Precision and Recall. So, the
F1 Score can be calculated as the harmonic mean of both precision and Recall,
assigning equal weight to each of them.

The formula for calculating the F1 score is given below:

When to use F-Score?

As F-score make use of both precision and recall, so it should be used if both of them
are important for evaluation, but one (precision or recall) is slightly more important to
consider than the other. For example, when False negatives are comparatively more
important than false positives, or vice versa.

VI. AUC-ROC
Sometimes we need to visualize the performance of the classification model on charts;
then, we can use the AUC-ROC curve. It is one of the popular and important metrics for
evaluating the performance of the classification model.

Firstly, let's understand ROC (Receiver Operating Characteristic curve) curve. ROC
represents a graph to show the performance of a classification model at different
threshold levels. The curve is plotted between two parameters, which are:

o True Positive Rate

o False Positive Rate

TPR or true Positive rate is a synonym for Recall, hence can be calculated as:

FPR or False Positive Rate can be calculated as:

To calculate value at any point in a ROC curve, we can evaluate a logistic regression
model multiple times with different classification thresholds, but this would not be much
efficient. So, for this, one efficient method is used, which is known as AUC.

AUC: Area Under the ROC curve

AUC is known for Area Under the ROC curve. As its name suggests, AUC calculates
the two-dimensional area under the entire ROC curve, as shown below image:
AUC calculates the performance across all the thresholds and provides an aggregate
measure. The value of AUC ranges from 0 to 1. It means a model with 100% wrong
prediction will have an AUC of 0.0, whereas models with 100% correct predictions will
have an AUC of 1.0.

When to Use AUC

AUC should be used to measure how well the predictions are ranked rather than their
absolute values. Moreover, it measures the quality of predictions of the model without
considering the classification threshold.

When not to use AUC

As AUC is scale-invariant, which is not always desirable, and we need calibrating

probability outputs, then AUC is not preferable.

Further, AUC is not a useful metric when there are wide disparities in the cost of false
negatives vs. false positives, and it is difficult to minimize one type of classification error.

2. Performance Metrics for Regression

Regression is a supervised learning technique that aims to find the relationships
between the dependent and independent variables. A predictive regression model
predicts a numeric or discrete value. The metrics used for regression are different from
the classification metrics. It means we cannot use the Accuracy metric (explained above)
to evaluate a regression model; instead, the performance of a Regression model is
reported as errors in the prediction. Following are the popular metrics that are used to
evaluate the performance of Regression models.

o Mean Absolute Error

o Mean Squared Error
o R2 Score
 o Adjusted R2

I. Mean Absolute Error (MAE)

Mean Absolute Error or MAE is one of the simplest metrics, which measures the
absolute difference between actual and predicted values, where absolute means taking
a number as Positive.

To understand MAE, let's take an example of Linear Regression, where the model draws
a best fit line between dependent and independent variables. To measure the MAE or
error in prediction, we need to calculate the difference between actual values and
predicted values. But in order to find the absolute error for the complete dataset, we
need to find the mean absolute of the complete dataset.

The below formula is used to calculate MAE:

Here,

Y is the Actual outcome, Y' is the predicted outcome, and N is the total number of data
points.

MAE is much more robust for the outliers. One of the limitations of MAE is that it is not
differentiable, so for this, we need to apply different optimizers such as Gradient
Descent. However, to overcome this limitation, another metric can be used, which is
Mean Squared Error or MSE.

II. Mean Squared Error

Mean Squared error or MSE is one of the most suitable metrics for Regression
evaluation. It measures the average of the Squared difference between predicted values
and the actual value given by the model.

Since in MSE, errors are squared, therefore it only assumes non-negative values, and it
is usually positive and non-zero.

Moreover, due to squared differences, it penalizes small errors also, and hence it leads
to over-estimation of how bad the model is.

MSE is a much-preferred metric compared to other regression metrics as it is

differentiable and hence optimized better.

The formula for calculating MSE is given below:

Here,

Y is the Actual outcome, Y' is the predicted outcome, and N is the total number of data
points.

III. R Squared Score

R squared error is also known as Coefficient of Determination, which is another popular
metric used for Regression model evaluation. The R-squared metric enables us to
compare our model with a constant baseline to determine the performance of the model.
To select the constant baseline, we need to take the mean of the data and draw the line
at the mean.

The R squared score will always be less than or equal to 1 without concerning if the
values are too large or small.

IV. Adjusted R Squared

Adjusted R squared, as the name suggests, is the improved version of R squared error.
R square has a limitation of improvement of a score on increasing the terms, even
though the model is not improving, and it may mislead the data scientists.

To overcome the issue of R square, adjusted R squared is used, which will always show
a lower value than R². It is because it adjusts the values of increasing predictors and
only shows improvement if there is a real improvement.

We can calculate the adjusted R squared as follows:

Here,

n is the number of observations

k denotes the number of independent variables

and Ra2 denotes the adjusted R2

Azure Machine Learning
Azure Machine Learning (Azure ML), part of Microsoft’s public cloud offerings, is a set of
managed cloud services and tools to help Data Science and Machine Learning professionals
build, test, deploy and share machine learning solutions.

Overview of Azure Machine Learning

Azure ML is a cloud service intended for Machine Learning professionals, data
scientists, and engineers to train and deploy machine learning models, manage
Machine Learning Operations (MLOps) and enhance their daily workflows. It’s
designed to help accelerate and manage machine learning project lifecycles with
collaboration, automation, deployment, scaling, integration, and security tools. Azure
Machine Learning enables professionals to create models natively, or use models
built from open-source platforms like Pytorch, TensorFlow, scikit-learn, etc., and
monitor, retrain and redeploy models using its MLOps tools.

The typical flow for a machine learning process is shown in Figure 1.

Figure 1. Typical Machine Learning project lifecycle

The Azure Machine Learning Studio (Azure ML Studio) facilitates quick model
creation and deployment, with an easy-to-use graphical user interface (GUI). The
automated Machine Learning (AutoML) feature in Azure ML speeds up the repetitive
and time-consuming process of feature and algorithm selection.

While Azure ML comes with a large selection of pre-built Machine Learning

algorithms and modules, it also allows for extending your models with custom R and
Python scripts. Azure ML facilitates easy deployment of models as web services, to
be consumed by applications in real-time as well as batch processing modes. All
these tasks can be done using Azure ML Studio Graphical User Interface (GUI) or
Python SDK.
Azure Machine Learning Architecture Overview

Figure 2. Azure Machine Learning high-level architecture

A high-level architecture overview of Azure ML will familiarize readers with its

various components and how they work together.

Note: There are a lot of terms and concepts, but throughout the series, it should
become clearer as I make user of these concepts to build out examples.

An Azure “Resource group” is a container of related resources for a solution, that

typically share the same solution lifecycle. It stores metadata about your resources
and is tied to a geographical region.

 Azure “Machine Learning workspace” is a top-level resource for Azure ML

and is the centralized place to manage resources used to train and deploy
models.
 Azure ML “Assets” like Environments, Experiments, Pipelines, Datasets,
Models, and Endpoints are created while using Azure ML.
o An “Environment” is a collection of Python or R packages and
libraries, environment variables, and various settings that encapsulate the
needs of the machine learning model’s training and scoring scripts
o A single execution of a training script, along with the record of its
metadata, metrics, and output, is called a “Run”. An “Experiment” is a
collection of multiple runs of a particular training script.
o Machine Learning “Pipelines” are used to create and manage
workflows that orchestrate various phases of machine learning like data
preparation, model training, model deployment, scoring, etc.
 o An Azure ML “Dataset” is a reference to the data source location and a
copy of its metadata. It is also known as a “Data Asset”.
o Creating a Machine Learning model involves choosing an algorithm,
training it with data, and hyperparameter tuning. A trained “Model” can accept
input and produce (infer) the output, which is commonly referred to as a
prediction.
o Machine learning models are registered in the Azure Machine Learning
workspace and deployed from the registry as a service “endpoint”, which is
simply an instance of the model hosted in the cloud as a web service. It can
be a “Real-time” or “batch-processing” endpoint
 An Azure ML “Datastore” is a reference to an existing storage account on
Azure and secures the connection information without risking the
authentication credentials and integrity of the source of data. It is used by
Data Assets (also known as datasets) to securely connect to Azure storage
services.
 “Dependencies” are the other Azure Resources like “Azure Storage
Account(s)”, “Azure Container Registry”, “Azure Key Vault” and “Azure
Application Insights”, used by Azure ML workspace.
 A “Compute target” is a designated compute resource of the environment
where the ML model training script is run, or inference service is deployed to
and hosted. It’s called a “Linked Service” when its location is the Machine
Learning professional’s local machine, on-premises resources, or cloud-based
resources not hosted on Azure. When the compute target is hosted in and
fully managed by Azure Machine Learning, it’s called a “Managed
Resource”. Azure Machine Learning offers two fully managed cloud-based
Virtual Machines (VM) for machine learning development tasks.
 “Compute instance” is intended to serve as the Machine Learning
professional’s development workstation. It’s a VM with multiple tools and
environments pre-installed for common machine-learning tasks
 “Compute cluster” is a set of VMs capable of scaling to multiple nodes when
needed for large workloads. It scales up automatically when a job is submitted
and is well-suited for dev/test deployments

When the compute target used to host production deployment of an ML model for
performing inference is fully managed by Azure Machine learning, it’s called an
“inference cluster”. This currently includes “Azure Machine Learning endpoints”
and “Azure Machine Learning Kubernetes”. Both can be used for Real-time and
Batch inference.

Azure Machine Learning Workflow Design

Figure 3. High-Level steps of typical Azure Machine Learning Workflow

The typical high-level design of an Azure Machine Learning Workflow involves the
following steps:

1. Acquire Data – This is typically the first step for any Machine Learning
workflow and involves making the raw data available to the Azure Machine
experiment. Azure ML offers several options to gather data including manual
data entry, Azure Blob storage, Azure SQL Database, Web URLs,
compressed files, etc.
2. Data Preparation – Azure ML offers numerous modules to prepare and
transform data. It includes tools for filtering, cleaning missing values, adding
rows and columns, changing data types, splitting the data set for training and
testing, etc.
3. Feature Engineering – Azure ML provides various methods for Feature
engineering like Filter-based feature selection, Fisher Linear Discriminant
Analysis, Permutation Feature Importance, etc.
4. Select and implement Machine Learning Algorithms – Azure ML comes
with a wide array of built-in Machine Learning Algorithms and options to tune
their parameters.
5. Train Machine Learning Model – Azure ML provides modules to quickly train
and score Machine Learning Models
6. Evaluate Machine Learning Models – Azure ML provides modules to easily
evaluate the performance of a trained model using industry-standard metrics.

In the rest of this article, we are going to start the process of building an Azure ML
example.
Azure Machine Learning workspace setup
Azure Machine Learning workspace is the central hub for all Azure ML artifacts and
the repository for storing experiments, logs, etc. This section walks you through the
steps of setting up their Azure Machine Learning workspace and upload a demo file
in their storage account.

Note: An Azure account with access to the Azure Portal is needed to perform these
steps. Please follow this link to learn more about creating your free account with
Microsoft Azure.

This first step is designed to help you understand the process of setting up your
workspace and upload data to be processed.

1. Log into the Azure Portal and click on “Create a resource” > select the “AI +
Machine Learning” category > Type “Azure Machine Learning” in the
search bar as shown in Figure 5, then select “Azure Machine Learning” from
the list of results:

Figure 4. Create the Azure Machine Learning Resource

 This brings up the “Azure Machine Learning” product page with a product
overview and tabs for additional details about Plans, Usage Information,
Support, and product reviews. Click the ‘Create’ button to launch the machine
learning creation wizard, as shown in Figure 5.
Figure 5. Azure Machine Learning product page

1. On the “Basics” tab for the machine learning workspace creation wizard. You
can see my choices in Figure 6, but your values will likely vary some. Names
are not important if they work. Many of the names will give you a default that
will work for the example, especially if you plan to delete everything after
following along with the article.
 Select your “Subscription” from the dropdown list.
 Select your “Resource group” from the dropdown list (You can also opt to
create a new resource group using the “create new” option under this field)
 Enter the “Workspace name” you like to create (Please follow the prompts
for naming rules)
 Select “Region” for your workspace from the dropdown list.
 Enter the name for your “Storage account” (or create a new one. Please
follow prompts for naming rules)
 Enter the name for your “Key vault” (or create a new one. Please follow the
prompts for naming rules)
 Enter the name for your “Application insights” (or create a new one. Please
follow the prompts for naming rules)
 Enter the name for your “Container registry” (or create a new one. Please
follow the prompts for naming rules)
 Then click the ‘Review + create’ button, which runs validation on your entries.
Figure 6. Create Machine Learning workspace

 In Figure 7, after validation has passed, you will be presented with a summary
of your entries for review. Click the ‘Create’ button.
Figure 7. Review entries and create machine learning workspace

 This initiates a deployment to create your machine-learning workspace. Once

your deployment is complete (typical run time is under one minute, but it may
take a few minutes), Azure will take you to the deployment overview screen,
as shown in Figure 8.

Please review this page to confirm the green check mark against all
Resources and the Status “OK”
Figure 8. Azure Machine Learning workspace deployment status

Click on the “Go to resource” button to navigate to your newly created machine
learning workspace. This section will familiarize readers with the newly created
machine-learning workspace. Figure 9 shows the screen that should follow, with the
values that match your choices.

Figure 9. Azure Machine Learning workspace overview

 1. The “Overview” section gives basic details of the Machine Learning
workspace. The “Studio web URL” or “Launch studio” button is used to
launch the Azure ML Studio
1. The “Download [Link]” button downloads a file containing
workspace configuration details, for use in Python scripts or Azure ML SDK
notebooks that are outside of your workspace. This is used to connect to the
workspace from any laptop or desktop.
2. The “Tags” menu is used to create, update, and delete tags associated with
your workspace. Tags help organize, identify, and maintain azure resources
based on your company’s needs like grouping by department, application,
functionality, purpose, etc.
3. “Access Control (IAM)” has options to manage access for other
users/groups to the workspace, defining various access levels via roles and
assigning roles to users/groups. In addition to granting access to
users/groups, you can block individual users or entire groups from accessing
your workspace with “Deny”. Such fine-grain access controls help implement
complex scenarios like “all members of the team except the quality assurance
tester, should have direct access to the team’s workspace”.
4. The “Settings” menu has
1. “Networking” option for enabling/disabling public network access to
your workspace, and configuring “private endpoint connections”
2. The “Properties” option listing out all details of the workspace.
3. “Locks” option to configure workspace as “Read-only” or blocking
users from “Delete” action
5. The remaining menu options include “Monitoring and alerts”, “Automation”
and “Support and troubleshooting”.

Let’s upload a test data file into the storage account (I will use it to create a
Datastore). Navigate to the Azure Portal’s left side Main menu > Storage
accounts > mhatreredgatedemostgacct > Containers, to review the list of existing
blob storage containers.
Figure 10. Create a new Blob storage container

As shown in Figure 10, click the ‘+ Container’ button on the top left > Enter a unique
name in the Name text box for the new container > click the ‘Create’ button. Once
the new container is created and shows up under the container list, click to open it.

Download the “datastore_text.txt” file from my GitHub repository using this

URL and upload it to this container as shown in Figure 11. Note that the file is very
small and just contains one record.

Figure 11. Upload text file to the Azure blob storage container

After the file upload completes, navigate back to the storage account (I will use this
file in a subsequent step to create the data asset).

Navigate the left side menu of the storage account, to “Access keys” under the
“Security +” networking section as shown in Figure 12. Access keys can be used to
authenticate your application’s requests to this storage account. Copy one of the
keys, to use in the subsequent step to create the Datastore.
Figure 12. Copy the Storage account access key

Azure Machine Learning Studio

Azure ML Studio (the successor of Machine Learning Studio classic) is the pivotal
platform for most machine learning tasks in Azure. This powerful platform enables a
wide gamut of machine learning professionals with varying levels of skills and
preferences, to work with a range of no-code to extensive-coding options.
Figure 13. Launch Azure Machine Learning Studio

Click on the ‘Launch Studio’ button to open the Azure ML Studio as shown in
Figure 13 (Note: you may be prompted to log in again as you are going to a different
tool in a different window.) This opens the tool as shown in Figure 14.
Figure 14. Azure Machine Learning Studio

The left side menu consists of three key sections: Author, Assets, and Manage

The “Author” section is geared towards creators of Machine learning pipelines and
currently offers three authoring experiences.

 “Notebooks” is the code-first experience popular with seasoned machine

learning professionals who are comfortable coding with Python. The
notebooks created here are stored in the default storage account associated
with the workspace. The notebook user interface experience is fully integrated
within Azure ML studio. Azure offers numerous tutorials and samples to help
accelerate model development
 “Designer” is the Drag and Drop tool for creating and deploying machine
learning models. The Graphical User Interface look and feel is comparable to
Azure Data Factory studio. Experiments are created using pipelines in the
designer interface, which offers a wide array of pre-built and configurable
components for numerous tasks.
 “Automated ML” is the no-code approach to building machine learning
models. It takes dataset, problem class, evaluation metric, and prediction
target as inputs and automatically performs steps of data preprocessing,
feature selection and engineering, algorithm selection, model training, testing,
and hyper-parameter tuning. A machine learning professional can review the
 performance of various models trained through this process and directly
deploy the best one.
 The “Assets” section enables machine learning professionals to create,
customize and manage the numerous assets generated during the authoring
process.
o The “Data” sub-section is used to register and manage Datastores.
Datastores enable a secure connection to storage services on Azure by
storing the connection information, thus eliminating the need to provide
credentials in scripts for data access. The “Dataset monitor” feature
(currently in preview when this article was published) can be configured to
detect data drift between training and inference data.
o The “Jobs” sub-section is used to create new experiments, or run
sample experiments with Notebooks and with code using the Python SDK
o “Components” are the basic building blocks used to perform a given
task like data processing, model training, scoring, etc. They have predefined
input/output ports, parameters, and environments that can be shared and
reused in multiple pipelines. This sub-section enables machine learning
professionals to register code from GitHub, Azure DevOps, or local files to
create shareable components that can be used as building blocks for several
machine learning projects.
o Pipelines authored using the designer can be viewed and orchestrated
via the “Pipelines” sub-section. The “Pipeline jobs” tab shows details of the
pipeline run, while the “Pipeline drafts” tab list pipelines that have never run
so far. When Azure ML pipelines are published as a REST endpoint (for
parameterized reuse or invoking jobs from external systems), they are listed
under the “Pipeline endpoints” tab.
o Environments specify the Docker image, Python packages, and
software settings for executing your training and scoring scripts. They are
managed and versioned entities that enable reproducible, auditable, and
portable machine-learning workflows across different compute targets. The
“Environments” sub-section contains a list of “curated environments”, and
an option for machine learning professionals to create their own user-defined
“custom environments”. Curated environments are predefined environments
that offer good starting points for building your environments. Curated
environments are backed by cached Docker images, providing a reduced run
preparation cost. Custom environments are user-defined environments
created from a Docker image, a Docker build context, and a conda
specification with a Docker image.
 The “Models” sub-section enables machine learning professionals to create,
manage and track their registered models. The model registry provides useful
features like version tracking and metadata tagging. All models created under
the workspace are listed here
 Azure Machine Learning “Endpoints” empower machine learning
professionals to deploy machine learning models as web services.
o Real-time endpoints are endpoints that are used for real-time
inferencing. Real-time endpoints contain deployments ready to receive data
from clients and send responses back in real time. They are listed under the
“Real-time endpoints” tab.
 o Batch endpoints are used to run inference on large volumes of data in
batch processing mode that could run for long durations. They take pointers to
data and run batch jobs asynchronously to distribute the workload on compute
clusters. Their outputs can be stored for further analysis in a data store. Batch
endpoints are listed under the “batch endpoints” tab.
 The “Manage” section is equipped with options for creating and managing
Compute, connecting with external Linked Services, and managing the
labelling of data sets.
o Machine learning professionals use the “compute” sub-section to
create and manage various types of compute targets like “compute
instances”, “compute clusters” and “Inference clusters”. The “Attached
computes” tab allows machine learning professionals to bring their own
compute like HDInsight clusters, Virtual machines, Databricks clusters, etc. to
use as compute targets in the Azure Machine Learning workspace.
o “Linked Services” is a collection of external (cloud services that are
outside of the Azure ML workspace) Azure services that can connect with the
Azure Machine learning workspace. The guided integrations experience for
linked services is currently in preview.
o The “Data Labeling” sub-section helps machine learning professionals
to create, scale, and manage labeling efforts for projects involving image
classification, object identification, text classification, and text Named Entity
Recognition. Its “ML Assist” feature improves the speed and accuracy of
labeling large datasets by leveraging the power of machine learning.

Azure Machine Learning Datastore and Data asset

Figure 15. Azure Machine Learning Datastores and Data assets (Datasets)

Data Asset and Datastore are the two key components in Azure ML workspace, to
connect the actual sources of data with the machine learning authoring tools. Figure
15 shows how Data Asset encompasses a logical representation of the actual
sources of data, while the Datastore safeguards the connection details to the actual
sources of data by keeping the credentials in a separate secure location
(represented by the lock icon)

“Data asset” (also known as a dataset) in Azure Machine Learning studio is a

reference to the collection of related data sources, used in various authoring
mechanisms like Automated ML, Notebooks, Designer, and Experiments. The data
referenced in any Data asset can come from a wide variety of Data Sources like
local or web files, Azure blob storage and datastores, Azure Open Datasets, Azure
Data Lake, numerous cloud databases, and an array of other data sources.

“Datastore” in Azure Machine Learning facilitates the connection between Data

assets and the various sources of data, by securely storing the connection and
authentication information. With Datastores in the picture, machine learning
professionals no longer need to provide credentials and data source connection
details in their scripts, pipelines, or any other authoring tools.

Azure best practices recommend storing business data in a separate storage

account to store, manage, secure, and access control it separately from workspace
data. However, for the simplicity of demonstrations in these articles, I am using the
same storage account “mhatreredgatedemostgacct” for storing my demo files.

In Azure Machine Learning Studio, navigate to Data > Datastores as seen in Figure
16.

Figure 16. Azure Machine Learning Datastores list

The Azure Machine Learning workspace has a default Azure Blob Storage Datastore
“woekspaceblobstorage(Default)” for temporarily saving various files generated
during the authoring process. Three other Datastores “workspacefilestore”,
“workspaceworkingdirectory” and “workspaceartifactstore” found in the Storage
Account are set up as part of creating the workspace. Click on each one of them to
reveal details such as storage account name, Storage URI and creation date.

To create a new datastore and link it to the blob storage container created
previously, click on the ‘+ Create’ button as shown in Figure 17.
Figure 17. Create Azure Machine Learning Datastore

On the “Create datastore” screen

1. Enter a name for the new datastore

2. Select Datastore type “Azure Blob Storage” from the drop-down list.
3. Choose “From Azure subscription” as the Account selection method.
4. Select your “Subscription ID” and “Storage account” values from the drop-
down list.
5. Select the “Blob container” name from the drop-down list (I am using the blob
container created earlier in this article, which is part of the Storage account)
6. Set the “Authentication type” value as “Account key” from the drop-down
list.
7. Paste the Access Key copied from the earlier step, into the “Account Key”
field.
8. Then Click the ‘Create’ button at the bottom of the screen.

Once the new datastore is created, it shows up under the list of datastores. Click to
open the “Overview” page > navigate to the “Browse preview” section to confirm
the “datastore_test.txt” file is listed as shown in Figure 18.
Figure 18. Datastore with test file from a blob container

To create the data asset from this datastore, click the ‘create data asset’ button and
in the “Data type” section.

 enter a name for the new Data asset into the “Name” field.
 enter a description for the Data asset into the “Description” field.
 select “File (uri_file)” from the drop-down list in the “Type” field.

Click ‘Next’ to navigate to the “Storage path” section as seen in Figure 19.
Figure 19. Create data asset

After setting the values, click Next to choose the storage path.

 Choose the “Browse to storage path” radio button option.

 Select the “datastore_text.txt” file.

Click Next to navigate to the “Review” section > Review the data asset settings >
Click the ‘Create’ button.

This creates the data asset for the test_dataasset.txt file and makes it available for
the machine learning authoring processes and tools in Azure Machine Learning.
5.3 Image Recognition

Image recognition is one of the most common applications of machine learning. It is used to
identify objects, persons, places, digital images, etc. The popular use case of image
recognition and face detection is, Automatic friend tagging suggestion:

Facebook provides us a feature of auto friend tagging suggestion. Whenever we upload a

photo with our Facebook friends, then we automatically get a tagging suggestion with name,
and the technology behind this is machine learning's face detection and recognition
algorithm.

It is based on the Facebook project named "Deep Face," which is responsible for face
recognition and person identification in the picture.

What is image recognition?

Image recognition is a mechanism used to identify an object within an image and to classify it
in a specific category, based on the way human people recognize objects within different sets
of images.

How does image recognition work with machines

Machines only recognize categories of objects that we have programmed into them. They are
not naturally able to know and identify everything that they see. If a machine is programmed
to recognize one category of images, it will not be able to recognize anything else outside of
the program. The machine will only be able to specify whether the objects present in a set of
images correspond to the category or not. Whether the machine will try to fit the object in the
category, or it will ignore it completely.

For a machine, an image is only composed of data, an array of pixel values. Each pixel
contains information about red, green, and blue color values (from 0 to 255 for each of them).
For black and white images, the pixel will have information about darkness and whiteness
values (from 0 to 255 for both of them).

Machines don’t have a look at the whole image; they are only interested in pixel values and
patterns in these values. They simply take pixel patterns of an item and compare them with
other patterns. If two patterns are close enough, the machine will associate them and
recognize the second pattern as something it has already encountered in the past. In that
sense, what is happening is the machine will look for groups of similar pixel values across
images and will try to place them in specific image categories.

It is very rare that a program recognizes an image at 100%. Pixel patterns are very rarely
100% the same when comparing them. Solving these problems and finding improvements is
the job of IT researchers, the goal being to propose the best experience possible to users.

Practicing Image recognition with Machine Learning

The goal of image recognition is to identify, label and classify objects which are detected into
different categories. Object or image recognition is a whole process that involves various
traditional computer vision tasks:

Image classification: labeling an image and creating categories

Object localization: identifying the location of an object in an image, by surrounding it with a

bounding box

Object Detection: determining the presence of objects with the help of bounding boxes and
categorizing it within the class it belongs

Object Segmentation: distinguishing the various elements. Identify and locate each and every
item of the picture. Segmentation doesn’t use bounding boxes but highlights the contour of
the object in the image.
Set up, Training and Testing

First of all, the machine has to know exactly what it has to look for. Thus, it is necessary to
give it the parameters you decide to work on. Defining the dimensions of bounding boxes and
what elements are inside is crucial. To do so, the machine has to be provided with some
references, which can be pictures, videos or photographs, etc. These elements will allow it to
be more efficient when analyzing future data. This will create a sort of data library that will
then be used by the Neural Network to distinguish the various objects. A Neural Network is
composed of multiple artificial neurons. These neurons are meant to imitate the human brain.
It works with a set of various algorithms also inspired by the way the brain functions. If we
want the image recognition model to analyze and categorize different races of dogs, the
model will need to have a database of the various races in order to recognize them.

Second, the model needs to go on a training phase. The dataset needs to be entered within a
program in order to function properly. And this phase is only meant to train the
Convolutional Neural Network (CNN) to identify specific objects and organize them
accurately in the correspondent classes.

Before an AI model is being used, it needs to be thoroughly tested. To do so, it is necessary to

propose images that were not part of the training phase. Based on whether or not the program
has been able to identify all the items and on the accuracy of classification, the model will be
approved or not.

PROGRAM

import [Link] as plt

from sklearn.model_selection import train_test_split
from [Link] import fetch_lfw_people
from [Link] import classification_report
from [Link] import PCA
from sklearn.neural_network import MLPClassifier
# Load data
lfw_dataset = fetch_lfw_people(min_faces_per_person=100)
_, h, w = lfw_dataset.[Link]
X = lfw_dataset.data
y = lfw_dataset.target
target_names = lfw_dataset.target_names
# split into a training and testing set
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)
# Compute a PCA
n_components = 100
pca = PCA(n_components=n_components, whiten=True).fit(X_train)
# apply PCA transformation
X_train_pca = [Link](X_train)
X_test_pca = [Link](X_test)
# train a neural network
print("Fitting the classifier to the training set")
clf = MLPClassifier(hidden_layer_sizes=(1024,), batch_size=256, verbose=True, early_stopp
ing=True).fit(X_train_pca, y_train)
y_pred = [Link](X_test_pca)
print(classification_report(y_test, y_pred, target_names=target_names))
# Visualization
def plot_gallery(images, titles, h, w, rows=3, cols=4):
[Link]()
for i in range(rows * cols):
[Link](rows, cols, i + 1)
[Link](images[i].reshape((h, w)), cmap=[Link])
[Link](titles[i])
[Link](())
[Link](())
def titles(y_pred, y_test, target_names):
for i in range(y_pred.shape[0]):
pred_name = target_names[y_pred[i]].split(' ')[-1]
true_name = target_names[y_test[i]].split(' ')[-1]
yield 'predicted: {0}\ntrue: {1}'.format(pred_name, true_name)
prediction_titles = list(titles(y_pred, y_test, target_names))
plot_gallery(X_test, prediction_titles, h, w)
[Link]()
5.2 Speech Recognition
Using Google, an option of "Search by voice," it comes under speech recognition, and it's a
popular application of machine learning. Speech recognition is a process of converting voice
instructions into text, and it is also known as "Speech to text", or "Computer speech
recognition." At present, machine learning algorithms are widely used by various applications
of speech recognition. Google assistant, Siri, Cortana, and Alexa are using speech recognition
technology to follow the voice instructions.

Rudimentary speech recognition software has a limited vocabulary and may only identify
words and phrases when spoken clearly. More sophisticated software can handle natural
speech, different accents and various languages.

Speech recognition uses a broad array of research in computer science, linguistics and
computer engineering. Many modern devices and text-focused programs have speech
recognition functions in them to allow for easier or hands-free use of a device.

Speech recognition and voice recognition are two different technologies and should not be
confused:

 Speech recognition is used to identify words in spoken language.

 Voice recognition is a biometric technology for identifying an individual's voice.

How does speech recognition work?

Speech recognition systems use computer algorithms to process and interpret spoken words
and convert them into text. A software program turns the sound a microphone records into
written language that computers and humans can understand, following these four steps:

1. analyze the audio;

2. break it into parts;

3. digitize it into a computer-readable format; and

4. use an algorithm to match it to the most suitable text representation.

Speech recognition software must adapt to the highly variable and context-specific nature of
human speech. The software algorithms that process and organize audio into text are trained
on different speech patterns, speaking styles, languages, dialects, accents and phrasings. The
software also separates spoken audio from background noise that often accompanies the
signal.

To meet these requirements, speech recognition systems use two types of models:

 Acoustic models. These represent the relationship between linguistic units of speech
and audio signals.

 Language models. Here, sounds are matched with word sequences to distinguish
between words that sound similar.

What applications is speech recognition used for?

Speech recognition systems have quite a few applications. Here is a sampling of them.

Mobile devices. Smartphones use voice commands for call routing, speech-to-text
processing, voice dialing and voice search. Users can respond to a text without looking at
their devices. On Apple iPhones, speech recognition powers the keyboard and Siri, the virtual
assistant. Functionality is available in secondary languages, too. Speech recognition can also
be found in word processing applications like Microsoft Word, where users can dictate words
to be turned into text.

Education. Speech recognition software is used in language instruction. The software hears
the user's speech and offers help with pronunciation.

Customer service. Automated voice assistants listen to customer queries and provides
helpful resources.

Healthcare applications. Doctors can use speech recognition software to transcribe notes in
real time into healthcare records.

Disability assistance. Speech recognition software can translate spoken words into text
using closed captions to enable a person with hearing loss to understand what others are
saying. Speech recognition can also enable those with limited use of their hands to work with
computers, using voice commands instead of typing.
Court reporting. Software can be used to transcribe courtroom proceedings, precluding the
need for human transcribers.

Emotion recognition. This technology can analyze certain vocal characteristics to determine
what emotion the speaker is feeling. Paired with sentiment analysis, this can reveal how
someone feels about a product or service.

Hands-free communication. Drivers use voice control for hands-free communication,

controlling phones, radios and global positioning systems, for instance.

Speech recognition algorithms

The power behind speech recognition features comes from a set of algorithms and
technologies. They include the following:

 Natural language processing (NLP): While NLP isn’t necessarily a specific

algorithm used in speech recognition, it is the area of artificial intelligence which
focuses on the interaction between humans and machines through language through
speech and text. Many mobile devices incorporate speech recognition into their
systems to conduct voice search—e.g. Siri—or provide more accessibility around
texting.
 Hidden markov models (HMM): Hidden Markov Models build on the Markov
chain model, which stipulates that the probability of a given state hinges on the
current state, not its prior states. While a Markov chain model is useful for observable
events, such as text inputs, hidden markov models allow us to incorporate hidden
events, such as part-of-speech tags, into a probabilistic model. They are utilized as
sequence models within speech recognition, assigning labels to each unit—i.e. words,
syllables, sentences, etc.—in the sequence. These labels create a mapping with the
provided input, allowing it to determine the most appropriate label sequence.

 N-grams: This is the simplest type of language model (LM), which assigns
probabilities to sentences or phrases. An N-gram is sequence of N-words. For
example, “order the pizza” is a trigram or 3-gram and “please order the pizza” is a 4-
gram. Grammar and the probability of certain word sequences are used to improve
recognition and accuracy.
  Neural networks: Primarily leveraged for deep learning algorithms, neural networks
process training data by mimicking the interconnectivity of the human brain through
layers of nodes. Each node is made up of inputs, weights, a bias (or threshold) and an
output. If that output value exceeds a given threshold, it “fires” or activates the node,
passing data to the next layer in the network. Neural networks learn this mapping
function through supervised learning, adjusting based on the loss function through the
process of gradient descent. While neural networks tend to be more accurate and can
accept more data, this comes at a performance efficiency cost as they tend to be
slower to train compared to traditional language models.

 Speaker Diarization (SD): Speaker diarization algorithms identify and segment

speech by speaker identity. This helps programs better distinguish individuals in a
conversation and is frequently applied at call centers distinguishing customers and
sales agents.

Advantages of speech recognition

There are several advantages to using speech recognition software, including the following:

 Machine-to-human communication. The technology enables electronic devices to

communicate with humans in natural language or conversational speech.

 Readily accessible. This software is frequently installed in computers and mobile

devices, making it accessible.

 Easy to use. Well-designed software is straightforward to operate and often runs in

the background.

 Continuous, automatic improvement. Speech recognition systems that incorporate

AI become more effective and easier to use over time. As systems complete speech
recognition tasks, they generate more data about human speech and get better at what
they do.

What are the disadvantages of speech recognition?

While convenient, speech recognition technology still has a few issues to work through.
Limitations include:
 Inconsistent performance. The systems may be unable to capture words accurately
because of variations in pronunciation, lack of support for some languages and inability
to sort through background noise. Ambient noise can be especially challenging. Acoustic
training can help filter it out, but these programs aren't perfect. Sometimes it's impossible
to isolate the human voice.

 Speed. Some speech recognition programs take time to deploy and master. The
speech processing may feel relatively slow.

 Source file issues. Speech recognition success depends on the recording equipment
used, not just the software.

PROGRAM

import speech_recognition as sr

import pyttsx3

r=[Link]()

def speakText(command):

engine=pyttsx3,init()

[Link](command)

[Link]()

with [Link]()as source2:

r.adjust_for_ambient_noise(source2,duiration=0.2)

audio2=[Link](source2)

MyText=[Link](source2)

MyText=[Link]()

Print(“Did you say “=MyText())

speakText(MyText)

5.3 Email spam and Malware Filtering

Spam email detection: Classifying emails as genuine or spam. This is one of the crucial
components of most email application and services these days. If an application can
successfully separate the spams emails from genuine emails, it can quickly become
application of choice by millions of people. The spam can further be divided into categories
like deal alerts, newsletters, fishing emails, etc. making it a multiclass problem.

Whenever we receive a new email, it is filtered automatically as important, normal, and

spam. We always receive an important mail in our inbox with the important symbol and spam
emails in our spam box, and the technology behind this is Machine learning. Below are some
spam filters used by Gmail:

o Content Filter
o Header filter
o General blacklists filter
o Rules-based filters
o Permission filters

Some machine learning algorithms such as Multi-Layer Perceptron, Decision tree, and Naïve
Bayes classifier are used for email spam filtering and malware detection.

PROGRAM:

# Importing Libraries
import numpy as np
import pandas as pd
import [Link] as plt
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.model_selection import GridSearchCV
from sklearn import svm
# Loading Dataset
data = pd.read_csv(Specific location Ex:
'[Link]
# Checking the information of the dataset
[Link]()
# Checking the information of the dataset
X = data['EmailText'].values
y = data['Label'].values
# Splitting our data into training and testing
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=0)
# Converting text into integer using CountVectorizer()
# Converting String to Integer
cv = CountVectorizer()
X_train = cv.fit_transform(X_train)
X_test = [Link](X_test)
# SVM algorithm
from [Link] import SVC
classifier = SVC(kernel = 'rbf', random_state = 0)
[Link](X_train, y_train)
# Accuracy
print([Link](X_test,y_test))
5.4 Online fraud detection

Machine learning is making our online transaction safe and secure by detecting fraud
transaction. Whenever we perform some online transaction, there may be various ways that a
fraudulent transaction can take place such as fake accounts, fake ids, and steal money in the
middle of a transaction. So to detect this, Feed Forward Neural network helps us by checking
whether it is a genuine transaction or a fraud transaction.

For each genuine transaction, the output is converted into some hash values, and these values
become the input for the next round. For each genuine transaction, there is a specific pattern
which gets change for the fraud transaction hence, it detects it and makes our online
transactions more secure.

#Packages related to general operating system & warnings

import os
import warnings
[Link]('ignore')
#Packages related to data importing, manipulation, exploratory data #analysis, data understan
ding
import numpy as np
import pandas as pd
from pandas import Series, DataFrame
from termcolor import colored as cl # text customization
#Packages related to data visualizaiton
import seaborn as sns
import [Link] as plt
%matplotlib inline
#Setting plot sizes and type of plot
[Link]("font", size=14)
[Link]['[Link]'] = True
[Link](figsize=(6,3))
[Link]()
from [Link].backend_pdf import PdfPages
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn import metrics
from [Link] import MissingIndicator, SimpleImputer
from [Link] import PolynomialFeatures, KBinsDiscretizer, FunctionTransfor
mer
from [Link] import StandardScaler, MinMaxScaler, MaxAbsScaler
from [Link] import LabelEncoder, OneHotEncoder, LabelBinarizer, OrdinalE
ncoder
import [Link] as smf
import [Link] as tsa
from sklearn.linear_model import LogisticRegression, LinearRegression, ElasticNet, Lasso,
Ridge
from [Link] import KNeighborsClassifier, KNeighborsRegressor
# from [Link] import DecisionTreeClassifier, DecisionTreeRegressor, export_graphviz,
export
from [Link] import BaggingClassifier, BaggingRegressor,RandomForestClassifier,
RandomForestRegressor
from [Link] import GradientBoostingClassifier,GradientBoostingRegressor, AdaB
oostClassifier, AdaBoostRegressor
from [Link] import LinearSVC, LinearSVR, SVC, SVR
from xgboost import XGBClassifier
from [Link] import f1_score
from [Link] import accuracy_score
from [Link] import confusion_matrix
data=pd.read_csv("/content/[Link]")
Total_transactions = len(data)
normal = len(data[[Link] == 0])
fraudulent = len(data[[Link] == 1])
fraud_percentage = round(fraudulent/normal*100, 2)
print(cl('Total number of Transactions are {}'.format(Total_transactions), attrs = ['bold']))
print(cl('Number of Normal Transactions are {}'.format(normal), attrs = ['bold']))
print(cl('Number of fraudulent Transactions are {}'.format(fraudulent), attrs = ['bold']))
print(cl('Percentage of fraud Transactions is {}'.format(fraud_percentage), attrs = ['bold']))
sc = StandardScaler()
amount = data['Amount'].values
data['Amount'] = sc.fit_transform([Link](-1, 1))
[Link](['Time'], axis=1, inplace=True)
[Link]
data.drop_duplicates(inplace=True)
[Link]
X = [Link]('Class', axis = 1).values
y = data['Class'].values
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.25, random_state = 1)
# XGBoost
xgb = XGBClassifier(max_depth = 4)
[Link](X_train, y_train)
xgb_yhat = [Link](X_test)
print('Accuracy score of the XGBoost model is {}'.format(accuracy_score(y_test, xgb_yhat)))
print('F1 score of the XGBoost model is {}'.format(f1_score(y_test, xgb_yhat)))

5.5 Medical Diagnosis

In medical science, machine learning is used for diseases diagnoses. With this, medical
technology is growing very fast and able to build 3D models that can predict the exact
position of lesions in the brain. It helps in finding brain tumors and other brain-related
diseases easily.

There are different types of sensing technologies available in medical diagnosis, like
ultrasound, X-ray, magnetic resonance imaging, etc. This diagnostic sensory data can then be
given to a machine learning system which can then analyze the signals and classify the
medical conditions into different predetermined types.
PROGRAM:

import numpy as np
import pandas as pd
import csv
from [Link] import MaximumLikelihoodEstimator
from [Link] import BayesianNetwork
from [Link] import VariableElimination
heartDisease = pd.read_csv('/content/[Link]')
heartDisease = [Link]('?',[Link])
print('Sample instances from the dataset are given below')
print([Link]())
model= BayesianNetwork([('age','heartdisease'),('gender','heartdisease'),
('exang','heartdisease'),('cp','heartdisease'),('heartdisease','restecg'),('heartdisease','chol')])
print('\nLearning CPD using Maximum likelihood estimators')
[Link](heartDisease, estimator=MaximumLikelihoodEstimator)
print('\n Inferencing with Bayesian Network:')
HeartDiseasetest_infer = VariableElimination(model)
print('\n 1. Probability of HeartDisease given evidence= restecg')
q1=HeartDiseasetest_infer.query(variables=['heartdisease'],evidence={'restecg':1})
print(q1)
print('\n 2. Probability of HeartDisease given evidence= cp')
q2=HeartDiseasetest_infer.query(variables=['heartdisease'],evidence={'cp':2})
print(q2)