UNIT- II

Machine Learning Methods

Linear methods – Regression -Classification –Perceptron and Neural

networks – Decision trees – Support vector machines – Probabilistic models –
–Unsupervised learning – Featurization

LINEAR METHODS:

The term linear model implies that the model is specified as a linear combination of features.
Based on training data, the learning process computes one weight for each feature to form a
model that can predict or estimate the target value.

We’ll explore two types of linear models:

1. Linear regression, which is used for regression (numerical predictions).

2. Logistic regression, which is used for classification (categorical predictions).

REGRESSION:

Regression is a supervised learning technique which helps in finding the correlation between
variables and enables us to predict the continuous output variable based on the one or more
predictor variables. It is mainly used for prediction, forecasting, time series modeling, and
determining the causal-effect relationship between variables.

Regression Analysis in Machine learning

Regression analysis is a statistical method to model the relationship between a dependent (target)
and independent (predictor) variables with one or more independent variables. More specifically,
Regression analysis helps us to understand how the value of the dependent variable is changing
corresponding to an independent variable when other independent variables are held fixed. It
predicts continuous/real values such as temperature, age, salary, price, etc.

We can understand the concept of regression analysis using the below example:
Example: Suppose there is a marketing company A, who does various advertisement every year
and get sales on that. The below list shows the advertisement made by the company in the last 5
years and the corresponding sales:

Now, the company wants to do the advertisement of $200 in the year 2019 and wants to know
the prediction about the sales for this year. So to solve such type of prediction problems in
machine learning, we need regression [Link] Tutorial

Regression is a Supervised learning technique which helps in finding the correlation between
variables and enables us to predict the continuous output variable based on the one or more
predictor variables. It is mainly used for prediction, forecasting, time series modeling, and
determining the causal-effect relationship between variable

In Regression, we plot a graph between the variables which best fits the given datapoints, using
this plot, the machine learning model can make predictions about the data. In simple
words, "Regression shows a line or curve that passes through all the datapoints on target-
predictor graph in such a way that the vertical distance between the datapoints and the
regression line is minimum." The distance between datapoints and line tells whether a model
has captured a strong relationship or not.

Some examples of regression can be as:

o Prediction of rain using temperature and other factors

o Determining Market trends
o Prediction of road accidents due to rash driving.
Terminologies Related to the Regression Analysis:
o Dependent Variable: The main factor in Regression analysis which we want to predict
or understand is called the dependent variable. It is also called target variable.
o Independent Variable: The factors which affect the dependent variables or which are
used to predict the values of the dependent variables are called independent variable, also
called as a predictor.
o Outliers: Outlier is an observation which contains either very low value or very high
value in comparison to other observed values. An outlier may hamper the result, so it
should be avoided.
o Multicollinearity: If the independent variables are highly correlated with each other than
other variables, then such condition is called Multicollinearity. It should not be present in
the dataset, because it creates problem while ranking the most affecting variable.
o Underfitting and Overfitting: If our algorithm works well with the training dataset but
not well with test dataset, then such problem is called Overfitting. And if our algorithm
does not perform well even with training dataset, then such problem is
called underfitting.

Why do we use Regression Analysis?

As mentioned above, Regression analysis helps in the prediction of a continuous variable. There
are various scenarios in the real world where we need some future predictions such as weather
condition, sales prediction, marketing trends, etc., for such case we need some technology which
can make predictions more accurately. So for such case we need Regression analysis which is a
statistical method and used in machine learning and data science. Below are some other reasons
for using Regression analysis:

o Regression estimates the relationship between the target and the independent variable.
o It is used to find the trends in data.
o It helps to predict real/continuous values.
o By performing the regression, we can confidently determine the most important factor,
the least important factor, and how each factor is affecting the other factors.

Types of Regression

There are various types of regressions which are used in data science and machine learning. Each
type has its own importance on different scenarios, but at the core, all the regression methods
analyze the effect of the independent variable on dependent variables. Here we are discussing
some important types of regression which are given below:
 o Linear Regression
o Logistic Regression
o Polynomial Regression
o Support Vector Regression
o Decision Tree Regression
o Random Forest Regression
o Ridge Regression
o Lasso Regression:

Linear Regression:

o Linear regression is a statistical regression method which is used for predictive analysis.
o It is one of the very simple and easy algorithms which works on regression and shows the
relationship between the continuous variables.
o It is used for solving the regression problem in machine learning.
o Linear regression shows the linear relationship between the independent variable (X-axis)
and the dependent variable (Y-axis), hence called linear regression.
o If there is only one input variable (x), then such linear regression is called simple linear
regression. And if there is more than one input variable, then such linear regression is
called multiple linear regression.
 o The relationship between variables in the linear regression model can be explained using
the below image. Here we are predicting the salary of an employee on the basis of the
year of experience.

o Below is the mathematical equation for Linear regression:

Y= aX+b
Here,

Y = dependent variables (target variables),

X= Independent variables (predictor variables),
a and b are the linear coefficients

Some popular applications of linear regression are:

o Analyzing trends and sales estimates
o Salary forecasting
o Real estate prediction
o Arriving at ETAs in traffic.

Logistic Regression:

o Logistic regression is another supervised learning algorithm which is used to solve the
classification problems. In classification problems, we have dependent variables in a
binary or discrete format such as 0 or 1.
o Logistic regression algorithm works with the categorical variable such as 0 or 1, Yes or
No, True or False, Spam or not spam, etc.
 o It is a predictive analysis algorithm which works on the concept of probability.
o Logistic regression is a type of regression, but it is different from the linear regression
algorithm in the term how they are used.
o Logistic regression uses sigmoid function or logistic function which is a complex cost
function. This sigmoid function is used to model the data in logistic regression. The
function can be represented as:

o f(x)= Output between the 0 and 1 value.

o x= input to the function
o e= base of natural logarithm.

When we provide the input values (data) to the function, it gives the S-curve as follows:

o It uses the concept of threshold levels, values above the threshold level are rounded up to
1, and values below the threshold level are rounded up to 0.

There are three types of logistic regression:

o Binary(0/1, pass/fail)
o Multi(cats, dogs, lions)
o Ordinal(low, medium, high)
Polynomial Regression:

o Polynomial Regression is a type of regression which models the non-linear

dataset using a linear model.
o It is similar to multiple linear regression, but it fits a non-linear curve between the value
of x and corresponding conditional values of y.
o Suppose there is a dataset which consists of datapoints which are present in a non-linear
fashion, so for such case, linear regression will not best fit to those datapoints. To cover
such datapoints, we need Polynomial regression.
o In Polynomial regression, the original features are transformed into polynomial
features of given degree and then modeled using a linear model. Which means the
datapoints are best fitted using a polynomial line.

o The equation for polynomial regression also derived from linear regression equation that
means Linear regression equation Y= b0+ b1 x, is transformed into Polynomial regression
equation Y= b0+b1 x+ b2x2+ b3x3+.....+ bnxn.
o Here Y is the predicted/target output, b0, b1,... bn are the regression coefficients. x is
our independent/input variable.
o The model is still linear as the coefficients are still linear with quadratic

Support Vector Regression:

Support Vector Machine is a supervised learning algorithm which can be used for regression as
well as classification problems. So if we use it for regression problems, then it is termed as
Support Vector Regression.

Support Vector Regression is a regression algorithm which works for continuous variables.
Below are some keywords which are used in Support Vector Regression:
 o Kernel: It is a function used to map a lower-dimensional data into higher dimensional
data.
o Hyperplane: In general SVM, it is a separation line between two classes, but in SVR, it
is a line which helps to predict the continuous variables and cover most of the datapoints.
o Boundary line: Boundary lines are the two lines apart from hyperplane, which creates a
margin for datapoints.
o Support vectors: Support vectors are the datapoints which are nearest to the hyperplane
and opposite class.

In SVR, we always try to determine a hyperplane with a maximum margin, so that maximum
number of datapoints are covered in that margin. The main goal of SVR is to consider the
maximum datapoints within the boundary lines and the hyperplane (best-fit line) must contain
a maximum number of datapoints. Consider the below image:

Here, the blue line is called hyperplane, and the other two lines are known as boundary lines.

Decision Tree Regression:

o Decision Tree is a supervised learning algorithm which can be used for solving both
classification and regression problems.
o It can solve problems for both categorical and numerical data
o Decision Tree regression builds a tree-like structure in which each internal node
represents the "test" for an attribute, each branch represent the result of the test, and each
leaf node represents the final decision or result.
 o A decision tree is constructed starting from the root node/parent node (dataset), which
splits into left and right child nodes (subsets of dataset). These child nodes are further
divided into their children node, and themselves become the parent node of those nodes.
Consider the below image:

Above image showing the example of Decision Tee regression, here, the model is trying to
predict the choice of a person between Sports cars or Luxury car.

o Random forest is one of the most powerful supervised learning algorithms which is
capable of performing regression as well as classification tasks.
o The Random Forest regression is an ensemble learning method which combines multiple
decision trees and predicts the final output based on the average of each tree output. The
combined decision trees are called as base models, and it can be represented more
formally as:

g(x)= f0(x)+ f1(x)+ f2(x)+....

o Random forest uses Bagging or Bootstrap Aggregation technique of ensemble learning
in which aggregated decision tree runs in parallel and do not interact with each other.
o With the help of Random Forest regression, we can prevent Overfitting in the model by
creating random subsets of the dataset.
Ridge Regression:

o Ridge regression is one of the most robust versions of linear regression in which a small
amount of bias is introduced so that we can get better long term predictions.
o The amount of bias added to the model is known as Ridge Regression penalty. We can
compute this penalty term by multiplying with the lambda to the squared weight of each
individual features.
o The equation for ridge regression will be:

o A general linear or polynomial regression will fail if there is high collinearity between the
independent variables, so to solve such problems, Ridge regression can be used.
o Ridge regression is a regularization technique, which is used to reduce the complexity of
the model. It is also called as L2 regularization.
o It helps to solve the problems if we have more parameters than samples.

Lasso Regression:
 o Lasso regression is another regularization technique to reduce the complexity of the
model.
o It is similar to the Ridge Regression except that penalty term contains only the absolute
weights instead of a square of weights.
o Since it takes absolute values, hence, it can shrink the slope to 0, whereas Ridge
Regression can only shrink it near to 0.
o It is also called as L1 regularization. The equation for Lasso regression will be:

CLASSIFICATION:
What is the Classification Algorithm?

The Classification algorithm is a Supervised Learning technique that is used to identify the
category of new observations on the basis of training data. In Classification, a program learns
from the given dataset or observations and then classifies new observation into a number of
classes or groups. Such as, Yes or No, 0 or 1, Spam or Not Spam, cat or dog, etc. Classes can
be called as targets/labels or categories.

Unlike regression, the output variable of Classification is a category, not a value, such as "Green
or Blue", "fruit or animal", etc. Since the Classification algorithm is a Supervised learning
technique, hence it takes labeled input data, which means it contains input with the
corresponding output.

In classification algorithm, a discrete output function(y) is mapped to input variable(x).

y=f(x), where y = categorical output

The best example of an ML classification algorithm is Email Spam Detector.

The main goal of the Classification algorithm is to identify the category of a given dataset, and
these algorithms are mainly used to predict the output for the categorical data.

Classification algorithms can be better understood using the below diagram. In the below
diagram, there are two classes, class A and Class B. These classes have features that are similar
to each other and dissimilar to other classes.
The algorithm which implements the classification on a dataset is known as a classifier. There
are two types of Classifications:

o Binary Classifier: If the classification problem has only two possible outcomes, then it is
called as Binary Classifier.
Examples: YES or NO, MALE or FEMALE, SPAM or NOT SPAM, CAT or DOG, etc.
o Multi-class Classifier: If a classification problem has more than two outcomes, then it is
called as Multi-class Classifier.
Example: Classifications of types of crops, Classification of types of music.

Learners in Classification Problems:

In the classification problems, there are two types of learners:

1. Lazy Learners: Lazy Learner firstly stores the training dataset and wait until it receives
the test dataset. In Lazy learner case, classification is done on the basis of the most
related data stored in the training dataset. It takes less time in training but more time for
predictions.
Example: K-NN algorithm, Case-based reasoning
2. Eager Learners:Eager Learners develop a classification model based on a training
dataset before receiving a test dataset. Opposite to Lazy learners, Eager Learner takes
more time in learning, and less time in prediction. Example: Decision Trees, Naïve
Bayes, ANN.

Types of ML Classification Algorithms:

Classification Algorithms can be further divided into the Mainly two category:
 o Linear Models
o Logistic Regression
o Support Vector Machines
o Non-linear Models
o K-Nearest Neighbours
o Kernel SVM
o Naïve Bayes
o Decision Tree Classification
o Random Forest Classification

Evaluating a Classification model:

Once our model is completed, it is necessary to evaluate its performance; either it is a

Classification or Regression model. So for evaluating a Classification model, we have the
following ways:

1. Log Loss or Cross-Entropy Loss:

o It is used for evaluating the performance of a classifier, whose output is a probability

value between the 0 and 1.
o For a good binary Classification model, the value of log loss should be near to 0.
o The value of log loss increases if the predicted value deviates from the actual value.
o The lower log loss represents the higher accuracy of the model.
o For Binary classification, cross-entropy can be calculated as:

?(ylog(p)+(1?y)log(1?p))

Where y= Actual output, p= predicted output.

2. Confusion Matrix:

o The confusion matrix provides us a matrix/table as output and describes the performance
of the model.
o It is also known as the error matrix.
o The matrix consists of predictions result in a summarized form, which has a total number
of correct predictions and incorrect predictions. The matrix looks like as below table:
 Actual Positive Actual Negative

Predicted Positive True Positive False Positive

Predicted Negative False Negative True Negative

3. AUC-ROC curve:

o ROC curve stands for Receiver Operating Characteristics Curve and AUC stands
for Area Under the Curve.
o It is a graph that shows the performance of the classification model at different
thresholds.
o To visualize the performance of the multi-class classification model, we use the AUC-
ROC Curve.
o The ROC curve is plotted with TPR and FPR, where TPR (True Positive Rate) on Y-axis
and FPR(False Positive Rate) on X-axis.

Use cases of Classification Algorithms

Classification algorithms can be used in different places. Below are some popular use cases of
Classification Algorithms:

o Email Spam Detection

o Speech Recognition
o Identifications of Cancer tumor cells.
o Drugs Classification
o Biometric Identification, etc.

PERCEPTRON:

In Machine Learning and Artificial Intelligence, Perceptron is the most commonly used term for
all folks. It is the primary step to learn Machine Learning and Deep Learning technologies,
which consists of a set of weights, input values or scores, and a threshold. Perceptron is a
building block of an Artificial Neural Network. Initially, in the mid of 19th century, Mr.
Frank Rosenblatt invented the Perceptron for performing certain calculations to detect input
data capabilities or business intelligence. Perceptron is a linear Machine Learning algorithm used
for supervised learning for various binary classifiers. This algorithm enables neurons to learn
elements and processes them one by one during preparation. In this tutorial, "Perceptron in
Machine Learning," we will discuss in-depth knowledge of Perceptron and its basic functions in
brief. Let's start with the basic introduction of Perceptron.

What is the Perceptron model in Machine Learning?

Perceptron is Machine Learning algorithm for supervised learning of various binary

classification tasks. Further, Perceptron is also understood as an Artificial Neuron or neural
network unit that helps to detect certain input data computations in business intelligence.

Perceptron model is also treated as one of the best and simplest types of Artificial Neural
networks. However, it is a supervised learning algorithm of binary classifiers. Hence, we can
consider it as a single-layer neural network with four main parameters, i.e., input values,
weights and Bias, net sum, and an activation function.

What is Binary classifier in Machine Learning?

In Machine Learning, binary classifiers are defined as the function that helps in deciding whether
input data can be represented as vectors of numbers and belongs to some specific class.

Binary classifiers can be considered as linear classifiers. In simple words, we can understand it
as a classification algorithm that can predict linear predictor function in terms of weight
and feature vectors.

Basic Components of Perceptron

Mr. Frank Rosenblatt invented the perceptron model as a binary classifier which contains three
main components. These are as follows:
 o Input Nodes or Input Layer:

This is the primary component of Perceptron which accepts the initial data into the system for
further processing. Each input node contains a real numerical value.

o Wight and Bias:

Weight parameter represents the strength of the connection between units. This is another most
important parameter of Perceptron components. Weight is directly proportional to the strength of
the associated input neuron in deciding the output. Further, Bias can be considered as the line of
intercept in a linear equation.

o Activation Function:

These are the final and important components that help to determine whether the neuron will fire
or not. Activation Function can be considered primarily as a step function.

Types of Activation functions:

o Sign function
o Step function, and
o Sigmoid function
The data scientist uses the activation function to take a subjective decision based on various
problem statements and forms the desired outputs. Activation function may differ (e.g., Sign,
Step, and Sigmoid) in perceptron models by checking whether the learning process is slow or has
vanishing or exploding gradients.

How does Perceptron work?

In Machine Learning, Perceptron is considered as a single-layer neural network that consists of

four main parameters named input values (Input nodes), weights and Bias, net sum, and an
activation function. The perceptron model begins with the multiplication of all input values and
their weights, then adds these values together to create the weighted sum. Then this weighted
sum is applied to the activation function 'f' to obtain the desired output. This activation function
is also known as the step function and is represented by 'f'.

This step function or Activation function plays a vital role in ensuring that output is mapped
between required values (0,1) or (-1,1). It is important to note that the weight of input is
indicative of the strength of a node. Similarly, an input's bias value gives the ability to shift the
activation function curve up or down.

Perceptron model works in two important steps as follows:

Step-1

In the first step first, multiply all input values with corresponding weight values and then add
them to determine the weighted sum. Mathematically, we can calculate the weighted sum as
follows:

∑wi*xi = x1*w1 + x2*w2 +…wn*xn

Add a special term called bias 'b' to this weighted sum to improve the model's performance.

∑wi*xi + b

Step-2

In the second step, an activation function is applied with the above-mentioned weighted sum,
which gives us output either in binary form or a continuous value as follows:

Y = f(∑wi*xi + b)

Types of Perceptron Models

Based on the layers, Perceptron models are divided into two types. These are as follows:

1. Single-layer Perceptron Model

2. Multi-layer Perceptron model

Single Layer Perceptron Model:

This is one of the easiest Artificial neural networks (ANN) types. A single-layered perceptron
model consists feed-forward network and also includes a threshold transfer function inside the
model. The main objective of the single-layer perceptron model is to analyze the linearly
separable objects with binary outcomes.

In a single layer perceptron model, its algorithms do not contain recorded data, so it begins with
inconstantly allocated input for weight parameters. Further, it sums up all inputs (weight). After
adding all inputs, if the total sum of all inputs is more than a pre-determined value, the model
gets activated and shows the output value as +1.

If the outcome is same as pre-determined or threshold value, then the performance of this model
is stated as satisfied, and weight demand does not change. However, this model consists of a few
discrepancies triggered when multiple weight inputs values are fed into the model. Hence, to find
desired output and minimize errors, some changes should be necessary for the weights input.

"Single-layer perceptron can learn only linearly separable patterns."

Multi-Layered Perceptron Model:

Like a single-layer perceptron model, a multi-layer perceptron model also has the same model
structure but has a greater number of hidden layers.

The multi-layer perceptron model is also known as the Backpropagation algorithm, which
executes in two stages as follows:

o Forward Stage: Activation functions start from the input layer in the forward stage and
terminate on the output layer.
o Backward Stage: In the backward stage, weight and bias values are modified as per the
model's requirement. In this stage, the error between actual output and demanded
originated backward on the output layer and ended on the input layer.

Hence, a multi-layered perceptron model has considered as multiple artificial neural networks
having various layers in which activation function does not remain linear, similar to a single
layer perceptron model. Instead of linear, activation function can be executed as sigmoid, TanH,
ReLU, etc., for deployment.

A multi-layer perceptron model has greater processing power and can process linear and non-
linear patterns. Further, it can also implement logic gates such as AND, OR, XOR, NAND,
NOT, XNOR, NOR.

Advantages of Multi-Layer Perceptron:

o A multi-layered perceptron model can be used to solve complex non-linear problems.

o It works well with both small and large input data.
o It helps us to obtain quick predictions after the training.
o It helps to obtain the same accuracy ratio with large as well as small data.

Disadvantages of Multi-Layer Perceptron:

o In Multi-layer perceptron, computations are difficult and time-consuming.

o In multi-layer Perceptron, it is difficult to predict how much the dependent variable
affects each independent variable.
 o The model functioning depends on the quality of the training.

Perceptron Function

Perceptron function ''f(x)'' can be achieved as output by multiplying the input 'x' with the learned
weight coefficient 'w'.

Mathematically, we can express it as follows:

f(x)=1; if w.x+b>0

otherwise, f(x)=0

o 'w' represents real-valued weights vector

o 'b' represents the bias
o 'x' represents a vector of input x values.

Characteristics of Perceptron

The perceptron model has the following characteristics.

1. Perceptron is a machine learning algorithm for supervised learning of binary classifiers.

2. In Perceptron, the weight coefficient is automatically learned.
3. Initially, weights are multiplied with input features, and the decision is made whether the
neuron is fired or not.
4. The activation function applies a step rule to check whether the weight function is greater
than zero.
5. The linear decision boundary is drawn, enabling the distinction between the two linearly
separable classes +1 and -1.
6. If the added sum of all input values is more than the threshold value, it must have an
output signal; otherwise, no output will be shown.

Limitations of Perceptron Model

A perceptron model has limitations as follows:

o The output of a perceptron can only be a binary number (0 or 1) due to the hard limit
transfer function.
 o Perceptron can only be used to classify the linearly separable sets of input vectors. If
input vectors are non-linear, it is not easy to classify them properly.

Future of Perceptron

The future of the Perceptron model is much bright and significant as it helps to interpret data by
building intuitive patterns and applying them in the future. Machine learning is a rapidly growing
technology of Artificial Intelligence that is continuously evolving and in the developing phase;
hence the future of perceptron technology will continue to support and facilitate analytical
behavior in machines that will, in turn, add to the efficiency of computers.

The perceptron model is continuously becoming more advanced and working efficiently on
complex problems with the help of artificial neurons.

NEURAL NETWORKS
What are neural networks?

Neural networks, also known as artificial neural networks (ANNs) or simulated neural networks
(SNNs), are a subset of machine learning and are at the heart of deep learning algorithms. Their
name and structure are inspired by the human brain, mimicking the way that biological neurons
signal to one another.

Artificial neural networks (ANNs) are comprised of a node layers, containing an input layer, one
or more hidden layers, and an output layer. Each node, or artificial neuron, connects to another
and has an associated weight and threshold. If the output of any individual node is above the
specified threshold value, that node is activated, sending data to the next layer of the network.
Otherwise, no data is passed along to the next layer of the network.
Neural networks rely on training data to learn and improve their accuracy over time. However,
once these learning algorithms are fine-tuned for accuracy, they are powerful tools in computer
science and artificial intelligence, allowing us to classify and cluster data at a high velocity.
Tasks in speech recognition or image recognition can take minutes versus hours when compared
to the manual identification by human experts. One of the most well-known neural networks is
Google’s search algorithm.

Think of each individual node as its own linear regression model, composed of input data,
weights, a bias (or threshold), and an output. The formula would look something like this:

∑wixi + bias = w1x1 + w2x2 + w3x3 + bias

output = f(x) = 1 if ∑w1x1 + b>= 0; 0 if ∑w1x1 + b < 0

Once an input layer is determined, weights are assigned. These weights help determine the
importance of any given variable, with larger ones contributing more significantly to the output
compared to other inputs. All inputs are then multiplied by their respective weights and then
summed. Afterward, the output is passed through an activation function, which determines the
output. If that output exceeds a given threshold, it “fires” (or activates) the node, passing data to
the next layer in the network. This results in the output of one node becoming in the input of the
next node. This process of passing data from one layer to the next layer defines this neural
network as a feedforward network.

Let’s break down what one single node might look like using binary values. We can apply this
concept to a more tangible example, like whether you should go surfing (Yes: 1, No: 0). The
decision to go or not to go is our predicted outcome, or y-hat. Let’s assume that there are three
factors influencing your decision-making:

1. Are the waves good? (Yes: 1, No: 0)

2. Is the line-up empty? (Yes: 1, No: 0)
3. Has there been a recent shark attack? (Yes: 0, No: 1)

Then, let’s assume the following, giving us the following inputs:

 X1 = 1, since the waves are pumping

 X2 = 0, since the crowds are out
 X3 = 1, since there hasn’t been a recent shark attack

Now, we need to assign some weights to determine importance. Larger weights signify that
particular variables are of greater importance to the decision or outcome.

 W1 = 5, since large swells don’t come around often

 W2 = 2, since you’re used to the crowds
 W3 = 4, since you have a fear of sharks

Finally, we’ll also assume a threshold value of 3, which would translate to a bias value of –3.
With all the various inputs, we can start to plug in values into the formula to get the desired
output.

Y-hat = (1*5) + (0*2) + (1*4) – 3 = 6

If we use the activation function from the beginning of this section, we can determine that the
output of this node would be 1, since 6 is greater than 0. In this instance, you would go surfing;
but if we adjust the weights or the threshold, we can achieve different outcomes from the model.
When we observe one decision, like in the above example, we can see how a neural network
could make increasingly complex decisions depending on the output of previous decisions or
layers.

In the example above, we used perceptrons to illustrate some of the mathematics at play here, but
neural networks leverage sigmoid neurons, which are distinguished by having values between 0
and 1. Since neural networks behave similarly to decision trees, cascading data from one node to
another, having x values between 0 and 1 will reduce the impact of any given change of a single
variable on the output of any given node, and subsequently, the output of the neural network.

As we start to think about more practical use cases for neural networks, like image recognition or
classification, we’ll leverage supervised learning, or labeled datasets, to train the algorithm. As
we train the model, we’ll want to evaluate its accuracy using a cost (or loss) function. This is also
commonly referred to as the mean squared error (MSE). In the equation below,

 i represents the index of the sample,

 y-hat is the predicted outcome,
 y is the actual value, and
 m is the number of samples.

𝐶𝑜𝑠𝑡 𝐹𝑢𝑛𝑐𝑡𝑖𝑜𝑛= 𝑀𝑆𝐸=1/2𝑚 ∑129_(𝑖=1)^𝑚▒(𝑦 ̂^((𝑖) )−𝑦^((𝑖) ) )^2

Ultimately, the goal is to minimize our cost function to ensure correctness of fit for any given
observation. As the model adjusts its weights and bias, it uses the cost function and
reinforcement learning to reach the point of convergence, or the local minimum. The process in
which the algorithm adjusts its weights is through gradient descent, allowing the model to
determine the direction to take to reduce errors (or minimize the cost function). With each
training example, the parameters of the model adjust to gradually converge at the minimum.
See this IBM Developer article for a deeper explanation of the quantitative concepts involved in
neural networks.

Most deep neural networks are feedforward, meaning they flow in one direction only, from input
to output. However, you can also train your model through backpropagation; that is, move in the
opposite direction from output to input. Backpropagation allows us to calculate and attribute the
error associated with each neuron, allowing us to adjust and fit the parameters of the model(s)
appropriately.

Neural networks can be classified into different types, which are used for different purposes.
While this isn’t a comprehensive list of types, the below would be representative of the most
common types of neural networks that you’ll come across for its common use cases:

The perceptron is the oldest neural network, created by Frank Rosenblatt in 1958. It has a single
neuron and is the simplest form of a neural network:
Feedforward neural networks, or multi-layer perceptrons (MLPs), are what we’ve primarily been
focusing on within this article. They are comprised of an input layer, a hidden layer or layers,
and an output layer. While these neural networks are also commonly referred to as MLPs, it’s
important to note that they are actually comprised of sigmoid neurons, not perceptrons, as most
real-world problems are nonlinear. Data usually is fed into these models to train them, and they
are the foundation for computer vision, natural language processing, and other neural networks.

Convolutional neural networks (CNNs) are similar to feedforward networks, but they’re usually
utilized for image recognition, pattern recognition, and/or computer vision. These networks
harness principles from linear algebra, particularly matrix multiplication, to identify patterns
within an image.

Recurrent neural networks (RNNs) are identified by their feedback loops. These learning
algorithms are primarily leveraged when using time-series data to make predictions about future
outcomes, such as stock market predictions or sales forecasting.

Deep Learning and neural networks tend to be used interchangeably in conversation, which can
be confusing. As a result, it’s worth noting that the “deep” in deep learning is just referring to the
depth of layers in a neural network. A neural network that consists of more than three layers—
which would be inclusive of the inputs and the output—can be considered a deep learning
algorithm. A neural network that only has two or three layers is just a basic neural network.

To learn more about the differences between neural networks and other forms of artificial
intelligence, like machine learning, please read the blog post “AI vs. Machine Learning vs. Deep
Learning vs. Neural Networks: What’s the Difference?”

The history of neural networks is longer than most people think. While the idea of “a machine
that thinks” can be traced to the Ancient Greeks, we’ll focus on the key events that led to the
evolution of thinking around neural networks, which has ebbed and flowed in popularity over the
years:
1943: Warren S. McCulloch and Walter Pitts published “A logical calculus of the ideas
immanent in nervous activity (PDF, 1 MB) (link resides outside IBM)” This research sought to
understand how the human brain could produce complex patterns through connected brain cells,
or neurons. One of the main ideas that came out of this work was the comparison of neurons with
a binary threshold to Boolean logic (i.e., 0/1 or true/false statements).

1958: Frank Rosenblatt is credited with the development of the perceptron, documented in his
research, “The Perceptron: A Probabilistic Model for Information Storage and Organization in
the Brain” (PDF, 1.6 MB) (link resides outside IBM). He takes McCulloch and Pitt’s work a step
further by introducing weights to the equation. Leveraging an IBM 704, Rosenblatt was able to
get a computer to learn how to distinguish cards marked on the left vs. cards marked on the right.

1974: While numerous researchers contributed to the idea of backpropagation, Paul Werbos was
the first person in the US to note its application within neural networks within his PhD
thesis (PDF, 8.1 MB) (link resides outside IBM).

1989: Yann LeCun published a paper (PDF, 5.7 MB) (link resides outside IBM) illustrating how
the use of constraints in backpropagation and its integration into the neural network architecture
can be used to train algorithms. This research successfully leveraged a neural network to
recognize hand-written zip code digits provided by the U.S. Postal Service.

For decades now, IBM has been a pioneer in the development of AI technologies and neural
networks, highlighted by the development and evolution of IBM Watson. Watson is now a
trusted solution for enterprises looking to apply advanced natural language processing and deep
learning techniques to their systems using a proven tiered approach to AI adoption and
implementation.

Watson uses the Apache Unstructured Information Management Architecture (UIMA)

framework and IBM’s DeepQA software to make powerful deep learning capabilities available
to applications. Utilizing tools like IBM Watson Studio, your enterprise can seamlessly bring
open source AI projects into production while deploying and running models on any cloud.

DECISION TREES:

o Decision Tree is a Supervised learning technique that can be used for both

classification and Regression problems, but mostly it is preferred for solving

Classification problems. It is a tree-structured classifier, where internal nodes represent

the features of a dataset, branches represent the decision rules and each leaf node

represents the outcome.

o In a Decision tree, there are two nodes, which are the Decision Node and Leaf

Node. Decision nodes are used to make any decision and have multiple branches,

whereas Leaf nodes are the output of those decisions and do not contain any further

branches.

o The decisions or the test are performed on the basis of features of the given dataset.

o It is a graphical representation for getting all the possible solutions to a

problem/decision based on given conditions.

o It is called a decision tree because, similar to a tree, it starts with the root node, which

expands on further branches and constructs a tree-like structure.

o In order to build a tree, we use the CART algorithm, which stands for Classification

and Regression Tree algorithm.

o A decision tree simply asks a question, and based on the answer (Yes/No), it further split

the tree into subtrees.

o Below diagram explains the general structure of a decision tree:

Why use Decision Trees?

There are various algorithms in Machine learning, so choosing the best algorithm for the given
dataset and problem is the main point to remember while creating a machine learning model.
Below are the two reasons for using the Decision tree:

o Decision Trees usually mimic human thinking ability while making a decision, so it is

easy to understand.

o The logic behind the decision tree can be easily understood because it shows a tree-like

structure.

Decision Tree Terminologies

How does the Decision Tree algorithm Work?

In a decision tree, for predicting the class of the given dataset, the algorithm starts from the root
node of the tree. This algorithm compares the values of root attribute with the record (real
dataset) attribute and, based on the comparison, follows the branch and jumps to the next node.
For the next node, the algorithm again compares the attribute value with the other sub-nodes and
move further. It continues the process until it reaches the leaf node of the tree. The complete
process can be better understood using the below algorithm:

o Step-1: Begin the tree with the root node, says S, which contains the complete dataset.

o Step-2: Find the best attribute in the dataset using Attribute Selection Measure (ASM).

o Step-3: Divide the S into subsets that contains possible values for the best attributes.

o Step-4: Generate the decision tree node, which contains the best attribute.

o Step-5: Recursively make new decision trees using the subsets of the dataset created in

step -3. Continue this process until a stage is reached where you cannot further classify

the nodes and called the final node as a leaf node.

Example: Suppose there is a candidate who has a job offer and wants to decide whether he
should accept the offer or Not. So, to solve this problem, the decision tree starts with the root
node (Salary attribute by ASM). The root node splits further into the next decision node (distance
from the office) and one leaf node based on the corresponding labels. The next decision node
further gets split into one decision node (Cab facility) and one leaf node. Finally, the decision
node splits into two leaf nodes (Accepted offers and Declined offer). Consider the below
diagram:
 Attribute Selection Measures

While implementing a Decision tree, the main issue arises that how to select the best attribute for
the root node and for sub-nodes. So, to solve such problems there is a technique which is called
as Attribute selection measure or ASM. By this measurement, we can easily select the best
attribute for the nodes of the tree. There are two popular techniques for ASM, which are:

o Information Gain

o Gini Index

1. Information Gain:

o Information gain is the measurement of changes in entropy after the segmentation of a

dataset based on an attribute.

o It calculates how much information a feature provides us about a class.

o According to the value of information gain, we split the node and build the decision tree.

o A decision tree algorithm always tries to maximize the value of information gain, and a

node/attribute having the highest information gain is split first. It can be calculated using

the below formula:

1. Information Gain= Entropy(S)- [(Weighted Avg) *Entropy(each feature)

Entropy: Entropy is a metric to measure the impurity in a given attribute. It specifies

randomness in data. Entropy can be calculated as:

Entropy(s)= -P(yes)log2 P(yes)- P(no) log2 P(no)

Where,

o S= Total number of samples

o P(yes)= probability of yes

 o P(no)= probability of no

2. Gini Index:

o Gini index is a measure of impurity or purity used while creating a decision tree in the

CART(Classification and Regression Tree) algorithm.

o An attribute with the low Gini index should be preferred as compared to the high Gini

index.

o It only creates binary splits, and the CART algorithm uses the Gini index to create binary

splits.

o Gini index can be calculated using the below formula:

Gini Index= 1- ∑jPj2

Pruning: Getting an Optimal Decision tree

Pruning is a process of deleting the unnecessary nodes from a tree in order to get the optimal
decision tree.

A too-large tree increases the risk of overfitting, and a small tree may not capture all the
important features of the dataset. Therefore, a technique that decreases the size of the learning
tree without reducing accuracy is known as Pruning. There are mainly two types of
tree pruning technology used:

o Cost Complexity Pruning

o Reduced Error Pruning.

Advantages of the Decision Tree

o It is simple to understand as it follows the same process which a human follow while

making any decision in real-life.

o It can be very useful for solving decision-related problems.

 o It helps to think about all the possible outcomes for a problem.

o There is less requirement of data cleaning compared to other algorithms.

Disadvantages of the Decision Tree

o The decision tree contains lots of layers, which makes it complex.

o It may have an overfitting issue, which can be resolved using the Random Forest

algorithm.

o For more class labels, the computational complexity of the decision tree may increase.

Python Implementation of Decision Tree

Now we will implement the Decision tree using Python. For this, we will use the dataset
"user_data.csv," which we have used in previous classification models. By using the same
dataset, we can compare the Decision tree classifier with other classification models such
as KNN SVM, LogisticRegression, etc.

Steps will also remain the same, which are given below:

o Data Pre-processing step

o Fitting a Decision-Tree algorithm to the Training set

o Predicting the test result

o Test accuracy of the result(Creation of Confusion matrix)

o Visualizing the test set result.

SUPPORT VECTOR MACHINE:

Support Vector Machine Algorithm

Support Vector Machine or SVM is one of the most popular Supervised Learning algorithms,
which is used for Classification as well as Regression problems. However, primarily, it is used
for Classification problems in Machine Learning.
The goal of the SVM algorithm is to create the best line or decision boundary that can segregate
n-dimensional space into classes so that we can easily put the new data point in the correct
category in the future. This best decision boundary is called a hyperplane.

SVM chooses the extreme points/vectors that help in creating the hyperplane. These extreme
cases are called as support vectors, and hence algorithm is termed as Support Vector Machine.
Consider the below diagram in which there are two different categories that are classified using a
decision boundary or hyperplane:

Example: SVM can be understood with the example that we have used in the KNN classifier.
Suppose we see a strange cat that also has some features of dogs, so if we want a model that can
accurately identify whether it is a cat or dog, so such a model can be created by using the SVM
algorithm. We will first train our model with lots of images of cats and dogs so that it can learn
about different features of cats and dogs, and then we test it with this strange creature. So as
support vector creates a decision boundary between these two data (cat and dog) and choose
extreme cases (support vectors), it will see the extreme case of cat and dog. On the basis of the
support vectors, it will classify it as a cat. Consider the below diagram:
SVM algorithm can be used for Face detection, image classification, text categorization, etc.

Types of SVM

SVM can be of two types:

o Linear SVM: Linear SVM is used for linearly separable data, which means if a dataset

can be classified into two classes by using a single straight line, then such data is termed

as linearly separable data, and classifier is used called as Linear SVM classifier.

o Non-linear SVM: Non-Linear SVM is used for non-linearly separated data, which means

if a dataset cannot be classified by using a straight line, then such data is termed as non-

linear data and classifier used is called as Non-linear SVM classifier.

Hyperplane and Support Vectors in the SVM algorithm:

Hyperplane: There can be multiple lines/decision boundaries to segregate the classes in n-

dimensional space, but we need to find out the best decision boundary that helps to classify the
data points. This best boundary is known as the hyperplane of SVM.

The dimensions of the hyperplane depend on the features present in the dataset, which means if
there are 2 features (as shown in image), then hyperplane will be a straight line. And if there are
3 features, then hyperplane will be a 2-dimension plane.
We always create a hyperplane that has a maximum margin, which means the maximum distance
between the data points.

Support Vectors:

The data points or vectors that are the closest to the hyperplane and which affect the position of
the hyperplane are termed as Support Vector. Since these vectors support the hyperplane, hence
called a Support vector.

How does SVM works?

Linear SVM:

The working of the SVM algorithm can be understood by using an example. Suppose we have a
dataset that has two tags (green and blue), and the dataset has two features x1 and x2. We want a
classifier that can classify the pair(x1, x2) of coordinates in either green or blue. Consider the
below image:

So as it is 2-d space so by just using a straight line, we can easily separate these two classes. But
there can be multiple lines that can separate these classes. Consider the below image:
Hence, the SVM algorithm helps to find the best line or decision boundary; this best boundary or
region is called as a hyperplane. SVM algorithm finds the closest point of the lines from both
the classes. These points are called support vectors. The distance between the vectors and the
hyperplane is called as margin. And the goal of SVM is to maximize this margin.
The hyperplane with maximum margin is called the optimal hyperplane.

Non-Linear SVM:
If data is linearly arranged, then we can separate it by using a straight line, but for non-linear
data, we cannot draw a single straight line. Consider the below image:

So to separate these data points, we need to add one more dimension. For linear data, we have
used two dimensions x and y, so for non-linear data, we will add a third dimension z. It can be
calculated as:

z=x2 +y2

By adding the third dimension, the sample space will become as below image:

So now, SVM will divide the datasets into classes in the following way. Consider the below
image:
Since we are in 3-d Space, hence it is looking like a plane parallel to the x-axis. If we convert it
in 2d space with z=1, then it will become as:

Hence we get a circumference of radius 1 in case of non-linear data.

Probabilistic Models

• Probabilistic models in machine learning is the use of codes of statistics to data

examination.
• It delivers a modelling framework for accepting the learning process.
• Naïve Bayes algorithm is one of the best known algorithm based on probabilistic models.
Naïve Bayes Algorithm :

• Naïve Bayes algorithm is a supervised learning algorithm, which is based on Bayes

theorem and used for solving classification problems.
• It is a probabilistic classifier, which means it predicts on the basis of the probability of an
object.
• Naïve Bayes Classifier is one of the simple and most effective Classification algorithms
which helps in building the fast machine learning models that can make quick
predictions.
• Some popular examples of Naïve Bayes Algorithm are spam filtration, Sentimental
analysis, and classifying articles.

The Naïve Bayes algorithm is comprised of two words Naïve and Bayes,

• Naïve: It is called Naïve because it assumes that the occurrence of a certain feature is
independent of the occurrence of other features.
• Bayes: It is called Bayes because it depends on the principle of Bayes' Theorem

Bayes' Theorem:

• Bayes' theorem is used to determine the probability of a hypothesis with prior knowledge.
It depends on the conditional probability.
• The formula for Bayes' theorem is given as:
• Where,
• P(A|B) is Posterior probability: Probability of hypothesis A on the observed event B.
• P(B|A) is Likelihood probability: Probability of the evidence given that the probability
of a hypothesis is true.
• P(A) is Prior Probability: Probability of hypothesis before observing the evidence.
• P(B) is Marginal Probability: Probability of Evidence.

Types of Naïve Bayes Model:

• Gaussian: The Gaussian model assumes that features follow a normal distribution. This
means if predictors take continuous values instead of discrete, then the model assumes
that these values are sampled from the Gaussian distribution.
• Multinomial: The Multinomial Naïve Bayes classifier is used when the data is
multinomial distributed. It is primarily used for document classification problems, it
means a particular document belongs to which category such as Sports, Politics,
education, etc.
The classifier uses the frequency of words for the predictors.
• Bernoulli: The Bernoulli classifier works similar to the Multinomial classifier, but the
predictor variables are the independent Booleans variables. Such as if a particular word is
present or not in a document. This model is also famous for document classification
tasks.

Advantages of Naïve Bayes Classifier:

 • Naïve Bayes is one of the fast and easy ML algorithms to predict a class of datasets.
• It can be used for Binary as well as Multi-class Classifications.
• It is the most popular choice for text classification problems.

Disadvantages of Naïve Bayes Classifier:

• Naive Bayes assumes that all features are independent or unrelated, so it cannot learn the
relationship between features.

Applications of Naïve Bayes Classifier:

• It is used for Credit Scoring.

• It is used in medical data classification.
• It is used in Text classification such as Spam filtering and Sentiment analysis.

Unsupervised Learning

 Unsupervised learning is different from the Supervised learning technique; as its name
suggests, there is no need for supervision.
 It means, in unsupervised machine learning, the machine is trained using the unlabeled
dataset, and the machine predicts the output without any supervision.
 The main aim of the unsupervised learning algorithm is to group or categories the
unsorted dataset according to the similarities, patterns, and differences.
 Machines are instructed to find the hidden patterns from the input dataset.
 Example: Suppose the unsupervised learning algorithm is given an input dataset
containing images of different types of cats and dogs. The algorithm is never trained
upon the given dataset, which means it does not have any idea about the features of the
dataset. The task of the unsupervised learning algorithm is to identify the image features
on their own. Unsupervised learning algorithm will perform this task by clustering the
image dataset into the groups according to similarities between images.

Why use Unsupervised Learning?

• Unsupervised learning is helpful for finding useful insights from the data.
• It is much similar as a human learns to think by their own experiences, which makes it
closer to the real AI.
• It works on unlabeled and uncategorized data which make this more important.
• In real-world, we do not always have input data with the corresponding output so to solve
such cases, we need unsupervised learning.

Working of Unsupervised Learning

 • Here, we have taken an unlabeled input data, which means it is not categorized and
corresponding outputs are also not given. Now, this unlabeled input data is fed to the
machine learning model in order to train it. Firstly, it will interpret the raw data to find
the hidden patterns from the data and then will apply suitable algorithms such as k-means
clustering, Decision tree, etc.
• Once it applies the suitable algorithm, the algorithm divides the data objects into groups
according to the similarities and difference between the objects.

Types of Unsupervised Learning Algorithm:

• Clustering: Clustering is a method of grouping the objects into clusters such that objects
with most similarities remains into a group and has less or no similarities with the objects
of another group. Cluster analysis finds the commonalities between the data objects and
categorizes them as per the presence and absence of those commonalities.
• Association: An association rule is an unsupervised learning method which is used for
finding the relationships between variables in the large database. It determines the set of
items that occurs together in the dataset. Association rule makes marketing strategy more
effective. Such as people who buy X item (suppose a bread) are also tend to purchase Y
(Butter/Jam) item. A typical example of Association rule is Market Basket Analysis.

Unsupervised Learning
K-means clustering Neural Networks

KNN (k-nearest neighbors) Principle Component Analysis

Hierarchal clustering Independent Component Analysis

Anomaly detection Singular value decomposition

Advantages of Unsupervised Learning

• Unsupervised learning is used for more complex tasks.

• Unsupervised learning is preferable.

Disadvantages of Unsupervised Learning

• It does not have corresponding output.

• It has less accurate as input data is not labeled, and algorithms do not know the exact
output in advance.

Featurization

• Featurization is the process to convert varied forms of data to numerical data which
can be used for basic ML algorithms. Data can be text data, images, videos, graphs,
various database tables, time-series, categorical features, etc.

Why Featurization?

• The machine learning model cannot work with row text data directly. In the end, machine
learning models work with numerical (categorical, real,…) features. So it is import to
change some type of data into numerical vector so that we can leverage the whole power
of linear algebra (making the decision boundary between data points)and statistics tools
with other types of data also.

Techniques to featurize text data!

• Bag of Words (BoW)

• Tfidf Vectorizer
• Weighted Word2Vec
• Tfidf Word2Vec

1. Bag of Words (BoW):

• BoW is a simple and flexible technique to change the text data into numerical vectors. It
is based upon counting the occurrence of a word in the document(sentence, review).
Tfidf Vectorizer

• “Term Frequency – Inverse Document Frequency ”. TF-IDF is a numerical statistic

which measures the importance of the word in a document. Term Frequency: Number
of time a word appears in a text document. Inverse Document Frequency: Measure the
word is a rare word or common word in a document.

Weighted Word2Vec

• Word2vec is a technique for natural language processing published in 2013. The

word2vec algorithm uses a neural network model to learn word associations from a large
corpus of text. Once trained, such a model can detect synonymous words or suggest
additional words for a partial sentence.

Tfidf Word2Vec

• TF-IDF is a statistical measure that we can apply to terms in a document and then use
that to form a vector whereas word2vec will produce a vector for a term and then more
work may need to be done to convert that set of vectors into a singular vector or other
UNIT- III

Ranking – Recommendation System - Designing and Tuning model pipelines-

Performance measurement – Azure Machine Learning – Open-source Machine
Learning libraries – Amazon’s Machine Learning Tool Kit: Sagemaker

RANKING:

Ranking is a type of machine learning that sorts data in a relevant order.

Companies use ranking to optimize search and recommendations.

 What is a ranking model?

 How does ranking work?
 Why should I care?
 Use cases
 The fastest way to build a ranking model

Ranking is a type of supervised machine learning (ML) that uses labeled datasets
to train its data and models to classify future data to predict outcomes. Quite
simply, the goal of a ranking model is to sort data in an optimal and relevant order.
Ranking was first largely deployed within search engines. People search for a
topic, while the ranking algorithm reorders search results based on the PageRank,
and the search engine is able to display the most relevant results to its customers.
Until recently, most ranking models, and ML as whole, were limited in their scope
of use, as most companies didn’t have enough data to power these algorithms.
Better methods for data collection and more intuitive ML tools have made it
possible for nearly anyone to deploy a successful ranking model within their
business.

How does It works?

As we’ll discuss later in this blog, ranking is incredibly versatile and dependent on
the data a company has. Even so, a common framework guides the construction of
all ranking models.
Ranking models are made up of 2 main factors: queries and documents. Queries
are any input value, such as a question on Google or an interaction on an e-
commerce site. Documents are the output value or results of the query. Given the
query, and the associated documents, a function, given a list of parameters to rank
on, will score the documents to be sorted in order of relevancy.
The machine learning algorithm learning to rank takes the scores from this model,
and uses them to predict future outcomes on a new and unseen list of documents.

As an example, a search for “Mage” is done on Google Search (“Mage” is the

query). After the search, a list of associated documents matching the query will be
displayed (Mage A.I., Mage definition, Mage World of Warcraft, etc.). The
function will score each of the documents based on their relevance to the query
(Mage A.I. = 1, Mage definition = 2, Mage World of Warcraft =3, and so on). The
documents with higher scores will be ranked higher when there is a search for
Mage.
Data required for a ranking model consists of documents from a query, user
profiles, user behaviors, search history, clicks, etc.

Ranking ensures that the most relevant results appear first on a customer’s search,
maximizing the chances they will find something of interest, and minimizing the
chances of churn. With so many options for organic web search, the need to stay
competitive has never been greater. According to a Google study, 61% of users
said if they didn’t find what they were looking for right away, they would quickly
move on to another site. Depending on available data, companies can use ranking
within their web pages and apps to serve their customers the most relevant results
as soon as they enter.

The most successful companies are using ranking within their software to improve
the user experience. Ranking has allowed these companies to create customized
feeds for each user based on their past search and buying history. Ranking carries
many use cases across industries, nearly anyone with data can and should be using
ranking in some capacity to optimize their business. A few use cases are:
 1. Search results
2. Targeted ads
3. Recommendations

Here are a few companies who have used ranking to maximize user engagement.

 Amazon
With millions of listings or documents, for every product search or query,
Amazon needed to find a way to rank its products in order to maximize the
chance of purchase. Using a combination of individual preferences, gathered
from users' search and purchasing history and a product’s popularity,
Amazon created a ranking system that would display the most relevant
products at the top of their feed. Additionally, ranking was used in
Amazon’s recommendation system, which would use users' ranked
preferences in order to predict what products a user is most likely to
purchase in the future.
 Netflix
Similar to Amazon, Netflix uses ranking to fuel their recommendation
system. The recommendation system predicts what content a user is most
likely to watch and displays the most relevant content at the top of the home
page. Netflix uses a few different features to rank and recommend content;
such as: watch history, search history, and general popularity. They also use
ranking to fuel their collaborative filtering.
 TikTok
TikTok’s standout feature is the For You page which is built on a ranking
system. This feature has allowed TikTok to customize each home page to be
reflective of the preferences and interests of its user. TikTok uses similar
metrics to Netflix to rank its content: watch history, re-watch rate, and
engagement. Similar to Netflix, TikTok’s ranking system also aids in
collaborative filtering.

- Starbucks
Starbucks found great success with their mobile app, which is one of the most
downloaded apps on the App Store. The app allows Starbucks to create a custom
user experience for their customers even when they’re not within a physical coffee
shop. The app uses ranking to recommend the most relevant products to users.
Taking into account order history, new products and general popularity of other
products, Starbucks is able to keep customers' favorite orders at the top of the
recommended search while introducing them to new products that they are most
likely to enjoy.
Fastest way to build ranking model:

For the companies listed above, entire teams of data scientists and AI engineers
were built to create and maintain the ranking systems in place. The cost to build
these teams is impractical for most businesses. Recently, there have been great
tools emerging which allow for the easy building and deployment of ranking
models–this with little to no programming experience.
Mage allows for the building and deployment of a ranking model with no ML
programming knowledge. To use Mage, a database containing a list of queries and
documents is first uploaded. Queries could contain a list of clothes or menu items,
their documents could be the number of engagement (clicks and purchases) each
received. The greater the quality and quantity of data uploaded, the better that
Mage is able to produce ranking predictions.
Once the data is uploaded, users will be given the option to transform their datasets
by removing and adding columns, applying transformer actions: split and filter
data, group values, aggregate data, and identifying what columns they would like
to rank. Mage will then produce a ranking model which can be deployed into your
data warehouses, downloaded to a CSV file, or saved directly to a Mage dataset.

PIPELINES:
What is a Machine Learning Pipeline?

Machine learning pipeline helps to automate ML Workflow and enable the

sequence data to be transformed and correlated together in a model to analyzed and
achieve outputs. ML pipeline is constructed to allow the flow of data from raw data
format to some valuable information. It provides a mechanism to build a Multi-ML
parallel pipeline system to examine different ML methods' outcomes. The
Objective of it is to exercise control over the ML model. A well-planned pipeline
helps to makes the implementation more flexible. It is like having an overview of a
code to pick the fault and replace them with the correct code.

With it enterprises can:

 Facilitate Real-Time Business Decision making.

 Improve the performance of predictive maintenance.
 Fraud Detection.
 Building Recommendation Systems.

Machine Learning Pipeline Architecture

A pipeline consists of several stages. Each stage of a pipeline is fed with the data
processed from its preceding stage, i.e., the output of a processing unit supplied as
an input to the next step. It consists of four main stages as Pre-processing,
Learning, Evaluation, and Prediction.

Pre-processing

Data preprocessing is a Data Mining technique that involves transferring raw data
into an understandable format. Real-world data is usually incomplete, inconsistent,
and lacks certain behaviors or trends, most likely to contain many inaccuracies.
The process of getting usable data for a Machine Learning algorithm follows steps
such as Feature Extraction and Scaling, Feature Selection, Dimensionality
reduction, and sampling. The product of Data Pre-processing is the final dataset
used for training the model and testing purposes.

Learning

A learning algorithm is used to process understandable data to extract patterns

appropriate for application in a new situation. In particular, the aim is to utilize a
system for a specific input-output transformation task. For this, choose the best-
performing model from a set of models produced by different hyperparameter
settings, metrics, and cross-validation techniques.

Evaluation

To Evaluate the Machine Learning model's performance, fit a model to the training
data, and predict the labels of the test set. Further, count the number of wrong
predictions on the test dataset to compute the model’s prediction accuracy.

Prediction

The model's performance to determine the outcomes of the test data set was not
used for any training or cross-validation activities.
What are the benefits?

There are many benefits some of them are:

  Flexibility - Computation units are easy to replace. For better
implementation, it is possible to rework that part without changing the
rest of the system.
 Extensibility - When the system is partitioned into pieces, it is easy to
create new functionality.
 Scalability - Each part of the computation is presented via a standard
interface. If any part has an issue, it is possible to scale that component
separately.

Why it is Matters?
As machines begin to learn through algorithms, it will help companies interpret
uncovered patterns to make better decisions.\

Timely Analysis And Assessment

It helps to understand customer behavior by streamlining Customer Acquisition

and Digital Marketing strategies.

Real-Time Predictions

ML algorithms are super fast. As a consequence that Large Data Processing takes
place rapidly. This, in turn, helps in making Real-Time predictions very beneficial
for businesses.

Transforming Industries

It has already commenced transforming industries with its expertise to provide

valuable insights in Real-Time.
PERFORMANCE MEASUREMENT:

There are various metrics which we can use to evaluate the performance of ML
algorithms, classification as well as regression algorithms. We must carefully
choose the metrics for evaluating ML performance because −
 How the performance of ML algorithms is measured and compared will be
dependent entirely on the metric you choose.
 How you weight the importance of various characteristics in the result will
be influenced completely by the metric you choose.
Performance Metrics for Classification Problems
We have discussed classification and its algorithms in the previous chapters. Here,
we are going to discuss various performance metrics that can be used to evaluate
predictions for classification problems.
Confusion Matrix
It is the easiest way to measure the performance of a classification problem where
the output can be of two or more type of classes. A confusion matrix is nothing but
a table with two dimensions viz. “Actual” and “Predicted” and furthermore, both
the dimensions have “True Positives (TP)”, “True Negatives (TN)”, “False
Positives (FP)”, “False Negatives (FN)” as shown below −
Explanation of the terms associated with confusion matrix are as follows −
 True Positives (TP) − It is the case when both actual class & predicted class
of data point is 1.
 True Negatives (TN) − It is the case when both actual class & predicted
class of data point is 0.
 False Positives (FP) − It is the case when actual class of data point is 0 &
predicted class of data point is 1.
 False Negatives (FN) − It is the case when actual class of data point is 1 &
predicted class of data point is 0.
We can use confusion_matrix function of [Link] to compute Confusion
Matrix of our classification model.
Classification Accuracy
It is most common performance metric for classification algorithms. It may be
defined as the number of correct predictions made as a ratio of all predictions
made. We can easily calculate it by confusion matrix with the help of following
formula −

We can use accuracy_score function of [Link] to compute accuracy of our

classification model.
Classification Report
This report consists of the scores of Precisions, Recall, F1 and Support. They are
explained as follows −
Precision
Precision, used in document retrievals, may be defined as the number of correct
documents returned by our ML model. We can easily calculate it by confusion
matrix with the help of following formula −
Recall or Sensitivity
Recall may be defined as the number of positives returned by our ML model. We
can easily calculate it by confusion matrix with the help of following formula −
Specificity
Specificity, in contrast to recall, may be defined as the number of negatives
returned by our ML model. We can easily calculate it by confusion matrix with the
help of following formula −
Support
Support may be defined as the number of samples of the true response that lies in
each class of target values.
F1 Score
This score will give us the harmonic mean of precision and recall. Mathematically,
F1 score is the weighted average of the precision and recall. The best value of F1
would be 1 and worst would be 0. We can calculate F1 score with the help of
following formula −
𝑭𝟏 = 𝟐 ∗ (𝒑𝒓𝒆𝒄𝒊𝒔𝒊𝒐𝒏 ∗ 𝒓𝒆𝒄𝒂𝒍𝒍) / (𝒑𝒓𝒆𝒄𝒊𝒔𝒊𝒐𝒏 + 𝒓𝒆𝒄𝒂𝒍𝒍)
F1 score is having equal relative contribution of precision and recall.
We can use classification_report function of [Link] to get the
classification report of our classification model.
AUC (Area Under ROC curve)
AUC (Area Under Curve)-ROC (Receiver Operating Characteristic) is a
performance metric, based on varying threshold values, for classification problems.
As name suggests, ROC is a probability curve and AUC measure the separability.
In simple words, AUC-ROC metric will tell us about the capability of model in
distinguishing the classes. Higher the AUC, better the model.
Mathematically, it can be created by plotting TPR (True Positive Rate) i.e.
Sensitivity or recall vs FPR (False Positive Rate) i.e. 1-Specificity, at various
threshold values. Following is the graph showing ROC, AUC having TPR at y-axis
and FPR at x-axis −
We can use roc_auc_score function of [Link] to compute AUC-ROC.
LOGLOSS (Logarithmic Loss)
It is also called Logistic regression loss or cross-entropy loss. It basically defined
on probability estimates and measures the performance of a classification model
where the input is a probability value between 0 and 1. It can be understood more
clearly by differentiating it with accuracy. As we know that accuracy is the count
of predictions (predicted value = actual value) in our model whereas Log Loss is
the amount of uncertainty of our prediction based on how much it varies from the
actual label. With the help of Log Loss value, we can have more accurate view of
the performance of our model. We can use log_loss function of [Link] to
compute Log Loss.
Example
The following is a simple recipe in Python which will give us an insight about how
we can use the above explained performance metrics on binary classification
model −
from [Link] import confusion_matrix
from [Link] import accuracy_score
from [Link] import classification_report
from [Link] import roc_auc_score
from [Link] import log_loss
X_actual = [1, 1, 0, 1, 0, 0, 1, 0, 0, 0]
Y_predic = [1, 0, 1, 1, 1, 0, 1, 1, 0, 0]
results = confusion_matrix(X_actual, Y_predic)
print ('Confusion Matrix :')
print(results)
print ('Accuracy Score is',accuracy_score(X_actual, Y_predic))
print ('Classification Report : ')
print (classification_report(X_actual, Y_predic))
print('AUC-ROC:',roc_auc_score(X_actual, Y_predic))
print('LOGLOSS Value is',log_loss(X_actual, Y_predic))
Output
Confusion Matrix :
[
[3 3]
[1 3]
]
Accuracy Score is 0.6
Classification Report :
precision recall f1-score support
0 0.75 0.50 0.60 6
1 0.50 0.75 0.60 4
micro avg 0.60 0.60 0.60 10
macro avg 0.62 0.62 0.60 10
weighted avg 0.65 0.60 0.60 10
AUC-ROC: 0.625
LOGLOSS Value is 13.815750437193334

Performance Metrics for Regression Problems

We have discussed regression and its algorithms in previous chapters. Here, we are
going to discuss various performance metrics that can be used to evaluate
predictions for regression problems.
Mean Absolute Error (MAE)
It is the simplest error metric used in regression problems. It is basically the sum of
average of the absolute difference between the predicted and actual values. In
simple words, with MAE, we can get an idea of how wrong the predictions were.
MAE does not indicate the direction of the model i.e. no indication about
underperformance or overperformance of the model. The following is the formula
to calculate MAE −

Here, 𝑌=Actual Output Values

And Y^Y^= Predicted Output Values.
We can use mean_absolute_error function of [Link] to compute MAE.
Mean Square Error (MSE)
MSE is like the MAE, but the only difference is that the it squares the difference of
actual and predicted output values before summing them all instead of using the
absolute value. The difference can be noticed in the following equation −

Here, 𝑌=Actual Output Values

And Y^Y^ = Predicted Output Values.
We can use mean_squared_error function of [Link] to compute MSE.
R Squared (R2)
R Squared metric is generally used for explanatory purpose and provides an
indication of the goodness or fit of a set of predicted output values to the actual
output values. The following formula will help us understanding it −

In the above equation, numerator is MSE and the denominator is the variance in 𝑌
values.
We can use r2_score function of [Link] to compute R squared value.
Example
The following is a simple recipe in Python which will give us an insight about how
we can use the above explained performance metrics on regression model −
from [Link] import r2_score
from [Link] import mean_absolute_error
from [Link] import mean_squared_error
X_actual = [5, -1, 2, 10]
Y_predic = [3.5, -0.9, 2, 9.9]
print ('R Squared =',r2_score(X_actual, Y_predic))
print ('MAE =',mean_absolute_error(X_actual, Y_predic))
print ('MSE =',mean_squared_error(X_actual, Y_predic))
Output
R Squared = 0.9656060606060606
MAE = 0.42499999999999993
MSE = 0.5674999999999999
AZURE MACHINE LEARNING:

Azure Machine Learning is a cloud service for accelerating and managing the
machine learning project lifecycle. Machine learning professionals, data scientists,
and engineers can use it in their day-to-day workflows: Train and deploy models,
and manage MLOps.

You can create a model in Azure Machine Learning or use a model built from an
open-source platform, such as Pytorch, TensorFlow, or scikit-learn. MLOps tools
help you monitor, retrain, and redeploy models.

Azure Machine Learning is for individuals and teams implementing MLOps within
their organization to bring machine learning models into production in a secure and
auditable production environment.

Data scientists and ML engineers will find tools to accelerate and automate their
day-to-day workflows. Application developers will find tools for integrating
models into applications or services. Platform developers will find a robust set of
tools, backed by durable Azure Resource Manager APIs, for building advanced
ML tooling.

Enterprises working in the Microsoft Azure cloud will find familiar security and
role-based access control (RBAC) for infrastructure. You can set up a project to
deny access to protected data and select operations.

Collaboration for machine learning teams

Machine learning projects often require a team with varied skillsets to build and
maintain. Azure Machine Learning has tools that help enable collaboration, such
as:

 Shared notebooks, compute resources, data, and environments

 Tracking and auditability that shows who made changes and when
 Asset versioning

Tools for developers

Developers find familiar interfaces in Azure Machine Learning, such as:

  Python SDK
 Azure Resource Manager REST APIs (preview)
 CLI v2

Studio UI

The Azure Machine Learning studio is a graphical user interface for a project
workspace. In the studio, you can:

 View runs, metrics, logs, outputs, and so on.

 Author and edit notebooks and files.
 Manage common assets, such as
o Data credentials
o Compute
o Environments
 Visualize run metrics, results, and reports.
 Visualize pipelines authored through developer interfaces.
 Author AutoML jobs.

Plus, the designer has a drag-and-drop interface where you can train and deploy
models.

If you're a ML Studio (classic) user, learn about Studio (classic) deprecation and
the difference between it and Azure Machine Learning studio.

Azure Machine Learning integrates with the Azure cloud platform to add security
to ML projects.

Security integrations include:

 Azure Virtual Networks (VNets) with network security groups

 Azure Key Vault where you can save security secrets, such as access
information for storage accounts
 Azure Container Registry set up behind a VNet

Other integrations with Azure services support a machine learning project from
end-to-end. They include:

 Azure Synapse Analytics to process and stream data with Spark

 Azure Arc, where you can run Azure services in a Kubernetes environment
  Storage and database options, such as Azure SQL Database, Azure Storage
Blobs, and so on
 Azure App Service allowing you to deploy and manage ML-powered apps

Machine learning project workflow

Typically models are developed as part of a project with an objective and goals.
Projects often involve more than one person. When experimenting with data,
algorithms, and models, development is iterative.

Project lifecycle

While the project lifecycle can vary by project, it will often look like this:

A workspace organizes a project and allows for collaboration for many users all
working toward a common objective. Users in a workspace can easily share the
results of their runs from experimentation in the studio user interface or use
versioned assets for jobs like environments and storage references.

When a project is ready for operationalization, users' work can be automated in a

machine learning pipeline and triggered on a schedule or HTTPS request.

Models can be deployed to the managed inferencing solution, for both real-time
and batch deployments, abstracting away the infrastructure management typically
required for deploying models.
In Azure Machine Learning, you can run your training script in the cloud or build a
model from scratch. Customers often bring models they've built and trained in
open-source frameworks, so they can operationalize them in the cloud.

Open and interoperable

Data scientists can use models in Azure Machine Learning that they've created in
common Python frameworks, such as:

 PyTorch
 TensorFlow
 scikit-learn
 XGBoost
 LightGBM

Other languages and frameworks are supported as well, including:

 R
 .NET

Automated featurization and algorithm selection (AutoML)

In a repetitive, time-consuming process, in classical machine learning data

scientists use prior experience and intuition to select the right data featurization
and algorithm for training. Automated ML (AutoML) speeds this process and can
be used through the studio UI or Python SDK.

Hyperparameter optimization

Hyperparameter optimization, or hyperparameter tuning, can be a tedious task.

Azure Machine Learning can automate this task for arbitrary parameterized
commands with little modification to your job definition. Results are visualized in
the studio.

Multinode distributed training

Efficiency of training for deep learning and sometimes classical machine learning
training jobs can be drastically improved via multinode distributed training. Azure
Machine Learning compute clusters offer the latest GPU options.

Supported via Azure ML Kubernetes and Azure ML compute clusters:

  PyTorch
 TensorFlow
 MPI

The MPI distribution can be used for Horovod or custom multinode logic.
Additionally, Apache Spark is supported via Azure Synapse Analytics Spark
clusters (preview).

Embarrassingly parallel training

Scaling a machine learning project may require scaling embarrassingly parallel

model training. This pattern is common for scenarios like forecasting demand,
where a model may be trained for many stores.

To bring a model into production, it is deployed. Azure Machine Learning's

managed endpoints abstract the required infrastructure for both batch or real-time
(online) model scoring (inferencing).

Real-time and batch scoring (inferencing)

Batch scoring, or batch inferencing, involves invoking an endpoint with a

reference to data. The batch endpoint runs jobs asynchronously to process data in
parallel on compute clusters and store the data for further analysis.

Real-time scoring, or online inferencing, involves invoking an endpoint with one

or more model deployments and receiving a response in near-real-time via HTTPs.
Traffic can be split across multiple deployments, allowing for testing new model
versions by diverting some amount of traffic initially and increasing once
confidence in the new model is established.

OPEN SOURCE MACHINE LEARNING LIBRARIES:

1. Numpy- Numerical python

Released in 2005, NumPy is an open-source Python package for numerical

computing. It provides the following features:
 Powerful n-dimensional arrays to allow indexing, vectorization, and broadcasting
operations
 Mathematical functions, Fourier transforms, random number generators, and linear
algebra methods
 Operable on various computing platforms, including GPU and distributed
computing
 Easy to use high-level syntax with optimized Python code to allow speed and
flexibility
In the Machine Learning ecosystem, NumPy serves as a foundation for advanced
ML libraries and frameworks like Scikit-learn, Tensorflow, PyTorch, MXNet, and
more. Not only that but NumPy facilitates the numerical processes of numerous
libraries related to things such as data visualisation, data science, quantum
computing, image processing, geographic processing, signal processing,
bioinformatics, and more.

And psst...you can familiarise yourself with NumPy programming with

this NumPy cheat sheet.

2. Pandas- Python Data Analysis

Open-sourced in 2009, Pandas holds a significant place in the heart of every ML

enthusiast as it provides some very robust methods for data manipulation and data
analysis. We know, we know - you want to know what the key features of Pandas
are. Well take a look:
 Powerful DataFrame object for extensive data manipulation support
 Handling missing data
 Indexing, reshaping, slicing, subsetting, merging and joining of large datasets
 Time series data handling
 Optimised code for Python using C and Cython
Other than its wide application in academia, Pandas supports various commercial
domains, including web and business analytics, statistics, economics, finance,
neuroscience, advertising, and more. It also serves as a foundational library for
advanced Python libraries.
 Quickly familiarise yourself with various data analysis methods of this library
using this Pandas cheat sheet.

3. Matplot

Matplotlib is as old as the dinosaurs, but it's not extinct or obsolete when it comes
to data visualisation. In fact, it's one of the most advanced data visualisation
libraries for Python, and the ML community loves it. Here are some of the great
features of the Matplotlib library:
 Provides a comprehensive list of plots suitable for any use case
 The interactive plots and charts allow compelling data storytelling
 Plots and charts are highly customisable and exportable to different file formats
 Provides embeddable visualisations with various GUI applications
 A wide array of Python libraries and frameworks extend Matplotlib.
Matplotlib is another one of the gems offered by the open-source ecosystem. Here
is a link to the Matplotlib cheat sheets to serve as a quick start guide.

4. OpenCV

Released in 2000, OpenCV is an open-source commercial-scale computer vision

and machine learning library-not your standard image editing tool. It has more than
2500 highly optimised algorithms for machine learning and computer vision that
can do just about anything with images (and videos). Some of the significant
OpenCV features include:
 Detecting objects and recognising faces in pictures and videos
 Tracking camera movements and moving objects
 Advanced application for 3D objects
 Cross-platform and with support for GPUs
 Optimised for commercial real-world and real-time CV and ML applications
If you want to familiarise yourself with OpenCV programming basics quickly,
explore this OpenCV cheat sheet.

5. Scikit Learn
 Every data scientist and ML enthusiast has used scikit-learn at some point in their
AI journey. It is a comprehensive machine learning framework. Sometimes people
tend to overlook it due to the availability of more advanced Python libraries and
frameworks. Still, it is a powerful library and does an excellent job solving some
complex Machine Learning tasks. Here are a few important features scikit-learn
includes:

 Simple tool for accurate predictive data analysis

 Helps in solving complex ML problems like preprocessing, classification,
regression, clustering, dimensionality reduction, and model selection
 Numerous built-in machine learning algorithms
 Building a basic to advanced level ML model
 Built on top of familiar libraries like NumPy, SciPy, and Matplotlib
Scikit-learn provides commercial-scale ML solutions (and, of course, it's open-
source as well). For a quick overview, have a look at this scikit-learn cheat sheet.

6. Keras

Released in 2015, Keras is an advanced open-source Python deep learning API and
framework built on top of Tensorflow-another powerful ML platform. Although
similar to Tensorflow in many aspects, it is designed with a human-centric
approach to make ML and DL easy and accessible for everyone. Key elements of
Keras include:
 Everything that TensorFlow offers but simpler and easier to understand
 Running different DL iterations quickly with full deployment capabilities
 Support for large GPU clusters and TPUs, enabling industrial-scale Python
machine learning
From computer vision to natural language processing, and generative deep learning
to reinforcement learning, Keras offers wide-ranging applications for structured,
audio, graph, and timeseries data. Here’s a brief Keras cheat sheet to get you up to
speed.

7. Tensor Flow
 Developed by Google and open-sourced later, TensorFlow powers some of the
biggest state-of-the-art AI models worldwide. It's an end-to-end Machine Learning
and Deep Learning library to solve real-world challenges. Some key features
included in TensorFlow are listed below:
 Complete control over building a robust neural network and machine learning
model
 Deploy models on web, cloud, mobile, or edge devices using [Link],
TensorFlow Lite, and TFX
 Supports numerous libraries and extensions for solving complex problems
 Supports various tools for integrating Responsible AI into ML solutions
TensorFlow Deep Learning framework is used by some of the top
companies worldwide. For example, Paypal applies TensorFlow to develop deep
transfer learning and generative modelling methods to recognise complex fraud
patterns. Spotify uses TFX to improve user recommendations. And, Airbnb uses
TensorFlow to detect objects and classify images to enhance the guest experience.
Hmm, the more you know!

8. PyTorch

In 2016, PyTorch was released by Facebook as a direct competitor of TensorFlow,

gaining massive popularity among ML and DL researchers. Today, both PyTorch
and TensorFlow are ruling the ML development and deployment ecosystem. Key
capabilities of PyTorch include:

 Full support for building customised deep neural networks

 Production-ready with TorchServe
 Supports distributed computing with the [Link] backend
 Supports a wide array of tools and extensions to solve complex problems
 Supported on all major cloud platforms for scalable deployment
PyTorch is also available on Github as an open-source Python framework and, of
course, comes with an official cheat sheet.
 9. NLTK- Natural Language Toolkit

Natural Language Processing (NLP) has recently seen rapid growth with the
release of massive language models like BERT and GPT-3, making waves
worldwide. One of the fundamental Python libraries for performing NLP tasks is
NLTK. Developers interested in NLP should gain hands-on experience with this
Python library. Some key features include:

 Supports more than 50 language datasets and trained language models

 Offers text classification, stemming, tokenisation, tagging, parsing, and much more
 Serves as a wrapper for industrial-scale NLP libraries
 It is a free and open-source project that allows development on any platform like
Windows, Linux, and Mac OS X

10. SpaCy

And last, but certainly not lesay we have SpaCY. Meant for solving advanced NLP
problems, SpaCy is an industrial-scale open-source Python library for NLP. SpaCy
is written in Cython with memory management optimisation to ensure state-of-the-
art speed. Some key aspects of this Python library include:
 60+ trained NLP pipelines supporting 19 languages
 Pre-trained word embeddings
 Production-ready pipelines
 Supports custom models that written in TensorFlow and PyTorch
 SpaCy Universe offers a wide variety of Python packages, plugins, and extensions
for NLP
SpaCy API supports many NLP tasks like lemmatisation, entity recognition,
tagging, sentence recognition, tokenisation, and more.
Open-source Python libraries and frameworks have greatly democratised AI
research and development. Every day AI practitioners are coming up with bigger
and better models for solving real-world problems. AI is not just a buzzword
anymore, it has penetrated our lives much more than we can imagine, and Python
programming lies at its core.
Python programming language has significantly matured over the last two decades
and we can't wait to see where it goes next. Learning these Python libraries and
frameworks will definitely benefit all current and future Python Developers and
Data Scientists.

SAGEMAKER:

What is Amazon SageMaker?

Amazon SageMaker is a managed service in the Amazon Web Services (AWS)
public cloud. It provides the tools to build, train and deploy machine learning (ML)
models for predictive analytics applications. The platform automates the tedious
work of building a production-ready artificial intelligence (AI) pipeline.

Machine learning has a range of uses and benefits. Among them are advanced
analytics for customer data and back-end security threat detection.

Deploying ML models is challenging, even for experienced application developers.

Amazon SageMaker aims to simplify the process. It uses common algorithms and
other tools to accelerate the machine learning process.

Machine learning in AWS SageMaker

Machine learning is an iterative process. It requires workflow tools and dedicated
hardware to process data sets. In a typical scenario, a data science team builds ML
models in two steps or pipelines: training and inferencing.

Data training teaches a machine to behave in a certain way based on

recurring pattern recognition within data sets. The data is then inferenced or trained
how to respond to new data patterns. Once data scientists tune the ML model,
software development teams convert the finished model into product or service
application program interfaces (APIs).

Many companies don't have the budget to bring in specialists and maintain
resources dedicated to AI development. AWS SageMaker uses integrated tools to
automate labor-intensive manual processes and reduce human error and hardware
costs. ML modeling components are packaged in an AWS SageMaker tool set.
Software capabilities are abstracted in intuitive SageMaker templates. They
provide a framework to build, host, train and deploy ML models at scale in the
Amazon public cloud.

How does Amazon SageMaker work?

AWS SageMaker simplifies ML modeling into three steps: preparation, training
and deployment.

Prepare and build AI models

Amazon SageMaker creates a fully managed ML instance in Amazon Elastic
Compute Cloud (EC2). It supports the open source Jupyter Notebook web
application that enables developers to share live code. SageMaker runs Jupyter
computational processing notebooks.

The notebooks include drivers, packages and libraries for common deep
learning platforms and frameworks. Developers can launch a prebuilt notebook,
which AWS supplies for a variety of applications and use cases. They can then
customize it according to the data set and schema that needs to be trained.

Developers also can use custom-built algorithms written in one of the supported
ML frameworks or any code that has been packaged as a Docker container image.
SageMaker can pull data from Amazon Simple Storage Service (S3), and there is
no practical limit to the size of the data set.

To get started, a developer logs into the SageMaker console and launches a
notebook instance. SageMaker provides a variety of built-in training algorithms,
such as linear regression and image classification, or the developer can import
custom algorithms.

Train and Tune

Developers doing model training specify the location of the data in an Amazon S3
bucket and the preferred instance type. They then initiate the training process.
SageMaker Model Monitor provides continuous automatic model tuning to find the
set of parameters, or hyperparameters, to best optimizes the algorithm. During this
step, data is transformed to enable feature engineering.

Deploy and analyze

When the model is ready for deployment, the service automatically operates and
scales the cloud infrastructure. It uses a set of SageMaker instance types that
include several graphics processing unit accelerators optimized for ML workloads.

SageMaker deploys across multiple availability zones, performs health checks,

applies security patches, sets up AWS Auto Scaling and establishes secure HTTPS
endpoints to connect to an app. A developer can track and trigger alarms for
changes in production performance via Amazon CloudWatch metrics.
 UNIT- IV

Machine Learning for Predictive Data Analytics – Data to Insights to

Decisions – Data Exploration – Information based Learning –
Similarity based learning – Probability based learning – Error based
learning – Evaluation – The art of Machine learning to Predictive
Data Analytics.

MACHINE LEARNING FOR PREDICTIVE ANALYTICS

Machine Learning and Predictive Analytics approach a problem differently.

Eventually, predictive analytics is likely to merge as one application of machine
learning.
It’s similar to how the thirsty and the quenched come to the same glass of water.
Machine learning is more adaptive, newer, and has larger degrees of freedom, so it
can afford to be more flexible with its approach to a problem. Predictive analytics
has been around longer and is more procedural in its use.

There is no problem predictive analytics can solve that machine learning cannot.
But predictive analytics always has an intended audience, whereas machine
learning does not. Let’s explore.

What is predictive analytics?

Both machine learning and predictive analytics are used to make predictions on a
set of data about the future. Predictive analytics uses predictive modelling, which
can include machine learning. Predictive analytics has a very specific purpose: to
use historical data to predict the likelihood of a future outcome.

At its most basic, analytics of any sort is simply applied mathematics—sometimes

known as data science.

Who uses predictive analytics?

The audience of predictive analytics tends to be people, adding an extra level of
necessary communication and interpretability to its work. People will ask, “What
are Q2 sales going to be?” Predictive analytics answers the question with a degree
of confidence.
To Predictive Analysts, machine learning is an extension of their practice, another
tool in their toolbox, that helps them to do their job better. Using ML, predictive
analysts can:

 Provide answers, with confidence, to more complex problems.

 Offer real-time answers to questions that persist through time with ever-
changing data.
 Explore entirely new kinds of problems.

Use cases
predictive analytics is usually conducted on numerical data. Predictive analytics is
used to figure out:

 When a sensor might fail

 When to trade a stock
 The chance of success of a marketing campaign
 Employee sentiment

Through the use of machine learning, predictive analytics can expand how it
conducts its sentiment analysis to see how happy its customers and employees are.

When to use which?

Machine learning is like calculus or physics: it is a tool to be used. Predictive
analytics is a role to play that equips itself with tools to accomplish its role—
machine learning is one of those tools.

Machine learning doesn’t have to answer people’s questions. Their applications

can be created for fun, generating life-like images and seemingly real blog posts.
Predictive analytics tends to have a use case in mind, such as business financial
predictions and employee satisfaction surveys.

What are Data Insights?

“Insight is the value obtained through the use of analytics. The insights gained
through analytics are incredibly powerful, and can be used to grow your business
while identifying areas of opportunity.” – Localytics Blog

Data insights refer to the understanding of a particular business phenomenon you

are able to achieve by using machine learning and artificial intelligence
(AI) technology to analyze a dataset. For example, a machine learning model that
estimates the likelihood of a customer to churn will reveal what factors drive churn
rates, allowing decision-makers to make changes to business strategies and
processes.

One of the best ways to understand and communicate meaningful insights from
data is to use tools that help visualize a model’s outcomes and give different ways
to explore and understand your data. This translates to real business value in the
form of increased ROI on advertising efforts, more accurate loan default
predictions, and much more. The clarity of vision from data insights allows users
to make better decisions based on increased model interpretability, allowing
analysts and other users to explain model outcomes to key stakeholders.

Why are Data Insights Important?

Insights allow users of all skill levels to understand what the model is doing
“behind the scenes,” which is especially important when it comes to highly
regulated industries like banking and healthcare. If you don’t understand why your
model is drawing the conclusions it does (i.e., you don’t have any insight into the
inner workings), the model’s practical usefulness is limited.

Data visualization tools help users understand and explain insights from machine
learning model outcomes. Whether it is through simple graphical representations
like word clouds or more complicated and flexible data visualization tools
like Tableau dashboards, these tools make it easier to understand and
communicate the value uncovered by the model and drive better business decision-
making.

Data Insights + DataRobot

DataRobot’s Insights functionality has a host of tools for model interpretation, all
of which allow users to easily understand the models. For example, if a hospital’s
dataset includes a feature with doctors’ notes in the form of free
text, DataRobot automatically generates a word cloud to indicate which words in
those notes are related to the hospital’s target variable.

The example below shows words from just such a dataset which relate to patient
readmission rates. The red words are correlated with high-risk patient notes and the
blue words are correlated with low-risk patient notes. The size of the word
represents how frequently it occurs in the dataset.

DataRobot also includes tools that measure and rank the impact of individual
features (see Feature Impact) and provide details of how the model works and the
processes it runs on input data, as well as Prediction Explanations that give the top
reasons for the model’s outcome for each individual record.

What are actionable insights?

An actionable insight is a process that is derived from raw data analysis. For
example, business leaders could track a customer's behavior on a platform that can
indicate their sentiment regarding a product. These data points are fed into an
analytics platform, from which users can derive conclusions.

Business leaders might notice that customers aren't as keen on a product feature
that was popular in the past and that they're asking for a new feature or
enhancement. Social media listening strategies also help to gather raw data and
turn them into actionable insights.

Every industry could use actionable insights. "Actionable insights arising from
analytics and AI are no longer a luxury, but a necessity for achieving
competitiveness," said Eitan Sofer, head of developer platform at Sisense, an
analytics provider.

Information by itself isn't an actionable insight. If the information a company

receives enables stakeholders to make decisions, modify current processes or
overhaul them, then that's an actionable insight.
These days, analytics platforms rely on AI and machine learning (ML) algorithms
to drive actionable insights. For example, a financial company can use an ML
algorithm to quickly assess a loan applicant's credit-worthiness and provide a
human agent with a recommendation to approve or disapprove the loan.

While the human agent still has discretion over the process, the algorithm makes
their job easier and more efficient.

Preparing data for analysis: Structured and unstructured data

While ML algorithms offer great insights, they're fully dependent on the data
they're fed during their learning and maturity stages.

"Analytics must cope with both structured and unstructured data to achieve optimal
results," Sofer said.

One of the main challenges of generating actionable insights is the need to deal
with both structured and unstructured data.

Analyzing structured data is straightforward. This data can be customer or user-

generated and is present in predefined templates. For example, customer
information forms are a source of structured data since they have predefined fields
that customers enter information into. Structured data is housed in a traditional
database and as a result, is highly schema-dependent.

Unstructured data is more challenging. Examples of this data include raw text,
social media comments, log files and call transcripts. Unstructured data needs to be
stored in a NoSQL database or a data lake, is free-form and doesn't adhere to
schemas.

Unstructured data, by some estimates, comprises 80% of all available data, and
using a powerful business intelligence (BI) platform can simplify turning this data
into actionable insights. A platform that connects to multiple sources will make it
easier to import disparate data and create efficient workflows.

Turning data into actionable insights

Actionable insights aren't the only kind of insight an analytics platform can
provide. For example, a company may receive confirmation of a business strategy
that requires no further action. These insights are just as valuable as actionable
ones.

It's important to formulate business questions before turning to an analytics

platform to determine which insights are actionable and which ones are not. Which
issues are you trying to solve, and how will you measure success? It's also helpful
to establish the context of the dataset before accepting the insight as valid.

Every data collection method has bias within it. For example, log files could
indicate that customers spend a lot of time on one particular function within the
platform. The company may conclude that this is the most important feature and
work to enhance it. However, the customers could be spending time there because
the feature is inefficient or unintuitive.

Always pay attention to the context in which the data was collected. "Tunnel vision
is a problem across a variety of fields, from sports to industry," said George
Walker, managing director at Novotek U.K. and Ireland. "It is easy to fixate on a
single goal and forget to take a step back for a wider view of the situation. Yet
taking this step back can provide invaluable insight and understanding of the
reality of a situation."

Once the data's context is established, construct hypotheses to test. Many

organizations measure data and then wander through the results looking for some
magical insight. Establishing a hypothesis and testing it for accuracy will deliver
actionable insights much faster.
Infusing insights into workflows to assist in decision making

"Modeling complex and multidimensional data parameters in innovative ways and

presenting those insights within workflows will change the future of every
industry," Sofer said.

However, the future can change only if insights are used to drive decision-making.

Many companies derive insights but neglect to act on them. To take full advantage
of a BI analytics platform, an organization must create an optimization plan. A Six
Sigma action plan can help integrate new ideas and findings into workflows, define
the problem and measure the effectiveness of the current process.

Use the BI platform to analyze the root causes of these issues and make
improvements to processes. Once implemented, measure and control the new
process for effectiveness.

Integrating insights into workflows is a cultural matter as well. Educate employees

and let them know that analytics augments their jobs and isn't intended to replace
them. That way, an organization can see greater adoption of data-driven insights
into everyday business processes.

It's also important to democratize the BI platform so that everyone, including

business users, can run ad hoc reports to derive insights. The company will
increase the number of actionable insights exponentially by breaking down silos
like these.

Negating all those guesses

Actionable insights directly affect an organization's bottom line by increasing

conversions and as a result, sales. Lower churn will accompany higher revenue
numbers since business leaders will have greater insight into what drives the
customers' value question.
The biggest benefit is an organization's ability to move from hunches and
"feelings" to backing up decisions with hard data. No more will business leaders
have to take shots in the dark and hope a new process works.

Creating actionable insights is a process like any other, and it's perhaps the most
important factor for a business' overall success.

DATA EXPLORIATION

What is Data Exploration?

Data exploration definition: Data exploration refers to the initial step in data analysis
in which data analysts use data visualization and statistical techniques to describe
dataset characterizations, such as size, quantity, and accuracy, in order to better
understand the nature of the data.

Data exploration techniques include both manual analysis and automated data
exploration software solutions that visually explore and identify relationships between
different data variables, the structure of the dataset, the presence of outliers, and the
distribution of data values in order to reveal patterns and points of interest, enabling
data analysts to gain greater insight into the raw data.

Data is often gathered in large, unstructured volumes from various sources and data
analysts must first understand and develop a comprehensive view of the data before
extracting relevant data for further analysis, such as univariate, bivariate, multivariate,
and principal components analysis.

Data Exploration Tools

Manual data exploration methods entail either writing scripts to analyze raw data or
manually filtering data into spreadsheets. Automated data exploration tools, such as
data visualization software, help data scientists easily monitor data sources and
perform big data exploration on otherwise overwhelmingly large datasets. Graphical
displays of data, such as bar charts and scatter plots, are valuable tools in visual data
exploration.
A popular tool for manual data exploration is Microsoft Excel spreadsheets, which
can be used to create basic charts for data exploration, to view raw data, and to
identify the correlation between variables. To identify the correlation between two
continuous variables in Excel, use the function CORREL() to return the correlation.
To identify the correlation between two categorical variables in Excel, the two-way
table method, the stacked column chart method, and the chi-square test are effective.

There is a wide variety of proprietary automated data exploration solutions,

including business intelligence tools, data visualization software, data preparation
software vendors, and data exploration platforms. There are also open source data
exploration tools that include regression capabilities and visualization features, which
can help businesses integrate diverse data sources to enable faster data exploration.
Most data analytics software includes data visualization tools.

Why is Data Exploration Important?

Humans process visual data better than numerical data, therefore it is extremely
challenging for data scientists and data analysts to assign meaning to thousands of
rows and columns of data points and communicate that meaning without any visual
components.

Data visualization in data exploration leverages familiar visual cues such as shapes,
dimensions, colors, lines, points, and angles so that data analysts can effectively
visualize and define the metadata, and then perform data cleansing. Performing the
initial step of data exploration enables data analysts to better understand and visually
identify anomalies and relationships that might otherwise go undetected.

INFORMATION BASED LEARNING.

In this chapter we discuss the ways in which concepts from information theory can
be used to build prediction models. We start by discussing decision trees, the
fundamental structure used in information-based machine learning, before
presenting the fundamental measures of information content that are used: entropy
and information gain. We then present the ID3 algorithm, the standard algorithm
used to induce a decision tree from a dataset. The extensions and variations to this
standard approach that we present describe how different data types can be
handled, how overfitting can be avoided using decision tree pruning, and how
multiple prediction models can be combined in ensembles to improve prediction
accuracy.

DECISION TREES:
o Decision Tree is a Supervised learning technique that can be used for both
classification and Regression problems, but mostly it is preferred for solving
Classification problems. It is a tree-structured classifier, where internal
nodes represent the features of a dataset, branches represent the
decision rules and each leaf node represents the outcome.
o In a Decision tree, there are two nodes, which are the Decision
Node and Leaf Node. Decision nodes are used to make any decision and
have multiple branches, whereas Leaf nodes are the output of those
decisions and do not contain any further branches.
o The decisions or the test are performed on the basis of features of the given
dataset.
o It is a graphical representation for getting all the possible solutions to a
problem/decision based on given conditions.
o It is called a decision tree because, similar to a tree, it starts with the root
node, which expands on further branches and constructs a tree-like structure.
o In order to build a tree, we use the CART algorithm, which stands
for Classification and Regression Tree algorithm.
o A decision tree simply asks a question, and based on the answer (Yes/No), it
further split the tree into subtrees.
o Below diagram explains the general structure of a decision tree:
Why use Decision Trees?

There are various algorithms in Machine learning, so choosing the best algorithm
for the given dataset and problem is the main point to remember while creating a
machine learning model. Below are the two reasons for using the Decision tree:

o Decision Trees usually mimic human thinking ability while making a

decision, so it is easy to understand.
o The logic behind the decision tree can be easily understood because it shows
a tree-like structure.

Decision Tree Terminologies

Root Node: Root node is from where the decision tree starts. It represents the
entire dataset, which further gets divided into two or more homogeneous sets.
Leaf Node: Leaf nodes are the final output node, and the tree cannot be
segregated further after getting a leaf node.
Splitting: Splitting is the process of dividing the decision node/root node into
sub-nodes according to the given conditions.
Branch/Sub Tree: A tree formed by splitting the tree.
Pruning: Pruning is the process of removing the unwanted branches from the
tree.
Parent/Child node: The root node of the tree is called the parent node, and
other nodes are called the child nodes.

How does the Decision Tree algorithm Work?

In a decision tree, for predicting the class of the given dataset, the algorithm starts
from the root node of the tree. This algorithm compares the values of root attribute
with the record (real dataset) attribute and, based on the comparison, follows the
branch and jumps to the next node.

For the next node, the algorithm again compares the attribute value with the other
sub-nodes and move further. It continues the process until it reaches the leaf node
of the tree. The complete process can be better understood using the below
algorithm:

29.2K
Machine Learning - Data Description - Measures of Central Tendency: Mean,
Median and Mode
 o Step-1: Begin the tree with the root node, says S, which contains the
complete dataset.
o Step-2: Find the best attribute in the dataset using Attribute Selection
Measure (ASM).
o Step-3: Divide the S into subsets that contains possible values for the best
attributes.
o Step-4: Generate the decision tree node, which contains the best attribute.
o Step-5: Recursively make new decision trees using the subsets of the dataset
created in step -3. Continue this process until a stage is reached where you
cannot further classify the nodes and called the final node as a leaf node.

Example: Suppose there is a candidate who has a job offer and wants to decide
whether he should accept the offer or Not. So, to solve this problem, the decision
tree starts with the root node (Salary attribute by ASM). The root node splits
further into the next decision node (distance from the office) and one leaf node
based on the corresponding labels. The next decision node further gets split into
one decision node (Cab facility) and one leaf node. Finally, the decision node splits
into two leaf nodes (Accepted offers and Declined offer). Consider the below
diagram:
 Attribute Selection Measures

While implementing a Decision tree, the main issue arises that how to select the
best attribute for the root node and for sub-nodes. So, to solve such problems there
is a technique which is called as Attribute selection measure or ASM. By this
measurement, we can easily select the best attribute for the nodes of the tree. There
are two popular techniques for ASM, which are:

o Information Gain
o Gini Index

1. Information Gain:

o Information gain is the measurement of changes in entropy after the

segmentation of a dataset based on an attribute.
o It calculates how much information a feature provides us about a class.
o According to the value of information gain, we split the node and build the
decision tree.
o A decision tree algorithm always tries to maximize the value of information
gain, and a node/attribute having the highest information gain is split first. It
can be calculated using the below formula:

1. Information Gain= Entropy(S)- [(Weighted Avg) *Entropy(each feature)

Entropy: Entropy is a metric to measure the impurity in a given attribute. It

specifies randomness in data. Entropy can be calculated as:

Entropy(s)= -P(yes)log2 P(yes)- P(no) log2 P(no)

Where,

o S= Total number of samples

o P(yes)= probability of yes
o P(no)= probability of no

2. Gini Index:

o Gini index is a measure of impurity or purity used while creating a decision

tree in the CART(Classification and Regression Tree) algorithm.
 o An attribute with the low Gini index should be preferred as compared to the
high Gini index.
o It only creates binary splits, and the CART algorithm uses the Gini index to
create binary splits.
o Gini index can be calculated using the below formula:

Gini Index= 1- ∑jPj2

Pruning: Getting an Optimal Decision tree

Pruning is a process of deleting the unnecessary nodes from a tree in order to get
the optimal decision tree.

A too-large tree increases the risk of overfitting, and a small tree may not capture
all the important features of the dataset. Therefore, a technique that decreases the
size of the learning tree without reducing accuracy is known as Pruning. There are
mainly two types of tree pruning technology used:

o Cost Complexity Pruning

o Reduced Error Pruning.

Advantages of the Decision Tree

o It is simple to understand as it follows the same process which a human
follow while making any decision in real-life.
o It can be very useful for solving decision-related problems.

o It helps to think about all the possible outcomes for a problem.

o There is less requirement of data cleaning compared to other algorithms.

Disadvantages of the Decision Tree

o The decision tree contains lots of layers, which makes it complex.

o It may have an overfitting issue, which can be resolved using the Random
Forest algorithm.
o For more class labels, the computational complexity of the decision tree may
increase.
Python Implementation of Decision Tree

Now we will implement the Decision tree using Python. For this, we will use the
dataset "user_data.csv," which we have used in previous classification models. By
using the same dataset, we can compare the Decision tree classifier with other
classification models such as KNN SVM, LogisticRegression, etc.

Steps will also remain the same, which are given below:

o Data Pre-processing step

o Fitting a Decision-Tree algorithm to the Training set
o Predicting the test result
o Test accuracy of the result(Creation of Confusion matrix)
o Visualizing the test set result.

SIMILARITY BASED LEARNING:

Similarity-based approaches to machine learning come from the idea that the best
way to make a predictions is to simply look at what has worked well in the past
and predict the same thing again. The fundamental concepts required to build a
system based on this idea are feature spaces and measures of similarity, and
these are covered in the fundamentals section of this chapter. These concepts allow
us to understand the standard approach to building similarity-based models: the
nearest neighbor algorithm. After covering the standard algorithm, we then look
at extensions and variations that allow us to handle noisy data (the k nearest
neighbor, or k-NN, algorithm), to make predictions more efficiently (k-d trees), to
predict continuous targets, and to handle different kinds of descriptive features
with varying measures of similarity. We also take the opportunity to introduce the
use of data normalization and feature selection in the context of similarity-based
learning. These techniques are generally applicable to all machine learning
algorithms but are especially important when similarity-based approaches are used.

KNN Algorithm

o K-Nearest Neighbour is one of the simplest Machine Learning algorithms

based on Supervised Learning technique.
o K-NN algorithm assumes the similarity between the new case/data and
available cases and put the new case into the category that is most similar to
the available categories.
o K-NN algorithm stores all the available data and classifies a new data point
based on the similarity. This means when new data appears then it can be
easily classified into a well suite category by using K- NN algorithm.
o K-NN algorithm can be used for Regression as well as for Classification but
mostly it is used for the Classification problems.
o K-NN is a non-parametric algorithm, which means it does not make any
assumption on underlying data.
o It is also called a lazy learner algorithm because it does not learn from the
training set immediately instead it stores the dataset and at the time of
classification, it performs an action on the dataset.
o KNN algorithm at the training phase just stores the dataset and when it gets
new data, then it classifies that data into a category that is much similar to
the new data.
o Example: Suppose, we have an image of a creature that looks similar to cat
and dog, but we want to know either it is a cat or dog. So for this
identification, we can use the KNN algorithm, as it works on a similarity
measure. Our KNN model will find the similar features of the new data set
to the cats and dogs images and based on the most similar features it will put
it in either cat or dog category.
Why do we need a K-NN Algorithm?

Suppose there are two categories, i.e., Category A and Category B, and we have a
new data point x1, so this data point will lie in which of these categories. To solve
this type of problem, we need a K-NN algorithm. With the help of K-NN, we can
easily identify the category or class of a particular dataset. Consider the below
diagram:

How does K-NN work?

The K-NN working can be explained on the basis of the below algorithm:

o Step-1: Select the number K of the neighbors

o Step-2: Calculate the Euclidean distance of K number of neighbors
o Step-3: Take the K nearest neighbors as per the calculated Euclidean
distance.
o Step-4: Among these k neighbors, count the number of the data points in
each category.
o Step-5: Assign the new data points to that category for which the number of
the neighbor is maximum.
o Step-6: Our model is ready.

Suppose we have a new data point and we need to put it in the required category.
Consider the below image:
o Firstly, we will choose the number of neighbors, so we will choose the k=5.
o Next, we will calculate the Euclidean distance between the data points. The
Euclidean distance is the distance between two points, which we have
already studied in geometry. It can be calculated as:

o By calculating the Euclidean distance we got the nearest neighbors, as three

nearest neighbors in category A and two nearest neighbors in category B.
Consider the below image:
 o As we can see the 3 nearest neighbors are from category A, hence this new
data point must belong to category A.

How to select the value of K in the K-NN Algorithm?

Below are some points to remember while selecting the value of K in the K-NN
algorithm:

o There is no particular way to determine the best value for "K", so we need to
try some values to find the best out of them. The most preferred value for K
is 5.
o A very low value for K such as K=1 or K=2, can be noisy and lead to the
effects of outliers in the model.
o Large values for K are good, but it may find some difficulties.

Advantages of KNN Algorithm:

o It is simple to implement.

o It is robust to the noisy training data

o It can be more effective if the training data is large.

Disadvantages of KNN Algorithm:

o Always needs to determine the value of K which may be complex some
time.
 o The computation cost is high because of calculating the distance between the
data points for all the training samples.

PROBABILITY BASED LEARNING:

In this chapter we introduce probability-based approaches to machine learning.

Probability-based prediction approaches are heavily based on Bayes’ Theorem, and
the fundamentals section of this chapter introduces this important cornerstone of
computer science after covering some other fundamentals of probability theory.
We then present the naive Bayes model, the standard approach to using probability-
based approaches to machine learning. The extensions and variations to this
standard approach that we describe are the use of smoothing to combat overfitting,
the modifications required to the standard naive Bayes model to allow it to handle
continuous features, and Bayesian network models that give us more control than a
naive Bayes model over the assumptions that are encoded in a model.

Naïve Bayes Algorithm :

• Naïve Bayes algorithm is a supervised learning algorithm, which is based

on Bayes theorem and used for solving classification problems.
• It is a probabilistic classifier, which means it predicts on the basis of the
probability of an object.
• Naïve Bayes Classifier is one of the simple and most effective Classification
algorithms which helps in building the fast machine learning models that can
make quick predictions.
• Some popular examples of Naïve Bayes Algorithm are spam filtration,
Sentimental analysis, and classifying articles.

The Naïve Bayes algorithm is comprised of two words Naïve and Bayes,

• Naïve: It is called Naïve because it assumes that the occurrence of a certain

feature is independent of the occurrence of other features.
• Bayes: It is called Bayes because it depends on the principle of Bayes'
Theorem

Bayes' Theorem:

• Bayes' theorem is used to determine the probability of a hypothesis with

prior knowledge. It depends on the conditional probability.
 • The formula for Bayes' theorem is given as:
• Where,
• P(A|B) is Posterior probability: Probability of hypothesis A on the
observed event B.
• P(B|A) is Likelihood probability: Probability of the evidence given that the
probability of a hypothesis is true.
• P(A) is Prior Probability: Probability of hypothesis before observing the
evidence.
• P(B) is Marginal Probability: Probability of Evidence.

Types of Naïve Bayes Model:

• Gaussian: The Gaussian model assumes that features follow a normal

distribution. This means if predictors take continuous values instead of
discrete, then the model assumes that these values are sampled from the
Gaussian distribution.
• Multinomial: The Multinomial Naïve Bayes classifier is used when the data
is multinomial distributed. It is primarily used for document classification
problems, it means a particular document belongs to which category such as
Sports, Politics, education, etc.
The classifier uses the frequency of words for the predictors.
• Bernoulli: The Bernoulli classifier works similar to the Multinomial
classifier, but the predictor variables are the independent Booleans variables.
Such as if a particular word is present or not in a document. This model is
also famous for document classification tasks.

Advantages of Naïve Bayes Classifier:

• Naïve Bayes is one of the fast and easy ML algorithms to predict a class of
datasets.
• It can be used for Binary as well as Multi-class Classifications.
• It is the most popular choice for text classification problems.

Disadvantages of Naïve Bayes Classifier:

• Naive Bayes assumes that all features are independent or unrelated, so it

cannot learn the relationship between features.

Applications of Naïve Bayes Classifier:

• It is used for Credit Scoring.

 • It is used in medical data classification.
• It is used in Text classification such as Spam filtering and Sentiment
analysis.

ERROR BASED LEARNING:

In error-based machine learning, we perform a search for a set of parameters for a

parameterized model that minimizes the total error across the predictions made by
that model with respect to a set of training instances. The fundamentals section of
this chapter introduces the key ideas of a parameterized model, measuring error
and an error surface. We then present the standard approach to building error-
based predictive models: multivariable linear regression with gradient descent. The
extensions and variations to this standard approach that we describe are how to
handle categorical descriptive features, the use of logistic regression to make
predictions for categorical target features, fine tuning regression models,
techniques for building non-linear and multinomial models, and support vector
machines, which take a slightly different approach to using error to build prediction
models.

SUPPORT VECTOR MACHINE:

Support Vector Machine or SVM is one of the most popular Supervised Learning
algorithms, which is used for Classification as well as Regression problems.
However, primarily, it is used for Classification problems in Machine Learning.

The goal of the SVM algorithm is to create the best line or decision boundary that
can segregate n-dimensional space into classes so that we can easily put the new
data point in the correct category in the future. This best decision boundary is
called a hyperplane.

SVM chooses the extreme points/vectors that help in creating the hyperplane.
These extreme cases are called as support vectors, and hence algorithm is termed
as Support Vector Machine. Consider the below diagram in which there are two
different categories that are classified using a decision boundary or hyperplane:
Example: SVM can be understood with the example that we have used in the
KNN classifier. Suppose we see a strange cat that also has some features of dogs,
so if we want a model that can accurately identify whether it is a cat or dog, so
such a model can be created by using the SVM algorithm. We will first train our
model with lots of images of cats and dogs so that it can learn about different
features of cats and dogs, and then we test it with this strange creature. So as
support vector creates a decision boundary between these two data (cat and dog)
and choose extreme cases (support vectors), it will see the extreme case of cat and
dog. On the basis of the support vectors, it will classify it as a cat. Consider the
below diagram:

SVM algorithm can be used for Face detection, image classification, text
categorization, etc.

Types of SVM

SVM can be of two types:

 o Linear SVM: Linear SVM is used for linearly separable data, which means

if a dataset can be classified into two classes by using a single straight line,

then such data is termed as linearly separable data, and classifier is used

called as Linear SVM classifier.

o Non-linear SVM: Non-Linear SVM is used for non-linearly separated data,

which means if a dataset cannot be classified by using a straight line, then

such data is termed as non-linear data and classifier used is called as Non-

linear SVM classifier.

Hyperplane and Support Vectors in the SVM algorithm:

Hyperplane: There can be multiple lines/decision boundaries to segregate the

classes in n-dimensional space, but we need to find out the best decision boundary
that helps to classify the data points. This best boundary is known as the
hyperplane of SVM.

The dimensions of the hyperplane depend on the features present in the dataset,
which means if there are 2 features (as shown in image), then hyperplane will be a
straight line. And if there are 3 features, then hyperplane will be a 2-dimension
plane.

We always create a hyperplane that has a maximum margin, which means the
maximum distance between the data points.

Support Vectors:

The data points or vectors that are the closest to the hyperplane and which affect
the position of the hyperplane are termed as Support Vector. Since these vectors
support the hyperplane, hence called a Support vector.

How does SVM works?

Linear SVM:
The working of the SVM algorithm can be understood by using an example.
Suppose we have a dataset that has two tags (green and blue), and the dataset has
two features x1 and x2. We want a classifier that can classify the pair(x1, x2) of
coordinates in either green or blue. Consider the below image:

So as it is 2-d space so by just using a straight line, we can easily separate these
two classes. But there can be multiple lines that can separate these classes.
Consider the below image:

Hence, the SVM algorithm helps to find the best line or decision boundary; this
best boundary or region is called as a hyperplane. SVM algorithm finds the
closest point of the lines from both the classes. These points are called support
vectors. The distance between the vectors and the hyperplane is called as margin.
And the goal of SVM is to maximize this margin. The hyperplane with maximum
margin is called the optimal hyperplane.
Non-Linear SVM:

If data is linearly arranged, then we can separate it by using a straight line, but for
non-linear data, we cannot draw a single straight line. Consider the below image:

So to separate these data points, we need to add one more dimension. For linear
data, we have used two dimensions x and y, so for non-linear data, we will add a
third dimension z. It can be calculated as:

z=x2 +y2

By adding the third dimension, the sample space will become as below image:
So now, SVM will divide the datasets into classes in the following way. Consider
the below image:

Since we are in 3-d Space, hence it is looking like a plane parallel to the x-axis. If
we convert it in 2d space with z=1, then it will become as:

Hence we get a circumference of radius 1 in case of non-linear data.

EVALUATION:

The train/test/validation split

The most important thing you can do to properly evaluate your model is to not
train the model on the entire dataset. I repeat: do not train the model on the entire
dataset. I talked about this in my post on preparing data for a machine learning
model and I'll mention it again now because it's that important. A typical train/test
split would be to use 70% of the data for training and 30% of the data for testing.
As I discussed previously, it's important to use new data when evaluating our
model to prevent the likelihood of overfitting to the training set. However,
sometimes it's useful to evaluate our model as we're building it to find that best
parameters of a model - but we can't use the test set for this evaluation or else we'll
end up selecting the parameters that perform best on the test data but maybe not the
parameters that generalize best. To evaluate the model while still building and
tuning the model, we create a third subset of the data known as the validation set.
 A typical train/test/validation split would be to use 60% of the data for training,
20% of the data for validation, and 20% of the data for testing.

I'll also note that it's very important to shuffle the data before making these splits
so that each split has an accurate representation of the dataset.

Metrics

In this session, I'll discuss common metrics used to evaluate models.

Classification metrics

When performing classification predictions, there's four types of outcomes that could
occur.

 True positives are when you predict an observation belongs to a class and it
actually does belong to that class.
 True negatives are when you predict an observation does not belong to a class and
it actually does not belong to that class.
 False positives occur when you predict an observation belongs to a class when in
reality it does not.
 False negatives occur when you predict an observation does not belong to a class
when in fact it does.
These four outcomes are often plotted on a confusion matrix. The following
confusion matrix is an example for the case of binary classification. You would
generate this matrix after making predictions on your test data and then identifying
each prediction as one of the four possible outcomes described above.
You can also extend this confusion matrix to plot multi-class classification
predictions. The following is an example confusion matrix for classifying
observations from the Iris flower dataset.

The three main metrics used to evaluate a classification model are accuracy,
precision, and recall.

Accuracy is defined as the percentage of correct predictions for the test data. It can
be calculated easily by dividing the number of correct predictions by the number of
total predictions.
accuracy=correctpredictionsallpredictionsaccuracy=correctpredictionsallprediction
s
Precision is defined as the fraction of relevant examples (true positives) among all
of the examples which were predicted to belong in a certain class.
precision=truepositivestruepositives+falsepositivesprecision=truepositivestrueposit
ives+falsepositives
Recall is defined as the fraction of examples which were predicted to belong to a
class with respect to all of the examples that truly belong in the class.
recall=truepositivestruepositives+falsenegativesrecall=truepositivestruepositives+f
alsenegatives
The following graphic does a phenomenal job visualizing the difference between
precision and recall.
Precision and recall are useful in cases where classes aren't evenly distributed. The
common example is for developing a classification algorithm that predicts whether
or not someone has a disease. If only a small percentage of the population (let's say
1%) has this disease, we could build a classifier that always predicts that the person
does not have the disease, we would have built a model which is 99% accurate and
0% useful.

However, if we measured the recall of this useless predictor, it would be clear that
there was something wrong with our model. In this example, recall ensures that
we're not overlooking the people who have the disease, while precision ensures
that we're not misclassifying too many people as having the disease when they
don't. Obviously, you wouldn't want a model that incorrectly predicts a person has
cancer (the person would end up in a painful and expensive treatment process for a
disease they didn't have) but you also don't want to incorrectly predict a person
does not have cancer when in fact they do. Thus, it's important to evaluate both the
precision and recall of a model.

Ultimately, it's nice to have one number to evaluate a machine learning model just
as you get a single grade on a test in school. Thus, it makes sense to combine the
precision and recall metrics; the common approach for combining these metrics is
known as the f-score.

The ββ parameter allows us to control the tradeoff of importance between precision

and recall. β<1β<1 focuses more on precision while β>1β>1 focuses more on
recall.
When I was searching for other examples to explain the tradeoff between precision
and recall, I came across the following article discussing using machine learning to
predict suicide. In this case, we'd want to put much more focus on the model's
recall than its precision. It would be much less harmful to have an intervention
with someone who was not actually considering suicide than it would be to miss
someone who was considering suicide. However, precision is still important
because you don't want too many instances where your model predicts false
positives or else you have the case of "The Model Who Cried Wolf" (this is the
reference for those not familiar with the story).
Note: The article only reports the accuracy of these models, not the precision or
recall! By now you should know that accuracy alone is not too informative
regarding a model's effectiveness. The original paper published, however, does
present the precision and recall.

Regression metrics

Evaluation metrics for regression models are quite different than the above metrics
we discussed for classification models because we are now predicting in a
continuous range instead of a discrete number of classes. If your regression model
predicts the price of a house to be $400K and it sells for $405K, that's a pretty
good prediction. However, in the classification examples we were only concerned
with whether or not a prediction was correct or incorrect, there was no ability to
say a prediction was "pretty good". Thus, we have a different set of evaluation
metrics for regression models.
Explained variance compares the variance within the expected outcomes, and
compares that to the variance in the error of our model. This metric essentially
represents the amount of variation in the original dataset that our model is able to
explain.
Mean squared error is simply defined as the average of squared differences
between the predicted output and the true output. Squared error is commonly used
because it is agnostic to whether the prediction was too high or too low, it just
reports that the prediction was incorrect.
The R2 coefficient represents the proportion of variance in the outcome that our
model is capable of predicting based on its features.
Bias vs Variance
The ultimate goal of any machine learning model is to learn from examples and
generalize some degree of knowledge regarding the task we're training it to
perform. Some machine learning models provide the framework for generalization
by suggesting the underlying structure of that knowledge. For example, a linear
regression model imposes a framework to learn linear relationships between the
information we feed it. However, sometimes we provide a model with too much
pre-built structure that we limit the model's ability to learn from the examples -
such as the case where we train a linear model on a exponential dataset. In this
case, our model is biased by the pre-imposed structure and relationships.
Models with high bias pay little attention to the data presented; this is also known
as underfitting.

It's also possible to bias a model by trying to teach it to perform a task without
presenting all of the necessary information. If you know the constraints of the
model are not biasing the model's performance yet you're still observed signs of
underfitting, it's likely that you are not using enough features to train the model.

On the other extreme, sometimes when we train our model it learns too much from
the training data. That is, our model captures the noise in the data in addition to the
signal. This can cause wild fluctuations in the model that does not represent the
true trend; in this case, we say that the model has high variance. In this case, our
model does not generalize well because it pays too much attention to the training
data without consideration for generalizing to new data. In other words,
we've overfit the model to the training data.
In summary, a model with high bias is limited from learning the true trend and
underfits the data. A model with high variance learns too much from the training
data and overfits the data. The best model sits somewhere in the middle of the two
extremes.

Next, I'll discuss two common tools that are used to diagnosed whether a model is
susceptible to high bias or variance.
Validation curves
As we discussed in the previous section, the goal with any machine learning model
is generalization. Validation curves allow us to find the sweet spot between
underfitting and overfitting a model to build a model that generalizes well.
A typical validation curve is a plot of the model's error as a function of some
model hyperparameter which controls the model's tendency to overfit or underfit
the data. The parameter you choose depends on the specific model you're
evaluating; for example, you might choose to plot the degree of polynomial
features (typically, this means you have polynomial features up to this degree) for
a linear regression model. Generally, the chosen parameter will have some degree
of control over the model's complexity. On this curve, we plot both the training
error and the validation error of the model. Using both of these errors combined,
we can diagnose whether a model is suffering from high bias or high variance.
In the region where both the training error and validation error are high, the model
is subject to high bias. Here, it was not able to learn from the data and it
performing poorly.

In the region where the training error and validation error diverge, with the training
error staying low and validation error increasing, we're beginning to see the effects
of high variance. The training error is low because we're overfitting the data and
learning too much from the training examples, while the validation error remains
high because our model isn't able to generalize from the training data to new data.
Learning curves
The second tool we'll discuss for diagnosing bias and variance in a model is
learning curves. Here, we'll plot the error of a model as a function of the number of
training examples. Similar to validation curves, we'll plot the error for both the
training data and validation data.

If our model has high bias, we'll observe fairly quick convergence to a high error
for the validation and training datasets. If the model suffers from high bias, training
on more data will do very little to improve the model. This is because models
which underfit the data pay little attention to the data, so feeding in more data will
be useless. A better approach to improving models which suffer from high bias is
to consider adding additional features to the dataset so that the model can be more
equipped to learn the proper relationships.
If our model has high variance, we'll see a gap between the training and validation
error. This is because the model is performing well for the training data, since it
has been overfit to that subset, and performs poorly for the validation data since it
was not able to generalize the proper relationships. In this case, feeding more data
during training can help improve the model's performance.

Other practical advice

Another common thing I'll do when evaluating classifier models is to reduce the
dataset into two dimensions and then plot the observations and decision boundary.
Sometimes it's helpful to visually inspect the data and your model when evaluating
its performance.
 UNIT- V
APPLICATIONS OF MACHINE LEARNING

Image Recognition – Speech Recognition – Email spam and Malware Filtering

– Online fraud detection – Medical Diagnosis.

IMAGE RECOGNITION:

What is image recognition?

Image recognition is a mechanism used to identify an object within an image and

to classify it in a specific category, based on the way human people recognize
objects within different sets of images.

How does image recognition work for humans?

When we see an object or an image, we, as human people, are able to know
immediately and precisely what it is. People class everything they see on different
sorts of categories based on attributes we identify on the set of objects. That way,
even though we don’t know exactly what an object is, we are usually able to
compare it to different categories of objects we have already seen in the past and
classify it based on its attributes. Let’s take the example of an animal that is
unknown to us. Even if we cannot clearly identify what animal it is, we are still
able to identify it as an animal.

People rarely think about what they are observing and how they can identify
objects, it completely happens subconsciously. People aren’t focused on
everything that surrounds them all the time. Our brain has been trained to identify
objects quite easily, based on our previous experiences, that is to say, objects we
have already encountered in the past. We do have an extraordinary power of
deduction: when we see something that resembles an object we have already seen
before, we are able to deduce that it belongs to a certain category of items. We
don’t necessarily need to look at every part of an image to identify the objects in it.
As soon as you see a part of the item that you recognized, you know what it is. We
usually use colors and contrasts to identify items.

For humans, most image recognition works subconsciously. But it is a lot more
complicated when it comes to image recognition with machines.
How does image recognition work with machines?

Machines only recognize categories of objects that we have programmed into

them. They are not naturally able to know and identify everything that they see. If
a machine is programmed to recognize one category of images, it will not be able
to recognize anything else outside of the program. The machine will only be able
to specify whether the objects present in a set of images correspond to the category
or not. Whether the machine will try to fit the object in the category, or it will
ignore it completely.

For a machine, an image is only composed of data, an array of pixel values. Each
pixel contains information about red, green, and blue color values (from 0 to 255
for each of them). For black and white images, the pixel will have information
about darkness and whiteness values (from 0 to 255 for both of them).

Machines don’t have a look at the whole image; they are only interested in pixel
values and patterns in these values. They simply take pixel patterns of an item and
compare them with other patterns. If two patterns are close enough, the machine
will associate them and recognize the second pattern as something it has already
encountered in the past. In that sense, what is happening is the machine will look
for groups of similar pixel values across images and will try to place them in
specific image categories.

It is very rare that a program recognizes an image at 100%. Pixel patterns are very
rarely 100% the same when comparing them. Solving these problems and finding
improvements is the job of IT researchers, the goal being to propose the best
experience possible to users.

Practicing Image recognition with machine learning

The goal of image recognition is to identify, label and classify objects which are
detected into different categories. Object or image recognition is a whole process
that involves various traditional computer vision tasks:

 Image classification: labeling an image and creating categories.

 Object localization: identifying the location of an object in an image, by
surrounding it with a bounding box.
 Object Detection: determining the presence of objects with the help of
bounding boxes and categorizing it within the class it belongs to.
  Object Segmentation: distinguishing the various elements. Identify and
locate each and every item of the picture. Segmentation doesn’t use
bounding boxes but highlights the contour of the object in the image.

For the past few years, this computer vision task has achieved big successes,
mainly thanks to machine learning applications.

Processes and Models

In order to go through these 4 tasks and to complete them, machine learning and
image recognition systems do require going through a few important steps.

Set up, Training and Testing

First of all, the machine has to know exactly what it has to look for. Thus, it is
necessary to give it the parameters you decide to work on. Defining the dimensions
of bounding boxes and what elements are inside is crucial. To do so, the machine
has to be provided with some references, which can be pictures, videos or
photographs, etc. These elements will allow it to be more efficient when analyzing
future data. This will create a sort of data library that will then be used by
the Neural Network to distinguish the various objects. A Neural Network is
composed of multiple artificial neurons. These neurons are meant to imitate the
human brain. It works with a set of various algorithms also inspired by the way the
brain functions. If we want the image recognition model to analyze and categorize
different races of dogs, the model will need to have a database of the various races
in order to recognize them.

Second, the model needs to go on a training phase. The dataset needs to be entered
within a program in order to function properly. And this phase is only meant to
train the Convolutional Neural Network (CNN) to identify specific objects and
organize them accurately in the correspondent classes.

Before an AI model is being used, it needs to be thoroughly tested. To do so, it is

necessary to propose images that were not part of the training phase. Based on
whether or not the program has been able to identify all the items and on the
accuracy of classification, the model will be approved or not.

Hereafter are some of the most popular Image Recognition with Machine Learning
Models and how they work.
Support Vector Machines (SVM)

SVM models use a set of techniques in order to create an algorithm that will
determine whether an image corresponds to the target object or if it does not. From
the dataset it was set with, the SVM model is trained to separate a hyper plan into
several categories. During the process, depending on the pixel values, the objects
are being placed in the hyper plan their position predicts a category based on the
category separation learned from the training phase.

Bag of Features Models

This bag of features models takes into account the image to be analyzed and a
reference sample photo. Then, the algorithm in the model tries to match pixel
patterns from the sample photo with some parts of the target picture to analyze.

Viola-Jones Algorithm

This is one of the most famous ones used for facial recognition. It was used even
before using CNNs. It scans the faces of people, extracts some of the features from
the faces, and classifies them. It also uses a boosting algorithm which is meant to
help have a much more accurate classification.

Convolutional Neural Networks

We have dealt with CNN earlier in this article. But it is necessary to go a little
deeper with this concept.

Machine learning relies on the things the Human Brain gave it. It is mainly
supervised by people, first when it comes to delivering the set of the reference
images, to training the machine into distinguishing the objects and testing the
method. CNN is a specific model architecture from Deep Learning techniques.
CNN algorithm allows machines to detect and classify with quite an impressive
precision all of the objects which are observed in a picture.

This type of algorithm works with different layers of perception. It is often hard to
interpret a specific layer role in the final prediction but research has made progress
on it. We can for example interpret that a layer analyzes colors, another one
shapes, a next one textures of the objects, etc. At the end of the process, it is the
superposition of all layers that makes a prediction possible.
Popular Image recognition Algorithms

Deep Learning has shown to be extremely efficient for detecting objects and
classifying them. Different approaches are available and each has their own
characteristics. Here are three of these proceedings.

Faster Region-based CNN (Faster RCNN)

Faster RCNN is a Convolutional Neural Network algorithm based on a Region

analysis. When analyzing a new image, after training with a reference set, Faster
RCNN is going to propose some regions in the picture where an object could be
possibly found. When the algorithm detects areas of interest, these are then
surrounded by bounding boxes and cropped, before being analyzed to be classified
within the proper category. Why is it called Faster RCNN? Because by proposing
regions where objects might be placed, it allows the algorithm to go much faster
since the program does not have to navigate throughout the whole image to
analyze each and every pixel pattern.

Single Shot Detector (SSD)

Using the Single Shot Detector algorithm is directly linked to RCNN. When
identifying and drawing bounding boxes, most of the time, they overlap each other.
This is mainly why SSDs are used. To prevent these boxes from overlapping,
SSDs use a grid with various ratios to divide the image. Then if we observe a box
being placed on top of another one, for example, because the system detected a girl
in front of a car, the algorithm proposes to create two different anchor boxes, in
order to separate the two items. That way, the picture is divided into different
feature plans and is treated separately, and the machine is able to handle the
analysis of more objects. This technique reveals to be very successful, accurate,
and can be executed quite rapidly.

You Only Look Once (YOLO)

As the name of the algorithm might suggest, the technique processes the whole
picture only one-time thanks to a fixed-size grid. It looks for elements in each part
of the grid and determines if there is any item. If so, it will be identified with
abounding boxes and then classify it with a category. Looking at the grid only once
makes the process quite rapid, but there is a risk that the method does not go deep
into details. The results are less accurate than with the SSD method.
Programming Image recognition

Some accessible solutions exist for anybody who would like to get familiar with
these techniques. An introduction tutorial is even available on Google on that
specific topic.

Various methods are used to detect items in a picture and classify them. But how
do we apply them to our devices?

Programming with Python language

Python is an IT coding language, meant to program your computer devices in order

to make them work the way you want them to work. One of the best things about
Python is that it supports many different types of libraries, especially the ones
working with Artificial Intelligence. Image detection and recognition are available
with Python.

To start working on this topic, Python and the necessary extension packages
should be downloaded and installed on your system. Some of the packages include
applications with easy-to-understand coding and make AI an approachable method
to work on. It is recommended to own a device that handles images quite
effectively. We are talking about good quality graphics cards for instance. The next
step will be to provide Python and the image recognition application with a free
downloadable and already labeled dataset, in order to start classifying the various
elements. Finally, a little bit of coding will be needed, including drawing the
bounding boxes and labeling them.

Application Programming Interface (API)

An API is an application meant to create a link between two different software, in

order to exchange data and/or functionalities. Regarding image recognition, this
solution is mainly used to get picture data from a Cloud API such as AWS Cloud
from Amazon. That way, you can get a wide library of image references.

Programming item recognition using this method can be done fairly easily and
rapidly. That way, you can deploy the program within a short period of time. But,
it should be taken into consideration that choosing this solution, taking images
from an online cloud, might lead to privacy and security issues. This process
should be used for testing or at least an action that is not meant to be permanent.
Edge AI

Contrarily to APIs, Edge AI is a solution that involves confidentiality regarding the

images. The images are uploaded and offloaded on the source peripheral where
they come from, so no need to worry about putting them on the cloud.

Edge AI is very often used with real-time videos. In most cases, it will be used
with connected objects or any item equipped with motion sensors.

AI Platform

Some online platforms are available to use in order to create an image recognition
system, without starting from zero. If you don’t know how to code, or if you are
not so sure about the procedure to launch such an operation, you might consider
using this type of pre-configured platform.

The different fields of application for image recognition with machine learning

Nowadays, Computer Vision and recognition are always around us. From
unlocking your phone with your face in the morning to coming into a mall to do
some shopping. Many different industries have decided to implement Artificial
Intelligence in their processes.

Facial recognition

Face analysis is a major recognition application. It is used by many companies to

detect different faces at the same time, in order to know how many people there are
in an image for example. Face recognition can be used by police and security
forces to identify criminals or victims. Face analysis involves gender detection,
emotion estimation, age estimation, etc.

The need for businesses to identify these characteristics is quite simple to

understand. It allows them to analyze precisely who their customers are. That way,
a fashion store can be aware that its clientele is composed of 80% of women, the
average age surrounds 30 to 45 years old, and the clients don’t seem to appreciate
an article in the store. Their facial emotion tends to be disappointed when looking
at this green skirt. Acknowledging all of these details is necessary for them to
know their targets and adjust their communication in the future.
Face analysis is also very much used for identification. Apple recently developed a
way to unlock your phone with your face. They even developed a method to do it
without taking off your surgical mask.

Health and Medicine

Treating patients can be challenging, sometimes a tiny element might be missed

during an exam, leading medical staff to deliver the wrong treatment. To prevent
this from happening, the Healthcare system started to analyze imagery that is
acquired during treatment. X-ray pictures, radios, scans, all of these image
materials can use image recognition to detect a single change from one point to
another point. Detecting the progression of a tumor, of a virus, the appearance of
abnormalities in veins or arteries, etc.

Smart Farming

Farmers’ daily lives are far from being easy. To keep taking good care of both their
animals and their plantations, they need to monitor them both.

Monitoring their animals has become a comfortable way for farmers to watch their
cattle. With cameras equipped with motion sensors and image detection programs,
they are able to make sure that all their animals are in good health. They can also
monitor animal births. Farmers can easily detect if a cow is having difficulties
giving birth to its calf. They can intervene rapidly to help the animal deliver the
baby, thus preventing the potential death of two animals.

Farmers also grow their own plants, mainly to feed their cattle. To see if the fields
are in good health, image recognition can be programmed to detect the presence of
a disease on a plant for example. The farmer can treat the plantation rapidly and be
able to harvest peacefully.

Security and Safety

Security and safety are two major concerns in today’s society. Thanks to image
recognition and detection, it gets easier to identify criminals or victims, and even
weapons. In an airport for example, where safety is crucial. All-day long, security
agents are scrutinizing screens. Helped by Artificial Intelligence, they are able to
detect dangers extremely rapidly. When a piece of luggage is unattended, the
watching agents can immediately get in touch with the field officers, in order to get
the situation under control and to protect the population as soon as possible. When
a passport is presented, the individual's fingerprints and face are analyzed to make
sure they match with the original document.

Insurance companies are also using recognition technologies. When somebody is

filing a complaint about the robbery and is asking for compensation from the
insurance company. The latter regularly asks the victims to provide video footage
or surveillance images to prove the felony did happen. And very often, the thief is
caught on camera and can be identified. Sometimes, the guilty individual gets sued
and can face charges thanks to facial recognition.

Ecommerce

Online stores are experiencing a boom since the beginning of the COVID-19
pandemic. They managed to develop their activities exponentially thanks to
various elements.

One of the recent advances they have come up with is image recognition to better
serve their customer. Many platforms are now able to identify the favorite products
of their online shoppers and to suggest them new items to buy, based on what they
have watched previously.

On another note, some new applications propose their users simply snap a picture
of an item found on somebody they have met in the street, in order to find a store
that has a similar or the same item available for purchase. The algorithm is then
able to give a list of places where you can buy the shoes your friend was wearing
today.

Improvements made in the field of AI and picture recognition for the past decades
have been tremendous. There is absolutely no doubt that researchers are already
looking for new techniques based on all the possibilities provided by these
exceptional technologies

SPEECH RECOGNITION:

What is speech recognition?

Speech recognition, or speech-to-text, is the ability of a machine or program to
identify words spoken aloud and convert them into readable text. Rudimentary
speech recognition software has a limited vocabulary and may only identify words
and phrases when spoken clearly. More sophisticated software can handle natural
speech, different accents and various languages.
Speech recognition uses a broad array of research in computer science, linguistics
and computer engineering. Many modern devices and text-focused programs have
speech recognition functions in them to allow for easier or hands-free use of a
device.

Speech recognition and voice recognition are two different technologies

and should not be confused:

 Speech recognition is used to identify words in spoken language.

 Voice recognition is a biometric technology for identifying an individual's
voice.
How does speech recognition work?
Speech recognition systems use computer algorithms to process and interpret
spoken words and convert them into text. A software program turns the sound a
microphone records into written language that computers and humans can
understand, following these four steps:

1. analyze the audio;

2. break it into parts;
3. digitize it into a computer-readable format; and
4. use an algorithm to match it to the most suitable text representation.

Speech recognition software must adapt to the highly variable and context-specific
nature of human speech. The software algorithms that process and organize audio
into text are trained on different speech patterns, speaking styles, languages,
dialects, accents and phrasings. The software also separates spoken audio from
background noise that often accompanies the signal.

To meet these requirements, speech recognition systems use two types of models:

 Acoustic models. These represent the relationship between linguistic units of

speech and audio signals.
 Language models. Here, sounds are matched with word sequences to
distinguish between words that sound similar.
What applications is speech recognition used for?
Speech recognition systems have quite a few applications. Here is a sampling of
them.

Mobile devices. Smartphones use voice commands for call routing, speech-to-text
processing, voice dialing and voice search. Users can respond to a text without
looking at their devices. On Apple iPhones, speech recognition powers the
keyboard and Siri, the virtual assistant. Functionality is available in secondary
languages, too. Speech recognition can also be found in word processing
applications like Microsoft Word, where users can dictate words to be turned into
text.

Education. Speech recognition software is used in language instruction. The

software hears the user's speech and offers help with pronunciation.

Customer service. Automated voice assistants listen to customer queries and

provides helpful resources.

Healthcare applications. Doctors can use speech recognition software to

transcribe notes in real time into healthcare records.

Disability assistance. Speech recognition software can translate spoken words into
text using closed captions to enable a person with hearing loss to understand what
others are saying. Speech recognition can also enable those with limited use of
their hands to work with computers, using voice commands instead of typing.

Court reporting. Software can be used to transcribe courtroom proceedings,

precluding the need for human transcribers.

Emotion recognition. This technology can analyze certain vocal characteristics to

determine what emotion the speaker is feeling. Paired with sentiment analysis, this
can reveal how someone feels about a product or service.

Hands-free communication. Drivers use voice control for hands-free

communication, controlling phones, radios and global positioning systems, for
instance.

What are the features of speech recognition systems?

Good speech recognition programs let users customize them to their needs. The
features that enable this include:

 Language weighting. This feature tells the algorithm to give special attention
to certain words, such as those spoken frequently or that are unique to the
conversation or subject. For example, the software can be trained to listen for
specific product references.
 Acoustic training. The software tunes out ambient noise that pollutes spoken
audio. Software programs with acoustic training can distinguish speaking
style, pace and volume amid the din of many people speaking in an office.
 Speaker labeling. This capability enables a program to label individual
participants and identify their specific contributions to a conversation.
 Profanity filtering. Here, the software filters out undesirable words and
language.
What are the different speech recognition algorithms?
The power behind speech recognition features comes from a set of algorithms and
technologies. They include the following:

 Hidden Markov model. HMMs are used in autonomous systems where a

state is partially observable or when all of the information necessary to make a
decision is not immediately available to the sensor (in speech recognition's
case, a microphone). An example of this is in acoustic modeling, where a
program must match linguistic units to audio signals using statistical
probability.
 Natural language processing. NLP eases and accelerates the speech
recognition process.
 N-grams. This simple approach to language models creates a probability
distribution for a sequence. An example would be an algorithm that looks at
the last few words spoken, approximates the history of the sample of speech
and uses that to determine the probability of the next word or phrase that will
be spoken.
 Artificial intelligence. AI and machine learning methods like deep learning
and neural networks are common in advanced speech recognition software.
These systems use grammar, structure, syntax and composition of audio and
 voice signals to process speech. Machine learning systems gain knowledge
with each use, making them well suited for nuances like accents.
What are the advantages of speech recognition?
There are several advantages to using speech recognition software, including the
following:

 Machine-to-human communication. The technology enables electronic

devices to communicate with humans in natural language or conversational
speech.
 Readily accessible. This software is frequently installed in computers and
mobile devices, making it accessible.
 Easy to use. Well-designed software is straightforward to operate and often
runs in the background.
 Continuous, automatic improvement. Speech recognition systems that
incorporate AI become more effective and easier to use over time. As systems
complete speech recognition tasks, they generate more data about human
speech and get better at what they do.
What are the disadvantages of speech recognition?
While convenient, speech recognition technology still has a few issues to work
through. Limitations include:

 Inconsistent performance. The systems may be unable to capture words

accurately because of variations in pronunciation, lack of support for some
languages and inability to sort through background noise. Ambient noise can
be especially challenging. Acoustic training can help filter it out, but these
programs aren't perfect. Sometimes it's impossible to isolate the human voice.
 Speed. Some speech recognition programs take time to deploy and master.
The speech processing may feel relatively slow.
 Source file issues. Speech recognition success depends on the recording
equipment used, not just the software.

E-MAIL SPAM AND MALEWARE FILTERING

In the new era of technical advancement, electronic mails (e-mails) have gathered
significant users for professional, commercial, and personal communications. In
2019,on average, every person was receiving 130 emails each day, and overall,
296 Billion emails have been sent in that year.
Because of the high demand and huge user base, there is an upsurge in unwanted
emails, also known as spam emails. There were times when more than 50% of the
total emails were spam emails. Even in the current date, people lose millions of
dollars to frauds every day.

But, in the figure shown below, it can be observed that the quantity of such
emails has decreased significantly after 2016 because of the evolution of the
software that can detect these spam emails and can filter them out.

Percentage of emails marked as Spam (Source: Statista)

Many several techniques are present in the market to detect spam e-mails. If we
want to classify broadly, there are 5 different techniques based on which
algorithms decide whether any mail is spam or not.
Content-Based Filtering Technique
Algorithms analyze words, the occurrence of words, and the distribution of words
and phrases inside the content of e-mails and segregate them into spam and non-
spam categories

Case Base Spam Filtering Method

Algorithms trained on well-annotated spam/non-spam marked emails try to classify

the incoming mails into two categories.

Heuristic or Rule-Based Spam Filtering Technique

Algorithms use pre-defined rules in the form of a regular expression to give a score
to the messages present in the e-mails. Based on the scores generated, they
segregate emails into spam and non-spam categories.

The Previous Likeness Based Spam Filtering Technique

Algorithms extract the incoming mails' features and create a multi-dimensional

space vector and draw points for every new instance. Based on the KNN algorithm,
these new points get assigned to the closest class of spam and non-spam.

Adaptive Spam Filtering Technique

Algorithms classify the incoming mails into various groups and, based on the
comparison scores of every group with the defined set of groups, spam, and non-
spam emails got segregated.

This article will give an idea for implementing content-based filtering using one of
the most famous algorithms for spam detection, which is K-Nearest Neighbour
(KNN).
k-NN based algorithms are widely used for clustering tasks. Let’s quickly know
the entire architecture of this implementation first and then explore every step.
Executing these 5 steps, one after the other will help us implement our spam
classifier smoothly.
Training Testing Phase

New Email Classification

Step 1: E-mail Data Collection

The dataset contained in a corpus plays a crucial role in assessing the performance
of any spam filter. Many open-source datasets are freely available in the public
domain. Below mentioned two datasets are widely popular as they contain a huge
amount of emails.

1. Enron corpus datasets (Created in 2006 and having 55% spam emails)

2. Tre c 2007 dataset ( Created in 2007 and having 67% spam emails)

Train/Test Split: Split the dataset into train and test datasets but make sure that
both sets must balance the numbers of ham and spam emails (ham is a fancy name
for non-spam emails).
Enron Corpus Dataset on Kaggle

Step 2: Pre-processing of E-mail content

At this step, we mainly perform tokenization of mails. Tokenization is a process
where we break the content of an email into words and transform big messages
into a sequence of representative symbols termed tokens. These tokens are
extracted from the email body, header, subject, and image.
Extracting words from images (For a simple implementation, this can be
ignored)

These days, senders have options to attach inline images to the mail. These emails
can be categorized as spam emails not based on their mail content but on the
images' content. This was not an easy task until Google came up with the open-
source library Tesser act. This library extracts the words from images
automatically with certain accuracy. But still, Times New Roman and Captcha
words are difficult to read automatically.

Step 3: Feature Extraction and Selection

After pre-processing, we can have a large number of words. Here we can maintain
adatabase that contains the frequency of the different words represented in each
column. These attributes can be categorized on a different basis, like:
Important attributes: Frequency of repeated words, Number of semantic
discrepancies, an Adult content bag of words, etc.

Additional Attributes: Sender account features like Sender country, IP address,

email, age of sender, Number of replies, number of recipients, and website
address. Note: These web addresses are converted in the word format only. For
example, [Link] can be converted to “HTTP
google.”Sometimes these processes are called Normalization.
Less important attributes: Geographical distance between sender and receiver
,Sender’s date of birth, Account lifespan, Sex of sender, and Age of the recipient.

You must be clear that the more the number of attributes → more the time
complexity of the model. These attributes can be huge, and hence techniques like
Stemming, noise removal ,and stop-words removal can be used. One of the famous
stemming algorithms is the Porter Stemmer Algorithm. Some general things that
we do in stemming are:

Removing suffixes (-ed, -ing, -full, -ness, etc.)

Removing prefixes (Un-, Re-, Pre-, etc.)

List of stop words

Example dataset format

Step 4: KNN (K-Nearest Neighbour) Implementation

Similar to the Nearest Neighbour algorithm, the K-Nearest Neighbour algorithm
serves the purpose of clustering. Still, instead of giving just one nearest instance, it
looks at the closest K instances to the new incoming instance. Based on the
frequency of those K instances, K-NN classifies the new instances. The value of K
is considered tobe a hyper parameter that needs tuning. To tune this, one can take
one of the famous Hit and Trial approaches where we try some K's values and then
check the model's performance.

To find the nearest instance, one can use the Euclidean distance. One can use the
Scikit-learn library to implement the K-NN algorithm for this task.
Step 5: Performance Analysis
Now our algorithm is ready, so we must check the performance of the model. Even
a single missed important message may cause a user to reconsider the value of
spam filtering. So we must be sure that our algorithm will be as close to
100%accurate. But some researchers feel that considering only the accuracy as the
evaluation parameter for spam classification is not enough.
According to the below table (also known as the confusion matrix), we must
evaluate our spam-classification model based on 4 different parameters.

Accuracy : (TP + TN)/(TP + FP + FN + TN)

Precision : TP / (TP + FP)
Sensitivity : TP / (TP + FN)
Specificity : TN / (TN + FP)

More advanced algorithms are available in the market for this classification, but
you can easily achieve more than 90% accuracy using k-NN based
implementation.

ONLINE FRAUD DETECTION

What is fraud detection?

According to the Cambridge Dictionary, fraud is “the crime of getting money by

deceiving people.” It’s as old as humanity. Ever since people started exchanging
goods and services, there has been a risk of one party scamming the other. And
there has always been a risk of a third party scamming both the seller and the
buyer. With the development and expansion of e-commerce, fraud has taken on
new forms and become more powerful than ever. As the scale of e-shopping,
online banking, and online insurance increases, fraudsters take full advantage of
every weak spot in every system they can find. Quite often, before professionals
can patch up a system, sensitive data is stolen and millions of dollars are lost.
Fraud has turned into a major issue and an uncontrolled expenditure for e-
commerce retailers on a global level.

Preventing, detecting, and eliminating fraud are some of the primary concerns of
the e-commerce and banking industries at present. One of the most promising
means for achieving them are machine learning development services.

Machine learning has already been used to successfully detect email spam. It also
makes focused product recommendations for millions of online shoppers. The
availability of big data allows machine learning to develop at a great scale and
improve significantly over a very short time. Advances in statistical modeling and
constantly increasing processing power make it possible for machine learning to
enter the e-commerce and banking sectors. These industries are placing big hopes
on effective fraud detection using machine learning as a tool that can prevent cyber
crime. Let’s see how this goal can be achieved.

Why use machine learning for fraud detection?

In a nutshell, machine learning (ML) is the science of creating and applying

algorithms that are capable of learning from the past. Machine learning finds a
perfect use case in fraud detection. Machine learning algorithms learn to tell
fraudulent operations from legitimate ones without raising the suspicions of those
executing the transactions. Machine learning can fight financial fraud by using big
data better and faster than humans ever will be able to.

Fraud detection machine learning models are more effective than humans

The concept behind using machine learning in fraud detection is that fraudulent
transactions have specific features that legitimate transactions do not. Based on this
assumption, machine learning algorithms detect patterns in financial operations
and decide whether a given transaction is legitimate. Machine learning fraud
detection algorithms are way more effective than humans. They can process a raft
of information faster than a team of the best analysts ever could. What’s more, ML
algorithms can spot patterns that seem unrelated or go unnoticed by a human. By
exploring and studying tons of cases of fraudulent behavior, ML algorithms
determine the most stealthy fraudulent patterns and remember them forever.
ML handles overload well

As online fraud becomes omnipresent, it also gets more sophisticated. Both

fraudsters and businesses use cutting-edge technologies and race to get a step
ahead. In such a stressful environment, companies need to analyze way more
information than they can handle to fight fraud. Even when a business hires a team
of top data scientists, they cannot chase fraudulent attempts as fast as they happen.
Fraud detection machine learning models come to the rescue, being able to work
24/7 and analyze enormous amounts of data at the snap of a finger.

ML beats traditional fraud detection systems

The traditional fraud detection model is based on a static rules-based system, also
referred to as a production or expert system. Although these systems have been
effective for a long time, some of their major disadvantages make them unsuitable
for modern digital environments. A static rules-based system is heavily dependent
on human labor. But naturally, top analysts are expensive. And their work takes
time. What’s more, even top experts create rules based on their knowledge, skills,
and experience, which are always limited. Such rules can grow to enormous sizes
and get so complex that it’s nearly impossible for an outsider to understand them
when needed. Also, creating a new rule and implementing it takes a while when
done by hand.

Fraud detection using machine learning can solve all of these issues. It can beat
traditional fraud detection systems in terms of speed, quality, and cost-
effectiveness. An unsupervised machine learning system can process new data
autonomously all the time and update its models and patterns immediately. As the
data assets of every business become more and more overwhelming, it becomes
clear that only machine learning can cope with this volume of information. The
more data ML algorithms process, the better they get and the more they learn.
Eventually, more data and higher workloads mean better and more precise fraud
detection algorithms.

How does machine learning in fraud detection work?

Machine learning models and algorithms for fraud detection

Machine learning algorithms come in the following types:

 Supervised learning

 Unsupervised learning

 Semi-supervised learning

 Reinforcement learning

Supervised learning

Supervised learning is the most common way of implementing machine learning.

It works for cases like fraud detection in deep learning environments in FinTech.
In a supervised learning model, all input information has to be labeled as good or
bad. A supervised learning model is based on predictive data analysis and is only
as accurate as the training set provided for it. A major drawback of the supervised
model is that it’s not able to detect fraud that was not included in the historical data
set from which it learned.
Unsupervised learning

An unsupervised learning model is meant to detect anomalous behavior in cases

where there is little transaction data or such data is not available at all. An
unsupervised learning model continuously processes and analyzes new data and
updates its models based on the findings. It learns to notice patterns and decide
whether they’re parts of legitimate or fraudulent operations. Deep learning in fraud
detection is usually associated with unsupervised learning algorithms.

Semi-supervised learning

Semi-supervised learning is somewhere between supervised and unsupervised

learning. It works for cases where labeling information is either impossible or too
expensive and requires the labor of human experts. A semi-supervised algorithm
for fraud detection in deep learning stores data about important group parameters
even when group membership of the unlabeled data is unknown. It does so based
on the assumption that the discovered patterns can still be valuable.

Reinforcement learning

A reinforcement learning algorithm allows machines to automatically detect ideal

behavior within a specified context. It constantly learns from the environment to
find actions that minimize risks and maximize rewards. A reinforcement feedback
signal is required for the model to learn its behavior.

MEDICAL DIAGNOSIS

Machine learning is simply making healthcare smarter. This powerful subset of

artificial intelligence may be familiar to many in use cases such as speech
recognition used by voice assistants, and in creating personalized online shopping
experiences through its ability to learn associations. However, machine learning
has demonstrated truly life-impacting potential in healthcare – particularly in the
area of medical diagnosis.

To demonstrate how machine learning and deep learning are able to provide a
medical diagnosis, I’ll walk you through a step-by-step example of how the
technology can be used to detect and diagnose breast cancer using a publicly
available data set.
Challenges of Applying Machine Learning in Healthcare

There are several obstacles impeding faster integration of machine learning in

healthcare today. One of the biggest challenges is the ability to obtain patient data
sets which have the necessary size and quality of samples needed to train state-of-
the-art machine learning models. Since patient data is protected by strict privacy
and security rules, the data is not easy to collect, share and distribute. Furthermore,
there are challenges with the format and quality of data which usually require
significant effort to clean and prepare for machine learning analyses.

As machine learning and data science are starting to be adopted as a tool in

healthcare applications, the industry is slowly pushing the boundaries on what it
can do. Its primary function will most likely involve data analysis based on the fact
that each patient generates large volumes of health data such as X-ray results,
vaccinations, blood samples, vital signs, DNA sequences, current medications,
other past medical history, and much more.

Using Machine Learning to Detect and Diagnose Breast Cancer

One application of machine learning in a healthcare context is digital diagnosis.

ML can detect patterns of certain diseases within patient electronic healthcare
records and inform clinicians of any anomalies. In this sense, the artificial
intelligence technique can be compared to a second pair of eyes that can evaluate
patient health based on the knowledge extracted from big data sets by summarizing
millions of observations of diseases that a patient could possibly have. To illustrate
just how useful machine learning as a medical diagnosis tool can be, I examined its
use in breast cancer detection using a publicly available Breast Cancer Wisconsin
(Diagnostic) Data Set.

Characteristics of benign and malignant tumors.

This data set consists of several instances of tumors. Tumors can either be benign
(non-cancerous) or malignant (cancerous). Benign tumors grow locally and do not
spread. As a result, they are not considered cancerous. However, they can still pose
a danger, especially if they press against vital organs like the brain. Malignant
tumors, in contrast, have the ability to spread and invade other tissues. This
process, known as metastasis, is a key feature of cancer. There are many different
types of malignancy-based tumors as well as locations that this type of cancer
tumor can originate, as described in the data set specification.

The breast cancer data set consists of 699 tumor samples where 458 (65.5%) are
benign (non-cancer) tumors and 241 (34.5%) malignant (cancer) tumors. Instances
in the data set have the following attributes:

Solving a problem with machine learning often involves many iterative

experiments meant to find the best model for solving the problem by further tuning
the model. Given that there are many machine learning algorithms and different
neural network architectures, a researcher (based on his/her experience, knowledge
and trusting his/her intuition) will select the most promising model to set up the
first experiment.

In our example, given the relatively small sizes of data sets, my intuition was to
start modeling using traditional machine learning algorithms (e.g. SVM, KNN etc.)
and shallow neural networks. To demonstrate some initial results using machine
learning to diagnose breast cancer, the following set of metrics are used: ROC
curve ≈ 0.99, Precision-Recall curve ≈ 0.99, and F1 ≈ 0.97.

Setting up the Diagnosis Model

Step 1: Dividing the Data Set

In order to get started modeling, the data set was split into two parts:

1. Train set (70%), for choosing and validating models, and

2. Test set (30%), hold out data on which we will see how well models are able
to generalize on unseen data.

Step 2: Defining the Metrics

Next, we need to define the key metrics to measure the efficiency of the models. In
order to describe the classifiers’ performance in the digital diagnoses problem, we
have four basic characteristics (numbers) based on which we can define derivative
measurement metrics. These four numbers are:

1. TP (True Positive) – number of correctly classified patients who have the

disease,
2. TN (True Negative) – number of correctly classified patients who are
healthy,
3. FP (False Positive) – number of misclassified patients who are healthy,
4. FN (False Negative) – number of misclassified patients who have the
disease.

Based on these numbers we define the metrics as follows:

 Accuracy – ratio of correctly classified patients to the total number of

patients (Accuracy = (TP+TN)/(TP+FP+FN+TN))
 Precision – ratio of correctly classified patients with the disease to the total
patients classified as having the disease. The intuition behind precision is
 how many patients classified as having disease truly have the disease
(Precision = TP/TP+FP).
 Recall – ratio of correctly classified diseased patients to patients who have
the disease. The intuition behind recall is how many patients who have
disease classified as having the disease. (Recall = TP/TP+FN).

Step 3: Evaluating the Models

The next step involves using precision and recall metrics to evaluate the models.
For the sake of simplifying the comparison of various models, we will use the
harmonic mean of precision and recall which is called an F1 score (F1 Score =
2*(Recall * Precision) / (Recall + Precision)).

After experimenting with different algorithms, the mean F1 scores, in cross-

validation, gained by each classifier is presented below. Given that accuracy is
considered the most intuitive measure, it has also been plotted on the graph.

Cross-validation scores of Machine Learning models.

As you can see from the graph, the classifiers are showing pretty good results in
terms of being able to better distinguish patients who have cancer versus those who
are healthy by reaching 0.94 F1 scores. Where the best value for F1 is 1, and the
worst value is 0. In order to gain higher scores, ensembles of these models were
created by using bagging techniques.
Cross-validation scores of the ensemble Machine Learning models.

As shown in the graph, the ensembles of models performed even better by reaching
0.95 F1 scores.

Step 4: Creating a Neural Network Model

In addition to the aforementioned diagnostic models, a Neural Network model was

created and tuned using the architecture shown below.
Neural Network model architecture.

This neural network classifier has resulted in 0.97 F1 mean scores on cross-
validation. This new neural network model’s F1 score is better compared to the
best model’s score gained in Step 3. Here are the top three models results so far.

Cross-validation results of the top three models.

Now let’s evaluate these models on the test data set which previously was not
shown to classifiers imitating new data. Below are the results demonstrating just
how well these models performed on the test data set.

As it’s shown in the graph, neural network classifier have performed better by
gaining 0.97 F1 scores on the test set.

Step 5: Evaluating Output Quality Through Receiver Operating Curves

In order to further evaluate classifiers’ output quality, let’s view their receiver
operating characteristic (ROC) curves.
The ROC graph efficiency is measured by the area under the curve. An area of 1
represents a perfect classifier, an area of 0.5 represents a worthless classifier (navy
color, dashed line in the graph). Here is the academic point system for judging
classifiers efficiency given to area under the curve.

0.90-1 = excellent (A)

0.80-0.90 = good (B)
0.70-0.80 = fair (C)
0.60-0.70 = poor (D)
0.50-0.60 = fail (F)

As it’s shown in the graph, all of three classifiers have above 0.99 area under the
curve which is considered excellent.

Step 6: Evaluating Output Quality Through Precision-Recall Curves

Let’s also look at the precision-recall curves associated with these classifiers.
The navy dashed line represents the baseline, where the perfect model is the one
with 1 average precision. As you can see, all three models’ average precision is
close to 1, which are excellent scores.

Step 7: Visualizing the Decision Boundaries

Lastly, an additional note about the models’ decision boundaries:

In order to gain some visual intuition about the data set and the algorithms decision
boundaries, we will reduce the dimensionality of 9D feature space to 2D
using PCA techniques and visualize the decision boundaries.

Conclusion

These models have shown excellent results on Breast Cancer Wisconsin

(Diagnostic) Data Set, however, in order to trust the models, we need to further test
them with new data and make sure they are still leading to excellent results. One
possible weakness associated with these models is that they do not include any
demography, race and genetic sequences attributes and other useful information
that could potentially strengthen the ground for classification. One last note of
caution: although the approaches outlined in this article may show promising
results, the intention was to demonstrate the potential of AI algorithms, it was not
intended for clinical use.