Explainable Artificial

Intelligence
Dr-Amer Abuhania
 2.7 Convolution Neural Network .. amr
Machine Learning Models:
2.8 Recurrent Neural Network . fathi
2.1 Support Vector Machine.- ahmad
2.9 Long Short-Term Memory .momeen
2.2 Multi-Layer Perceptron. rahma
2.10 Reinforcement Learning. Mohmad talal
2.3 Decision Tree . rawan
2.11 Boosting .shade
2.4 Random Forest . sadeel
2.12 Naive Bayes . malak
2.5 Linear Regression. shahd
2.13 Zero-Shot Learning .. yazan
2.6 Deep Neural Network . omar
Introduction
Artificial Intelligence (AI) enables the deployment of smart systems using Machine Learning
(ML) algorithms. There is a wide variety of machine learning algorithms that are used in
diverse domains, including transportation, cybersecurity, and business applications. A major
problem with classical ML algorithms is that they only provide a decision without any
explanation about how it reached the decision. In contrast, explainable AI provides insights
into the decision-making process so that a designer can use it for various activities, such as
accuracy improvement, localization of vulnerabilities, etc. In this chapter, we first provide an
overview of various machine learning algorithms. Next, we discuss popular approaches for
explainable machine learning.
Support Vector Machine
Support Vector Machine (SVM) is a typical supervised learning model. Intuitively, supervised
learning represents a learning process where an ML model is trained to satisfy given training
samples with determined labels in advance. After the stage of training using the labeled set,
the obtained ML model is expected to respond to new occurrences. SVM is trained to obtain
a hyper-plane in data space to separate them out with given labels while trying to maximize
the margin distance between data points and the hyper-plane. A trivial SVM example that
achieves a bi-classification between red and green circles is shown in Fig. 1. The cyan block
in the picture represents the maximized margin for an optimal classifier. Intuitively, the margin
distance reflects how “far” data points from two classes are from each other, and a large
margin distance indicates a better chance to correctly classify new occurrences.
Fig. 1 A simple example of using SVM for classification
Multi-Layer Perceptron

A commonly used Multi-Layer Perceptron (MLP) is a feed-forward model. An MLP

contains multiple layers of neurons with an activation function, each layer fully

connects the next layer with numerical values called weights. This activation function

maps weighted inputs to the output of the neuron. The objective of the MLP is to learn

these weights to match the inputs to the outputs as efficiently as possible. Figure 2

contains a simple example of MLP with four inputs.

Fig. 2 A simple example of MLP with four inputs
Decision Tree
Decision Tree (DT) is another commonly used supervised learning algorithm for classification. An example
decision tree is shown in Fig. 3 to illustrate its functionality and interpretability. The task is to decide the
location for exercise based on weather conditions. As we can see, DT is a tree-structure model with nodes
and edges. The basic workflow of DT is to perform a top-down tree traversal from the root. At each node, a
specific attribute becomes the key factor to determine the branch. This process continues until a category
label reaches the leaf node, which becomes the final decision.
The construction of such a decision tree relies on the selection of attributes for a given task. In many real-
world applications, this challenge is addressed using a large amount of historical data to automatically
build the decision tree. The construction of the tree is usually recursive and can be performed
automatically.
Fig. 3 Example decision tree to decide the exercise location based on weather conditions
Random Forest
Random Forest (RF), as the name suggests, is an ensemble of decision trees. RF is
actually an integrated algorithm. It first randomly selects different features and training
samples to generate many decision trees, where each decision tree is trained with a
subset of the training data. Then RF synthesizes the results of these decision trees by
voting or taking the average in the ensemble to present the final output. Random forest is
widely used in reality analysis. Compared with decision trees, it provides significant
improvement in accuracy and robustness at the same time.
 Linear Regression
The motivation for Linear Regression (LR) comes from statistics. Given a dataset, LR
learns from this dataset to produce a linear model that reflects the relationship
between 𝑥𝑖 , and 𝑦𝑖 as accurately as possible. Formally, the model can be written as

where is the bias term, and W

bold upper W is a set of weight parameters indicating the
weight of corresponding attributes. The learning process of LR is an optimization problem to
obtain optimal weight parameters W . The trained model with suitable weights is used to predict
values for new inputs.
Deep Neural Network
A Deep Neural Network (DNN) is an artificial neural network that can express or simulate a wide
variety of intrinsic functionalities in the fields of classification, regression, reconstruction, etc.
Analogous to neurons in our nervous system, the “neural network” of artificial intelligence is a
system built from “neurons” as shown in Fig. 4.

Fig. 4 A typical DNN neuron. The activation layer is applied to induce non-linearity
The functionality of one single neuron is limited but DNN makes use of multiple neurons and arranges
them in layers. Typically, DNN consists of an input layer, an output layer, and arbitrary number of hidden
layers in between for enabling DNNs to approximate the complex mapping of given data’s inputs and
outputs. The training process of DNNs can be summarized as follows:

1. Determine the structure and initialize the weights.

2. Feed training samples to compute training loss.
3. Compute the gradient of loss, use backpropagation to update the weight coefficients, and repeat
until convergence.

The flexibility of DNNs also enables their different variations to be successfully applied in various
contexts. The next three ML models show three specific examples of such variations.
Convolution Neural Network
Convolution Neural Network (CNN) is a
variant of DNN utilizing convolution
layers. The emergence of convolutional
neural networks has surpassed ordinary
neural networks in the field of image
processing with the characteristics of
fewer parameters, fast training, high
scores, and easy migration. A simple
example of CNN is shown in Fig. 5. The
convolution operation works by moving Basic structure of CNN
the convolution filters (kernel) across the
input images, extracting structural
features from the images at each location.
Recurrent Neural Network
Recurrent Neural Network (RNN) is another variant of neural networks. The general structure of RNN
is shown in Fig. 6. In the picture,brepresents the neural network architecture, where x0, x1, x2, . . . , xt

represents the time series inputs and h Subscare

hs i the outputs of hidden layers. For each single
input x Subscript
x i
i, RNN not only provides immediate response h Subscript
h ibut also stores
i

the information of the current input by updating the architecture itself.

Meanwhile, stored information will also be fed into the architecture in the next iteration to
supply extra information. Therefore, it is widely applied in security domain as it is efficient to
capture temporal dependencies of variables (signals).

Basic structure of RNN

Long Short-Term Memory
A specific type of RNN model is called Long Short-Term Memory (LSTM). It
applies gate mechanism to solve vanishing gradient and exploding gradient. Mean-
while, the gate mechanism provides feature filtering, saving useful features and
discarding useless features, which greatly enriches the information represe
Reinforcement Learning
Reinforcement learning (RL) is a branch of machine learning, but unlike the commonly known supervised
learning, it is closer to human learning. Its exploration process is actually a process of gradually learning
the rule so finter action through trials and responding to feedback from the environment. The RL model
continuously communicates with the environment to find an optimal strategy through a series of attempts
and constantly adjusts its behavior based on feedback. Figure 7 provides the basic framework of
reinforcement learning. RL framework consists of five core components: Agent, Environment, Action,
State, and Reward.

The basic framework of reinforcement learning

Agent refers to the object that can interact with the environment through actions. The agent of
reinforcement learning is usually the set of test cases to be optimized, which is continuously
updated through the learning process.
Environment is the receiver of the action, such as the optimization problem itself.
Action consists of all possible operations that may affect the environment, such as using the
current strategy for one-step calculation.
State refers to information about the environment that can be perceived by the Agent, such as
conditions and parameters.
Reward is the feedback information from the environment that describes the effect of the latest
action. For optimization problems, it often refers to the gain of objective function after performing
the current operation.
Boosting
Boosting is a learning model where multiple weak learners are combined to generate a strong classifier. Figure 8
shows an overview of a boosting framework. Initially, a base weak classifier learns the training data. Next, in each
iteration, a weak learner is added to reduce the training error of previously applied weak learners. Freund et al.
proposed the first practical Boosting algorithm, Ada Boost. The gradient boosting algorithm proposed by Breiman et
al. is also widely applied for many optimization problems. In each iteration of the gradient boosting algorithm, the
negative gradient of the current model on all samples is calculated. Next, a new weak classifier is trained with this
value for adjusting the weight of the weak classifier. Finally, the model gets updated accordingly.

Fig. 8 The ensemble consists of a set of weak classifiers. Subsequent models focus on fixing the
weakness of previous models. The final decision is based on the overall voting result
Naive Bayes
The naive Bayes Classifier is one of the simple probabilistic classifiers based on Bayes’ theorem. This
classifier has a strong assumption that the features are independent among themselves. From the
training data, a likelihood probability is calculated for each feature. For unknown input data, the
posterior probability for each class is calculated using Bayes’ theorem. The class having maximum
posterior probability value becomes the predicted level for the input data.
Zero-Shot Learning
Zero-Shot Learning (ZSL) is a variant of transfer learning, which relies on zero training samples to
handle unseen categories. The key idea of ZSL is to focus on learning the “general knowledge” of
given data, and unlike the commonly known learning approaches, it is closer to the human brain
when making judgments. For example, assume that a child has never seen a tiger before, and only
pictures of cats and dogs are shown to the child. The child is likely to identify the tiger as a cat. The
reason is that, even though the “tiger” category is never seen before, the brain is able to extract
information from the picture and make comparisons with known species based on similarity.
Similarly, the goal of ZSL is to train the model with clever adjustments during the training stage, so
that the model is capable of exploiting information to understand unseen data.
A typical implementation of ZSL is a matching network structure as shown in Fig. 9,
which consists of two major components, an extractor g
gaand a comparator ff Subscrip.
θ
θ

Extractor is responsible for recognizing and extracting general knowledge from training
set, while compactor works by comparing the similarity of test input and known genre
to assign label to it based on similarity score. In this scenario, the task is to classify the
genre of dog, where the model is not trained to merely remember human-defined
features from known samples but trained to be sophisticated in mining underlying
features by themselves, and make reasoning by comparing afterwards. By extracting
general knowledge from the entire sample set S, four different types of general
g Subscript theta

g in the model.
knowledge were extracted by g θ

Then f f Sworks by comparing the given sample’s feature with pre-stored ones and
θ

computing similarity scores. The one with the highest score becomes the classified
label.
Fig. 9 ZSL utilizing matching networks as proposed in
This section described a wide variety of machine learning (ML) models. Table 1 provides a list of
pros and cons for these ML models. This comparison will enable a designer to choose the most
appropriate ML model for a given application based on the application-specific requirements.
Table 1 Advantages and
disadvantages of various
machine learning techniques