MODULE 1

2. Machine Learning

2. Second reason is that the cost of storage has reduced. The hardware cost has also dropped.
Therefore, it is easier now to capture, process, store, distribute, and transmit the digital
information.

3. Thirdreason for popularity of machine learning is the availability of complex algorithms

now. Especially with the advent of deep learning, many algorithms are available for
machine learning.

With the popularityready adaption of machine learning by business organizations, it

and
has become a dominant technology trend now. Before starting the machine learning journey, let
us establish these terms data, information, knowledge, intelligence, and wisdom. A knowledge
-

pyramid is shown in Figure 1.1.

Wisdom

Intelligence
(applied
knowledge)

Knowledge
(condensed
information)

Information (processed data)

Data (mostly available as raw facts and

symbols)

Figure 1.1: The Knowledge Pyramid

What is data? All facts are data. Data can be numbers or text that can be processed by a
computer. Today, organizations are accumulating vast and growing amounts of data with data

sources such as flat files, databases, or data warehouses in different storage formats.

Processed data is called information. This includes patterns, associations, or relationships

among data. For example, sales data can be analyzed to extract information like which is the
fast selling product. Condensed knowledge. For example, the historical
information is called
patterns and future trends obtained in the above sales data can be called knowledge. Unless

knowledge is extracted, data is of no use. Similarly, knowledge is not useful unless it is put
into action. Intelligence is the applied knowledge for actions. An actionable form of knowledge
is called intelligence. Computer systems have been successful till this stage. The ultimate
objective of knowledge pyramid is wisdom that represents the maturity of mind that is, so far,
exhibited only by humans.

Here comes the need for machine learning. objective of machine learning is to process
The
these archival data for organizations to take better decisions to
design new products, improve the
business processes, and to develop effective decision support systems.
 Introduction to Machine Learning .3

1.2 MACHINE LEARNING EXPLAINED

Machine learning important sub-branch of Artificial Intelligence (AI). A frequently quoted

is an

definition of machine learning was by Arthur Samuel, one of the pioneers of Artificial Intelligence.
He stated that "Machine learning is the field of study that gives the computers ability to learn without being
explicitly programmed."
The key to this definition is that the systems should learn by itself without explicit programming.
How is it possible? It is widely known that to perform a computation, one needs to write programs
that teach the computers how to do that computation.

In conventional programming, after understandingthe problem, a detailed design of the

program such as a flowchart or an algorithm needs to be created and converted into programs
using a language. This approach could be difficult for many real-world
suitable programming
problems such as puzzles, games, and complex image recognition applications. Initially, artificial
intelligence aims to understand these problems and develop general purpose rules manually.
Then, these rules are formulated into logic and implemented in a program to create intelligent
systems. This idea of developing intelligent systems by using logic and reasoning by converting an
expert's knowledge into a set of rules and programs is called an expert system. An expert system
like MYCIN was designed for medical diagnosis after converting the expert knowledge of many
doctors into a system. However, this approach did not progress much as as programs lacked real

intelligence. The word MYCIN is derived from the fact that most of the antibiotics' names end with
'mycin'.
The above approach was impractical in many domains as programs still depended on human
expertise and hence did not truly exhibit intelligence. Then, the momentum shifted to machine
learning in the form develop intelligent systems by
of data driven systems. The focus of AI is to
using data-driven approach, where data is used as an input to develop intelligent models. The
models can then be used to predict new inputs. Thus, the aim of machine learning is to learn a
model or set of rules from the given dataset automatically so that it can predict the unknown data
correctly.
As humans take decisions based experience, computers make models based on extracted
on an

patterns in the input data and then use these data-filled models for prediction and to take decisions.
For computers, the learnt model is equivalent to human experience. This is shown in Figure 1.2.
Decisions
Experiencе
Humans

(a)
_ Data Model

Data-
Learning program
base

(b)

Figure 1.2: (a) A Learning System for Humans (b) A Learning System
for Machine Learning
Often, the quality of data determines the quality of experience and, therefore, the quality of
the learning system. In statistical learning, the relationship between the input x and output y is
4. Machine Learning

modeled function in the form y = f(x). Here, f is the learning function that maps the input x
as a

to output y. Learning of function fis the crucial aspect of forming a model in statistical learning.
In machine learning, this is simply called mapping of input to output.

The learning program summarizes the raw data in a model. Formally stated, a model is an
explicit description of patterns within the data in the form of:
1. Mathematical equation
2. Relational diagrams like trees/graphs
3. Logical if/else rules, or
4. Groupings called clusters
In summary, a model can be a formula, procedure or representation that can generate data
ca
decisions. The difference between pattern and model is that the former is local and applicable only
tocertain attributes but the latter is global and fits the entire dataset. For example, a model can be
helpful to examine whether a given email is spam or not. The point is that the model is generated
automatically from the given data.
Another pioneer of AI, Tom Mitchell's definition of machine learning states that, "A computer
program is said to learn from experience E, with respect to task T and some performance measure P,
if its performance on T measured by P improves with experience E." The important components of this
definition are experience E, task T, and performance measure P.
For example, the task T could be detecting an object in an image. gain the
The machine can

knowledge of object using training dataset of thousands of images. This is called experience E.
object detection T. The ability of the system
So, the focus is to use this experience E for this task of
to detect the object is measured by performance measures like precision and recall. Based on the
performance measures, course correction can be done to improve the performance of the system.
Models of computer systems are equivalent to human experience. Experience is based on data.
Humans gain experience by various means. They gain knowledge by rote learning. They observe
others and imitate it. Humans gain a lot of knowledge from teachers and [Link] learn many things
by trial and error. Once the knowledge is gained, when a new problem is encountered, humans
search for similar past situations and then formulate the heuristics and use that for prediction.
But, in systems, experience is gathered by these steps:
1. Collection of data

2. Once data is gathered, abstract concepts are formed out of that data. Abstraction is used
to generate concepts. This is equivalent to humans' idea of objects, for example, we have
some idea about how an elephant looks like.

3. Generalization converts the abstraction into an intelligence.

actionable form of
It can be viewed as ordering of all possible concepts. So, generalization involves ranking
of concepts, inferencing from them and formation of heuristics, an actionable aspect of
intelligence. Heuristics are educated guesses for all tasks. For example, if one runs or
encounters a danger, it is the resultant of human experience or his heuristics formation.
In machines, it happens the same way.

4. Heuristics normally works! But, occasionally, it may fail too. It is not the fault
of heuristics as it is just a 'rule of thumb'. The course correction is done by taking
evaluation measures. Evaluation checks the thoroughness of the models and to-do
course correction, if necessary, to generate better formulations.
 Introduction to Machine Learning• 5

1.3 MACHINE LEARNING IN RELATION TO OTHER FIELDS

Machine learning uses the concepts of Artificial Intelligence, Data Science, and Statistics primarily.
It is the resultant of combined ideas of diverse fields.

1.3.1 Маchine Learning and Artificial Intelligence

Machine learning is an important branch of AI, which is a much broader subject. The aim of AI is
to develop intelligent agents. An agent can be a robot, humans, or any autonomous systems.
Initially, the idea of AI was ambitious, that is, to develop intelligent systems like human beings.
The focus was on logic and logical inferences. It had seen many ups and downs. These down
periods were called AI winters.
The resurgence in Al happened due to development of data driven systems. The aim is to find
relations and regularities present in the data. Machine learning is the subbranch of AI, whose aim
is to extract the patterns forprediction. It is a broad field that includes learning from examples and
other areas like reinforcement learning. The relationship of AI and machine learning is shown in

Figure 1.3. The model can take an unknown instance and generate results.
Artificial
intelligence

Machine learning
Deep
learning

Figure 1.3: Relationship of Al with Machine Learning

Deep learning is a subbranch of machine learning. In deep learning, the modelsconstructed are

using neural network technology. Neural networks are based on the human neuron models. Many
neurons form a network connected with the activation functions that trigger further neurons to

perform tasks.

1.3.2 Machine Learning, Data Science, Data Mining, and Data Analytics
Data science is an 'Umbrella' term that encompasses many fields. Machine learning starts with
data. Therefore, data science and machine learning are interlinked. Machine learning is a branch
of data science. Data science deals with gathering of data for analysis. It is a broad field that
includes:
6. Machine Learning

Big Data Data science concerns about collection of data. Big data is a field of data science that
deals with data's following characteristics:

1. Volume: Huge amount of data is generated by big companies like Facebook, Twitter,
YouTube.

2. Variety: Data is available in variety of forms like images, videos, and in different formats.

3. Velocity: It refers to the speed at which the data is generated and processed.
Big data is used by many machine learning algorithms for applications such as language trans-
lation and image recognition. Big data influences the growth of subjects like Deep learning. Deep
learning is a branch of machine learning that deals with constructing models using neural networks.

Data Mining Data mining's original genesis is in the business. Like while mining the earth one

gets into precious resources, it is often believed that unearthing of the data produces hidden infor-
mation that otherwise would have eluded the attention of the management. Nowadays, many
consider that data mining and machine learning are same. There is no difference between these
fields except that data mining aims to extract the hidden patterns that are present in the data,
whereas, machine learning aims to use it for prediction.

Data Analytics Another branch of data science is data analytics. It aims to extract useful
knowledge from crude data. There are different types of analytics. Predictive data analytics is used
for making predictions. Machine learning is closely related to this branch of analytics and shares
almost all algorithms.
Pattern Recognition It is an engineering field. It uses machine learning algorithms to extract
the features for pattern analysis and pattern classification. One can view pattern recognition as a
specific application of machine learning.
These relations are summarized in Figure 1.4.
Data science

Data Data
mining analytics

Machine
learning

Pattern
Big data
recognition

Figure 1.4: Relationship of Machine Learning with Other Major Fields

1.3.3 Machine Learning and Statistics

Statistics is a branch of mathematics that has a solid theoretical foundation regarding statistical learning.
Like machine learning (ML),it can learn from data. But the difference between statistics and ML is that
statistical methods look for regularity in data called
patterns. Initially, statistics sets a hypothesis and
performs experiments to verify and validate the hypothesis in order to find relationships among data.
 Introduction to Machine Learning • 7

Statistics requires knowledge of the statistical procedures and the guidance of a good
statistician. It is mathematics intensive and models are often complicated equations and involve
many assumptions. Statistical developed in relation to the data being analysed.
methods are

In addition, statistical methods are coherent and rigorous. It has strong theoretical foundations
and interpretations that require a strong statistical knowledge.

Machine learning, comparatively, has less assumptions and requires less statistical knowledge.
But, it often requires interaction with various tools to automate the process of learning.
Nevertheless, there is a school of thought that machine learning is just the latest version of old
Statistics' and hence this relationship should be recognized.

1.4 TYPES OF MACHINE LEARNING

What does the word 'learn' mean? Learning, like adaptation, occurs as the result of interaction of
the program with its environment. It can be compared with the interaction between a teacher and
a student. There are four types of machine learning as shown in Figure 1.5.

Machine
learning

Supervised Unsupervised Semi-supervised Reinforcement

learning learning learning learning

Cluster Association Dimension

Classification Regression
analysis mining reduction

Figure 1.5: Types of Machine Learning

Before discussing the types of learning, it is necessary to discuss about data.

Labelled and Unlabelled Data Data is a raw fact. Normally, data is represented in the form
of a table. Data also can be referred to point, sample, or an example. Each row of the
as a data
table represents a data point. Features are attributes or characteristics of an object. Normally, the
columns of the table are attributes. Out of all attributes, one attribute is important and is called a
label. Label is the feature that we aim to predict. Thus, there are two types of data labelled and -

unlabelled.

Labelled Data To illustrate labelled data, let us take one example dataset called Iris flower dataset
or Fisher's Iris dataset. The dataset has 50 samples of Iris with four attributes, length and width
-

of sepals and petals. The target variable is called class. There are three classes Iris setosa, Iris -

virginica, and Iris versicolor.

The partial data of Iris dataset is shown in Table 1.1.

8. Machine Learning

Table 1.1: Iris Flower Dataset

[Link]. Length of Width of Length of Width of Class

Petal Petal Sepal Sepal
1. 5.5 4.2 1.4 0.2 Setosa

2. 7 3.2 4.7 1.4 Versicolor

3. 7.3 2.9 6.3 1.8 Virginica

A dataset need not be always numbers. It can be images or video frames. Deep neural networks
can handle images with labels. In the following Figure 1.6, the deep neural network takes images of

dogs and cats with labels for classification.

Input Label

dog

Cat

(a)

(b)

Figure 1.6: (a) Labelled Dataset (b) Unlabelled Dataset

In unlabelled data, there are no labels in the dataset.

1.4.1 Supervised Learning

Supervised algorithms use labelled dataset. As the name suggests, there is a supervisor or teacher
component in supervised learning. A supervisor provides labelled data so that the model is
constructed and generates test data.

Insupervised learning algorithms, learning takes place in two stages. In layman terms, during the
first stage, the teacher communicates the information to the student that the student is supposed to
master. The student receives the information and understands it. During this stage, the teacher has no
knowledge of whether the information is grasped by the student.
This leads to the second stage of learning. The teacher then asks the student a set of questions
to find out how much information has been grasped by the student. Based on these questions,