Unit-5

Unit-5 (Expert Systems)

Introduction, Knowledge acquisition, Knowledge base, Working memory,
Inference engine, Expert system shells, Explanation, Application of expert
systems. Fundamentals of Neural Networks: Introduction and research history,
Model of artificial neuron, Characteristics of neural networks, learning methods
in neural networks, Single-layer neural network system, Applications of neural
networks. Fundamentals of Genetic Algorithms: Introduction, Encoding,
Operators of genetic algorithm, Basic genetic algorithm.

What is an Expert System?

An expert system is a computer program that is designed to solve complex
problems and to provide decision-making ability like a human expert. It
performs this by extracting knowledge from its knowledge base using the
reasoning and inference rules according to the user queries.
The expert system is a part of AI, and the first ES was developed in the year
1970, which was the first successful approach of artificial intelligence. It solves
the most complex issue as an expert by extracting the knowledge stored in its
knowledge base. The system helps in decision making for compsex problems
using both facts and heuristics like a human expert. It is called so because it
contains the expert knowledge of a specific domain and can solve any complex
problem of that particular domain. These systems are designed for a specific
domain, such as medicine, science, etc.

The performance of an expert system is based on the expert's knowledge

stored in its knowledge base. The more knowledge stored in the KB, the more
that system improves its performance. One of the common examples of an ES
is a suggestion of spelling errors while typing in the Google search box.
Below is the block diagram that represents the working of an expert system:

Unit-5 1
 Note: It is important to remember that an expert system is not
used to replace the human experts; instead, it is used to assist
the human in making a complex decision. These systems do
not have human capabilities of thinking and work on the basis
of the knowledge base of the particular domain.
Below are some popular examples of the Expert System:

DENDRAL: It was an artificial intelligence project that was made as a

chemical analysis expert system. It was used in organic chemistry to detect
unknown organic molecules with the help of their mass spectra and
knowledge base of chemistry.

MYCIN: It was one of the earliest backward chaining expert systems that
was designed to find the bacteria causing infections like bacteraemia and
meningitis. It was also used for the recommendation of antibiotics and the
diagnosis of blood clotting diseases.

PXDES: It is an expert system that is used to determine the type and level
of lung cancer. To determine the disease, it takes a picture from the upper
body, which looks like the shadow. This shadow identifies the type and
degree of harm.

CaDeT: The CaDet expert system is a diagnostic support system that can
detect cancer at early stages.

Characteristics of Expert System

High Performance: The expert system provides high performance for

solving any type of complex problem of a specific domain with high
efficiency and accuracy.

Unit-5 2
 Understandable: It responds in a way that can be easily understandable by
the user. It can take input in human language and provides the output in the
same way.

Reliable: It is much reliable for generating an efficient and accurate output.

Highly responsive: ES provides the result for any complex query within a
very short period of time.

Components of Expert System

An expert system mainly consists of three components:

User Interface

Inference Engine

Knowledge Base

1. User Interface
With the help of a user interface, the expert system interacts with the user,
takes queries as an input in a readable format, and passes it to the inference
engine. After getting the response from the inference engine, it displays the

Unit-5 3
 output to the user. In other words, it is an interface that helps a non-expert
user to communicate with the expert system to find a solution.

2. Inference Engine(Rules of Engine)

The inference engine is known as the brain of the expert system as it is the
main processing unit of the system. It applies inference rules to the
knowledge base to derive a conclusion or deduce new information. It helps
in deriving an error-free solution of queries asked by the user.

With the help of an inference engine, the system extracts the knowledge
from the knowledge base.

There are two types of inference engine:

Deterministic Inference engine: The conclusions drawn from this type of

inference engine are assumed to be true. It is based on facts and rules.

Probabilistic Inference engine: This type of inference engine contains

uncertainty in conclusions, and based on the probability.

Inference engine uses the below modes to derive the solutions:

Forward Chaining: It starts from the known facts and rules, and applies the
inference rules to add their conclusion to the known facts.

Backward Chaining: It is a backward reasoning method that starts from the

goal and works backward to prove the known facts.

3. Knowledge Base
The knowledgebase is a type of storage that stores knowledge acquired
from the different experts of the particular domain. It is considered as big
storage of knowledge. The more the knowledge base, the more precise will
be the Expert System.

It is similar to a database that contains information and rules of a particular

domain or subject.

One can also view the knowledge base as collections of objects and their
attributes. Such as a Lion is an object and its attributes are it is a mammal, it
is not a domestic animal, etc.

Components of Knowledge Base

Unit-5 4
 Factual Knowledge: The knowledge which is based on facts and accepted
by knowledge engineers comes under factual knowledge.

Heuristic Knowledge: This knowledge is based on practice, the ability to

guess, evaluation, and experiences.

Knowledge Representation: It is used to formalize the knowledge stored in the

knowledge base using the If-else rules.
Knowledge Acquisitions: It is the process of extracting, organizing, and
structuring the domain knowledge, specifying the rules to acquire the
knowledge from various experts, and store that knowledge into the knowledge
base.

Development of Expert System

Here, we will explain the working of an expert system by taking an example of
MYCIN ES. Below are some steps to build an MYCIN:

Firstly, ES should be fed with expert knowledge. In the case of MYCIN,

human experts specialized in the medical field of bacterial infection,
provide information about the causes, symptoms, and other knowledge in
that domain.

The KB of the MYCIN is updated successfully. In order to test it, the doctor
provides a new problem to it. The problem is to identify the presence of the
bacteria by inputting the details of a patient, including the symptoms,
current condition, and medical history.

The ES will need a questionnaire to be filled by the patient to know the

general information about the patient, such as gender, age, etc.

Now the system has collected all the information, so it will find the solution
for the problem by applying if-then rules using the inference engine and
using the facts stored within the KB.

In the end, it will provide a response to the patient by using the user
interface.

Participants in the development of Expert System

There are three primary participants in the building of Expert System:

1. Expert: The success of an ES much depends on the knowledge provided

by human experts. These experts are those persons who are specialized in
that specific domain.

Unit-5 5
 2. Knowledge Engineer: Knowledge engineer is the person who gathers the
knowledge from the domain experts and then codifies that knowledge to
the system according to the formalism.

3. End-User: This is a particular person or a group of people who may not be

experts, and working on the expert system needs the solution or advice for
his queries, which are complex.

Why Expert System?

Before using any technology, we must have an idea about why to use that
technology and hence the same for the ES. Although we have human experts in
every field, then what is the need to develop a computer-based system. So
below are the points that are describing the need of the ES:

1. No memory Limitations: It can store as much data as required and can

memorize it at the time of its application. But for human experts, there are
some limitations to memorize all things at every time.

2. High Efficiency: If the knowledge base is updated with the correct

knowledge, then it provides a highly efficient output, which may not be
possible for a human.

Unit-5 6
 3. Expertise in a domain: There are lots of human experts in each domain,
and they all have different skills, different experiences, and different skills,
so it is not easy to get a final output for the query. But if we put the
knowledge gained from human experts into the expert system, then it
provides an efficient output by mixing all the facts and knowledge

4. Not affected by emotions: These systems are not affected by human

emotions such as fatigue, anger, depression, anxiety, etc.. Hence the
performance remains constant.

5. High security: These systems provide high security to resolve any query.

6. Considers all the facts: To respond to any query, it checks and considers
all the available facts and provides the result accordingly. But it is possible
that a human expert may not consider some facts due to any reason.

7. Regular updates improve the performance: If there is an issue in the result

provided by the expert systems, we can improve the performance of the
system by updating the knowledge base.

Capabilities of the Expert System

Below are some capabilities of an Expert System:

Advising: It is capable of advising the human being for the query of any
domain from the particular ES.

Provide decision-making capabilities: It provides the capability of decision

making in any domain, such as for making any financial decision, decisions
in medical science, etc.

Demonstrate a device: It is capable of demonstrating any new products

such as its features, specifications, how to use that product, etc.

Problem-solving: It has problem-solving capabilities.

Explaining a problem: It is also capable of providing a detailed description

of an input problem.

Interpreting the input: It is capable of interpreting the input given by the

user.

Predicting results: It can be used for the prediction of a result.

Diagnosis: An ES designed for the medical field is capable of diagnosing a

disease without using multiple components as it already contains various

Unit-5 7
 inbuilt medical tools.

Advantages of Expert System

These systems are highly reproducible.

They can be used for risky places where the human presence is not safe.

Error possibilities are less if the KB contains correct knowledge.

The performance of these systems remains steady as it is not affected by

emotions, tension, or fatigue.

They provide a very high speed to respond to a particular query.

Limitations of Expert System

The response of the expert system may get wrong if the knowledge base
contains the wrong information.

Like a human being, it cannot produce a creative output for different

scenarios.

Its maintenance and development costs are very high.

Knowledge acquisition for designing is much difficult.

For each domain, we require a specific ES, which is one of the big
limitations.

It cannot learn from itself and hence requires manual updates.

Applications of Expert System

In designing and manufacturing domainIt can be broadly used for
designing and manufacturing physical devices such as camera lenses and
automobiles.

In the knowledge domainThese systems are primarily used for publishing

the relevant knowledge to the users. The two popular ES used for this
domain is an advisor and a tax advisor.

In the finance domainIn the finance industries, it is used to detect any type
of possible fraud, suspicious activity, and advise bankers that if they should
provide loans for business or not.

In the diagnosis and troubleshooting of devicesIn medical diagnosis, the

ES system is used, and it was the first area where these systems were

Unit-5 8
 used.

Planning and SchedulingThe expert systems can also be used for planning
and scheduling some particular tasks for achieving the goal of that task.

Working memory is a cognitive system with a limited capacity that can hold
information temporarily.[1] It is important for reasoning and the guidance of
decision-making and behavior.[2][3] Working memory is often used
synonymously with short-term memory, but some theorists consider the two
forms of memory distinct, assuming that working memory allows for the
manipulation of stored information, whereas short-term memory only refers to
the short-term storage of information.[2][4] Working memory is a theoretical
concept central to cognitive psychology, neuropsychology, and neuroscience.

Forward Chaining and backward chaining

in AI
In artificial intelligence, forward and backward chaining is one of the important
topics, but before understanding forward and backward chaining lets first
understand that from where these two terms came.

Inference engine:
The inference engine is the component of the intelligent system in artificial
intelligence, which applies logical rules to the knowledge base to infer new
information from known facts. The first inference engine was part of the expert
system. Inference engine commonly proceeds in two modes, which are:

1. Forward chaining

2. Backward chaining

Horn Clause and Definite clause:

Horn clause and definite clause are the forms of sentences, which enables
knowledge base to use a more restricted and efficient inference algorithm.
Logical inference algorithms use forward and backward chaining approaches,
which require KB in the form of the first-order definite clause.
Backward Skip 10sPause VideoForward Skip 10s

Unit-5 9
 Definite clause: A clause which is a disjunction of literals with exactly one
positive literal is known as a definite clause or strict horn clause.

Horn clause: A clause which is a disjunction of literals with at most one

positive literal is known as horn clause. Hence all the definite clauses are horn
clauses.

Example: (¬ p V ¬ q V k). It has only one positive literal k.

It is equivalent to p ∧ q → k.

A. Forward Chaining
Forward chaining is also known as a forward deduction or forward reasoning
method when using an inference engine. Forward chaining is a form of
reasoning which start with atomic sentences in the knowledge base and
applies inference rules (Modus Ponens) in the forward direction to extract more
data until a goal is reached.
The Forward-chaining algorithm starts from known facts, triggers all rules
whose premises are satisfied, and add their conclusion to the known facts. This
process repeats until the problem is solved.
Properties of Forward-Chaining:

It is a down-up approach, as it moves from bottom to top.

It is a process of making a conclusion based on known facts or data, by

starting from the initial state and reaches the goal state.

Forward-chaining approach is also called as data-driven as we reach to the

goal using available data.

Forward -chaining approach is commonly used in the expert system, such

as CLIPS, business, and production rule systems.

Consider the following famous example which we will use in both approaches:

Example:
"As per the law, it is a crime for an American to sell weapons to hostile
nations. Country A, an enemy of America, has some missiles, and all the
missiles were sold to it by Robert, who is an American citizen."

Prove that "Robert is criminal."

Unit-5 10
 To solve the above problem, first, we will convert all the above facts into first-
order definite clauses, and then we will use a forward-chaining algorithm to
reach the goal.

Facts Conversion into FOL:

It is a crime for an American to sell weapons to hostile nations. (Let's say p,
∧ ∧
q, and r are variables)American (p) weapon(q) sells (p, q, r) ∧
hostile(r) → Criminal(p) ...(1)

Country A has some missiles. ?p Owns(A, p) ∧ Missile(p). It can be written

in two definite clauses by using Existential Instantiation, introducing new
Constant [Link](A, T1) ......(2)Missile(T1) .......(3)

All of the missiles were sold to country A by Robert.?p Missiles(p) ∧ Owns

(A, p) → Sells (Robert, p, A) ......(4)

Missiles are [Link](p) → Weapons (p) .......(5)

Enemy of America is known as [Link](p, America) →Hostile(p)

........(6)

Country A is an enemy of [Link] (A, America) .........(7)

Robert is AmericanAmerican(Robert). ..........(8)

Forward chaining proof:

Step-1:
In the first step we will start with the known facts and will choose the sentences
which do not have implications, such as: American(Robert), Enemy(A,
America), Owns(A, T1), and Missile(T1). All these facts will be represented as
below.

Step-2:
At the second step, we will see those facts which infer from available facts and
with satisfied premises.
Rule-(1) does not satisfy premises, so it will not be added in the first iteration.

Unit-5 11
 Rule-(2) and (3) are already added.
Rule-(4) satisfy with the substitution {p/T1}, so Sells (Robert, T1, A) is added,
which infers from the conjunction of Rule (2) and (3).
Rule-(6) is satisfied with the substitution(p/A), so Hostile(A) is added and which
infers from Rule-(7).

Step-3:

At step-3, as we can check Rule-(1) is satisfied with the substitution {p/Robert,

q/T1, r/A}, so we can add Criminal(Robert) which infers all the available facts.
And hence we reached our goal statement.

Hence it is proved that Robert is Criminal using forward chaining approach.

B. Backward Chaining:
Backward-chaining is also known as a backward deduction or backward
reasoning method when using an inference engine. A backward chaining

Unit-5 12
 algorithm is a form of reasoning, which starts with the goal and works
backward, chaining through rules to find known facts that support the goal.
Properties of backward chaining:

It is known as a top-down approach.

Backward-chaining is based on modus ponens inference rule.

In backward chaining, the goal is broken into sub-goal or sub-goals to

prove the facts true.

It is called a goal-driven approach, as a list of goals decides which rules are

selected and used.

Backward -chaining algorithm is used in game theory, automated theorem

proving tools, inference engines, proof assistants, and various AI
applications.

The backward-chaining method mostly used a depth-first search strategy

for proof.

Example:
In backward-chaining, we will use the same above example, and will rewrite all
the rules.

American (p) ∧ weapon(q) ∧ sells (p, q, r) ∧ hostile(r) → Criminal(p) ...

(1)Owns(A, T1) ........(2)

Missile(T1)

?p Missiles(p) ∧ Owns (A, p) → Sells (Robert, p, A) ......(4)

Missile(p) → Weapons (p) .......(5)

Enemy(p, America) →Hostile(p) ........(6)

Enemy (A, America) .........(7)

American(Robert). ..........(8)

Backward-Chaining proof:
In Backward chaining, we will start with our goal predicate, which
is Criminal(Robert), and then infer further rules.
Step-1:

Unit-5 13
 At the first step, we will take the goal fact. And from the goal fact, we will infer
other facts, and at last, we will prove those facts true. So our goal fact is
"Robert is Criminal," so following is the predicate of it.

Step-2:
At the second step, we will infer other facts form goal fact which satisfies the
rules. So as we can see in Rule-1, the goal predicate Criminal (Robert) is
present with substitution {Robert/P}. So we will add all the conjunctive facts
below the first level and will replace p with Robert.
Here we can see American (Robert) is a fact, so it is proved here.

Step-3:t At step-3, we will extract further fact Missile(q) which infer from
Weapon(q), as it satisfies Rule-(5). Weapon (q) is also true with the substitution
of a constant T1 at q.

Unit-5 14
 Step-4:

At step-4, we can infer facts Missile(T1) and Owns(A, T1) form Sells(Robert, T1,
r) which satisfies the Rule- 4, with the substitution of A in place of r. So these
two statements are proved here.

Step-5:

Unit-5 15
 At step-5, we can infer the fact Enemy(A, America) from Hostile(A) which
satisfies Rule- 6. And hence all the statements are proved true using backward
chaining.

An Introduction to Expert System Shells

A Brief Overview ES Shell Structure and Comparison
of Popular Expert System Shells

Expert System is a classical symbolic Artificial Intelligence approach which is

one of the best examples to represent Artificial Cognitive Systems (ACS).
Today, though the standalone products like DENDRAL (1960, ACS
only), DART (1991, ACS only), IBM’s Cognitive Computer Watson (2011,
ML+ACS), etc. are rare, the concepts and implementation of inferencing and
knowledge representation of expert systems are still in use in modern AI
systems. For example the ROSS (2017): ‘the first artificial intelligence-powered
attorney’ built on IBM’s Watson can understand language, postulate
hypotheses, and backup the generated conclusions with references and

Unit-5 16
 citations. Hence the study about expert systems and ACSs cannot be thrown
away.
However, the following article is not about the expert system; it is about
building them.
In order to build expert systems easily, it is recommended to employ an expert
system shell that saves both time and effort. Simply an expert system shell (ES
Shell) is a toolkit that consists of built-in essential components of a typical
expert system. Hence, this article aims to describe the structure and
application of ES Shell in building expert systems and the benefits and
limitations of popular shells.
The content of this article is organized as follows:

1. Expert Systems

2. Expert System Shells

3. ES Shell structure

4. Popular ES Shells

5. Conclusion

6. References

1. Expert Systems
An intelligent program that can mimic the problem-solving behavior of a human
expert can be identified as an Expert System. Humans knowledge consists of
subject-specific knowledge/ domain knowledge and problem-solving
knowledge. The functionality of an expert system is similar to a human expert
who solves a problem by applying his problem-solving knowledge to his
domain-specific knowledge. Hence, an expert system is a computer program
that symbolizes the knowledge of an expert in a certain domain.

Following are key features of an expert system:

Operates on a specific domain.

Dominates the question asking process.

Process incomplete information.

Provide certainty of an answer given by the expert system.

Process alternative solutions.

Unit-5 17
 Provide reasons for answers

Expert Systems with the above features can be developed from scratch by
using Logic Programming language like ProLog, or can easily be implemented
by using an expert system shell.

2. Expert System Shells

Expert system shells are toolkits that can be used to develop expert systems.
They consist of some built expert system components with an empty
knowledge base. Hence, in most cases, the knowledge engineer is left with
only populating the knowledge base. It is essentially a special-purpose tool that
is built-in in line with the requirements and standards of a particular domain or
expert-knowledge area applications. It may be defined as a software package
that facilitates the building of knowledge-based expert systems by providing a
knowledge representation scheme and an inference engine.
Difference ES shells offer various ways to model the knowledge into the
knowledge base. They are
1. as rules
Knowledge is populated using rules. These rules can be programmed by
Prolog, Java, Python or any other preferred language based on the expert
system shell being used
2. in the form of a decision tree

Unit-5 18
 Figure 02: A tree-based expert system created based on the rules mentioned
above ( Source: Created by Author)
3. as objects (frames) — A data structure with typical knowledge about a
particular object or concept.

ES shell is an ideal option for the rapid development of expert systems. Due to
this reason, it can be considered as one of the most commercially viable
approaches in expert system implementation.

3. Expert System Shell Structure

The Expert System Shell refers to a software module containing an:

1. User interface (built-in)

2. Inference engine (built-in)

3. A structured skeleton of a knowledge base (in its empty state) with the
suitable knowledge representation facilities

Not only the above components but also some ES Shells provide facilities for
database connectivity through interpreter, web integration, and natural

Unit-5 19
 language processing (NLP) features.

Figure 03: Illustration by Author (Reference: Expert System Architecture

([Link]))
The user interface is the portal available for both end-users (who use the
expert system to get solutions) and the knowledge engineer (who perform the
knowledge engineering and modelling).
The knowledge base can be connected with an external database (like MySQL)
since the knowledge base is not optimal in storing extensive data. The
knowledge base cannot directly access the database and these access
features are mediated through an interpreter.
Some ES Shells have inbuilt knowledge base editors which facilitate the
Knowledge Engineer to easily update and check the knowledge base.
Knowledge Engineer collects the expertise knowledge in a specific domain and
models in populating the knowledge base.

Inference engine which is the most important part of an expert system access
the knowledge base and solves the problem by either backward chaining or
forward chaining of facts and rules in the knowledge base. In ES Shells, the
inference engine is also a built-in component that is usually programmed in
ProLog.

Unit-5 20
 Most ES shells are composed of another component called ‘Explanation
System’ which provides the user with reasons and explanations to provide a
certain answer, by considering the ‘case specification data’ available.
In an expert system shell, the design of the user interface and other software
components are programmed by the software engineer. Therefore an expert
system is a collaborative design of 03 major parties: expert, knowledge
engineer and software engineer. (Depending on the size these parties may vary
from individuals to large teams)

4. Popular Expert System shells

Following are a few examples of expert systems available for developers to
build expert systems at various scales. Depending on the ES Shell there are
various pros and cons associated with each shell.

4.1 ES-Builder
ES Builder stands for Expert System Builder. It is one of the most famous Expert
System Shell. It is a free ES shell that is implemented especially for students
and researchers to develop expert system shells. The shell consists of an
improved web interface built using the AJAX framework. The ES-Builder stores
the facts and rules of the knowledge base in an online MySQL database. There
is a built-in inference engine (written in Prolog)and user interfaces are
developed using simple HTML and CSS. The database connectivity is
maintained to access the knowledge base using PHP.
The rule base knowledge base can also be developed using a decision tree that
consists of attributes, values and conclusions.

Attribute: Attributes are characteristics of possible conclusions that are to

be tested in the ES.

Value: Each Value node represents the most correct response to an

Attribute for a particular conclusion.

Conclusion: The final decision made based on the attributes and values is
the conclusion.

Benefits of ES Builder:

Autogenerated User Interfaces (Simple HTML CSS design of the user

interface)

Unit-5 21
 Ability to create backup files of the knowledge base and easy integration
with MYSQL database.

Ability populates the knowledge base using the decision tree.

Optimized view of the rule-based knowledge base.

Limitations of ES-Builder:

Only forward chaining rules are supported backward chaining rules are not
supported.

Desktop tools support only Windows 98 & Windows 7.

4.2 CLIPS
CLIPS stands for C-Language Integrated Production System. As the meaning
suggests the CLIPS expert system shell is written in the procedural langue C.
CLIPS was developed in 1985 at NASA’s Johnson Space Center. It is a rule-
based programming language that is used for creating an expert system. CLIPS
are used in systems where the heuristic solution is easier to implement and
maintain than a traditional algorithmic approach.
CLIPS provides 03 different tools for knowledge representation in the form of
programming methodologies/ programming paradigms. The 03 methods are:

Procedural

Object-oriented

Rule-based programming

Since the CLIPS is written in C language, the expert system developed by

CLIPS requires ANSI compiler also. Since the systems that already have the
ANSI compiler can easily run the expert system without changing the source
code.
Benefits of CLIPS:

Can be easily embedded in other subroutines made in procedural

languages to extend the existing functionality.

Support the methods like fuzzy logic and agents.

Limitations of CLIPS:

Support only forward chaining rules but does not support backward
chaining rules.

Unit-5 22
 4.3 JESS
JESS stands for Java Expert System Shell. JESS is another expert system shell
and a scripting language that is completely written by Java programming
language. JESS is heavily used to build rule-based expert systems which is
tightly linked to systems that are written on Java. The expert systems built by
using JESS can be either run on command-line interface or can be used as an
Applet.
JESS based expert systems are widely used as intelligent agents, in enterprise
resource planning systems, and in order validation of eCommerce platforms.
Benefits of JESS:

Better to be applied in bigger problems where performance is dominated by

algorithm quality.

Include backward chaining and working memory queries.

Faster than the expert systems developed using expert system shells
written on C language.

Contains some commands which allow less memory usage when executing
the system.

Limitations of JESS:

Some capabilities are limited when used in Browser.

4.4 PyKE
PyKE stands for Python Knowledge Engine. PyKE uses logic programming that
is inspired by Prolog, but PyKE is entirely written in the programming language
Python. The python functions, PyKE rules, and PyKE pattern variables, graph
plans are the major features of PyKE knowledge base. This is an inference
engine that uses rules and facts to create additional facts using forward
chaining of rules to prove goals.

And also, through backward chaining, it assembles Python functions into

customizable call graphs which are also known as Plans.
Benefits of PyKE:

Highly customizable and code adaptability.

High performance.

Unit-5 23
 Code reusability.

Both forward chaining and backward chaining is supported.

Ability to run commands on the local system to guide the inferencing.

Provide end-user question capability.

Limitations of PyKE:

The UI should be created manually.

Integration issues occur when integrating inference engines with different

UIs.

5. Conclusion
It is obvious that ES Shells is a far better option rather than developing an
expert system from the scratch. There are many ES Shells that support different
programming languages that have been developed which open gates for
developers who are familiar with various programming languages. The ES
Shells should be properly selected mainly considering the requirements and the
scale/size/complexity of the expert system to be developed.

Artificial Neural Network Tutorial

Artificial Neural Network Tutorial provides basic and advanced concepts of
ANNs. Our Artificial Neural Network tutorial is developed for beginners as well
as professions.
The term "Artificial neural network" refers to a biologically inspired sub-field of
artificial intelligence modeled after the brain. An Artificial neural network is
usually a computational network based on biological neural networks that
construct the structure of the human brain. Similar to a human brain has
neurons interconnected to each other, artificial neural networks also have
neurons that are linked to each other in various layers of the networks. These
neurons are known as nodes.
Artificial neural network tutorial covers all the aspects related to the artificial
neural network. In this tutorial, we will discuss ANNs, Adaptive resonance
theory, Kohonen self-organizing map, Building blocks, unsupervised learning,
Genetic algorithm, etc.

Unit-5 24
 What is Artificial Neural Network?
The term "Artificial Neural Network" is derived from Biological neural
networks that develop the structure of a human brain. Similar to the human
brain that has neurons interconnected to one another, artificial neural networks
also have neurons that are interconnected to one another in various layers of
the networks. These neurons are known as nodes.

The given figure illustrates the typical diagram of Biological Neural Network.
The typical Artificial Neural Network looks something like the given figure.

Dendrites from Biological Neural Network represent inputs in Artificial Neural

Networks, cell nucleus represents Nodes, synapse represents Weights, and
Axon represents Output.

Unit-5 25
 Relationship between Biological neural network and artificial neural network:

Biological Neural Network Artificial Neural Network

Dendrites Inputs

Cell nucleus Nodes

Synapse Weights

Axon Output

An Artificial Neural Network in the field of Artificial intelligence where it

attempts to mimic the network of neurons makes up a human brain so that
computers will have an option to understand things and make decisions in a
human-like manner. The artificial neural network is designed by programming
computers to behave simply like interconnected brain cells.
There are around 1000 billion neurons in the human brain. Each neuron has an
association point somewhere in the range of 1,000 and 100,000. In the human
brain, data is stored in such a manner as to be distributed, and we can extract
more than one piece of this data when necessary from our memory parallelly.
We can say that the human brain is made up of incredibly amazing parallel
processors.
We can understand the artificial neural network with an example, consider an
example of a digital logic gate that takes an input and gives an output. "OR"
gate, which takes two inputs. If one or both the inputs are "On," then we get
"On" in output. If both the inputs are "Off," then we get "Off" in output. Here the
output depends upon input. Our brain does not perform the same task. The
outputs to inputs relationship keep changing because of the neurons in our
brain, which are "learning."

The architecture of an artificial neural

network:
To understand the concept of the architecture of an artificial neural network,
we have to understand what a neural network consists of. In order to define a
neural network that consists of a large number of artificial neurons, which are
termed units arranged in a sequence of layers. Lets us look at various types of
layers available in an artificial neural network.
Artificial Neural Network primarily consists of three layers:

Unit-5 26
 Input Layer:

As the name suggests, it accepts inputs in several different formats provided by

the programmer.
Hidden Layer:
The hidden layer presents in-between input and output layers. It performs all
the calculations to find hidden features and patterns.
Output Layer:
The input goes through a series of transformations using the hidden layer,
which finally results in output that is conveyed using this layer.
The artificial neural network takes input and computes the weighted sum of the
inputs and includes a bias. This computation is represented in the form of a
transfer function.

It determines weighted total is passed as an input to an activation function to

produce the output. Activation functions choose whether a node should fire or
not. Only those who are fired make it to the output layer. There are distinctive
activation functions available that can be applied upon the sort of task we are
performing.

Unit-5 27
 Advantages of Artificial Neural Network
(ANN)
Parallel processing capability:
ADVERTISEMENT
Artificial neural networks have a numerical value that can perform more than
one task simultaneously.

Storing data on the entire network:

Data that is used in traditional programming is stored on the whole network, not
on a database. The disappearance of a couple of pieces of data in one place
doesn't prevent the network from working.
Capability to work with incomplete knowledge:
After ANN training, the information may produce output even with inadequate
data. The loss of performance here relies upon the significance of missing
data.
Having a memory distribution:
For ANN is to be able to adapt, it is important to determine the examples and to
encourage the network according to the desired output by demonstrating these
examples to the network. The succession of the network is directly proportional
to the chosen instances, and if the event can't appear to the network in all its
aspects, it can produce false output.
Having fault tolerance:
Extortion of one or more cells of ANN does not prohibit it from generating
output, and this feature makes the network fault-tolerance.

Disadvantages of Artificial Neural

Network:
Assurance of proper network structure:
There is no particular guideline for determining the structure of artificial neural
networks. The appropriate network structure is accomplished through
experience, trial, and error.
Unrecognized behavior of the network:

Unit-5 28
 It is the most significant issue of ANN. When ANN produces a testing solution,
it does not provide insight concerning why and how. It decreases trust in the
network.
Hardware dependence:
Artificial neural networks need processors with parallel processing power, as
per their structure. Therefore, the realization of the equipment is dependent.
Difficulty of showing the issue to the network:

ANNs can work with numerical data. Problems must be converted into
numerical values before being introduced to ANN. The presentation mechanism
to be resolved here will directly impact the performance of the network. It relies
on the user's abilities.
The duration of the network is unknown:
The network is reduced to a specific value of the error, and this value does not
give us optimum results.

Science artificial neural networks that have steeped into the

world in the mid-20th century are exponentially developing.
In the present time, we have investigated the pros of artificial
neural networks and the issues encountered in the course of
their utilization. It should not be overlooked that the cons of
ANN networks, which are a flourishing science branch, are
eliminated individually, and their pros are increasing day by
day. It means that artificial neural networks will turn into an
irreplaceable part of our lives progressively important.

History of Artificial Neural Network

The history of neural networking arguably began in the late 1800s with
scientific endeavors to study the activity of the human brain. In 1890, William
James published the first work about brain activity patterns.
In 1943, McCulloch and Pitts created a model of the neuron that is still used
today in an artificial neural network. This model is segmented in two parts

A summation over-weighted inputs.

Unit-5 29
 An output function of the sum.

Artificial Neural Network (ANN):

In 1949, Donald Hebb published "The Organization of Behavior," which
illustrated a law for synaptic neuron learning. This law, later known as Hebbian
Learning in honor of Donald Hebb, is one of the most straight-forward and
simple learning rules for artificial neural networks.
In 1951, Narvin Minsky made the first Artificial Neural Network (ANN) while
working at Princeton.
In 1958, "The Computer and the Brain" were published, a year after Jhon von
Neumann's death. In that book, von Neumann proposed numerous extreme
changes to how analysts had been modeling the brain.

Perceptron:
Perceptron was created in 1958, at Cornell University by Frank Rosenblatt.
The perceptron was an endeavor to use neural network procedures for
character recognition. Perceptron was a linear system and was valuable for
solving issues where the input classes were linearly separable in the input
space. In 1960, Rosenblatt published the book principles of neurodynamics,
containing a bit of his research and ideas about modeling the brain.
Despite the early accomplishment of the perceptron and artificial neural
network research, there were many individuals who felt that there was a
constrained guarantee in these methods. Among these were Marvin Minsky
and Seymour Papert, whose 1969 book perceptrons were used to dishonor
ANN research and focus attention on the apparent constraints of ANN work.
One of the limitations that Minsky and Papert's highlight was the fact that the
Perceptron was not capable of distinguishing patterns that are not linearly
separable in input space with a linear classification problem. Regardless of the
disappointment of Perceptron to deal with non-linearly separable data, it was
not an inherent failure of the technology, but a matter of scale. Hecht-Nielsen
showed a two-layer perceptron (Mark) in 1990 that is a three-layer machine
that was equipped for tackling non-linear separation problems. Perceptrons
introduced what some call the "quiet years," where ANN research was at a
minimum of interest.
The backpropagation algorithm, initially found by Werbos in 1974, was
rediscovered in 1986 with the book Learning Internal Representation by Error

Unit-5 30
 Propagation by Rumelhart, Hinton, and Williams. Backpropagation is a type of
gradient descent algorithm used with artificial neural networks for reduction
and curve-fitting.
In 1987, the IEEE annual international ANN conference was begun for ANN
scientists. In 1987, the International Neural Network Society(INNS) was
formed, along with INNS neural Networking journal in 1988.

How do artificial neural networks work?

Artificial Neural Network can be best represented as a weighted directed
graph, where the artificial neurons form the nodes. The association between
the neurons outputs and neuron inputs can be viewed as the directed edges
with weights. The Artificial Neural Network receives the input signal from the
external source in the form of a pattern and image in the form of a vector.
These inputs are then mathematically assigned by the notations x(n) for every n
number of inputs.

Afterward, each of the input is multiplied by its corresponding weights ( these

weights are the details utilized by the artificial neural networks to solve a
specific problem ). In general terms, these weights normally represent the

Unit-5 31
 strength of the interconnection between neurons inside the artificial neural
network. All the weighted inputs are summarized inside the computing unit.
If the weighted sum is equal to zero, then bias is added to make the output non-
zero or something else to scale up to the system's response. Bias has the same
input, and weight equals to 1. Here the total of weighted inputs can be in the
range of 0 to positive infinity. Here, to keep the response in the limits of the
desired value, a certain maximum value is benchmarked, and the total of
weighted inputs is passed through the activation function.
The activation function refers to the set of transfer functions used to achieve
the desired output. There is a different kind of the activation function, but
primarily either linear or non-linear sets of functions. Some of the commonly
used sets of activation functions are the Binary, linear, and Tan hyperbolic
sigmoidal activation functions. Let us take a look at each of them in details:

Binary:
In binary activation function, the output is either a one or a 0. Here, to
accomplish this, there is a threshold value set up. If the net weighted input of
neurons is more than 1, then the final output of the activation function is
returned as one or else the output is returned as 0.

Sigmoidal Hyperbolic:
The Sigmoidal Hyperbola function is generally seen as an "S" shaped curve.
Here the tan hyperbolic function is used to approximate output from the actual
net input. The function is defined as:
F(x) = (1/1 + exp(-????x))

Where ???? is considered the Steepness parameter.

Types of Artificial Neural Network:

There are various types of Artificial Neural Networks (ANN) depending upon
the human brain neuron and network functions, an artificial neural network
similarly performs tasks. The majority of the artificial neural networks will have
some similarities with a more complex biological partner and are very effective
at their expected tasks. For example, segmentation or classification.

Feedback ANN:

Unit-5 32
 In this type of ANN, the output returns into the network to accomplish the best-
evolved results internally. As per the University of Massachusetts, Lowell
Centre for Atmospheric Research. The feedback networks feed information
back into itself and are well suited to solve optimization issues. The Internal
system error corrections utilize feedback ANNs.

Feed-Forward ANN:
A feed-forward network is a basic neural network comprising of an input layer,
an output layer, and at least one layer of a neuron. Through assessment of its
output by reviewing its input, the intensity of the network can be noticed based
on group behavior of the associated neurons, and the output is decided. The
primary advantage of this network is that it figures out how to evaluate and
recognize input patterns.

What Does Single-Layer Neural Network

Mean?
A single-layer neural network represents the most simple form of neural
network, in which there is only one layer of input nodes that send weighted
inputs to a subsequent layer of receiving nodes, or in some cases, one
receiving node. This single-layer design was part of the foundation for systems
which have now become much more complex.
Single-Layer Neural Network
One of the early examples of a single-layer neural network was called a
“perceptron.” The perceptron would return a function based on inputs, again,
based on single neurons in the physiology of the human brain. In some senses,
perceptron models are much like “logic gates” fulfilling individual functions: A
perceptron will either send a signal, or not, based on the weighted inputs.
Another type of single-layer neural network is the single-layer binary linear
classifier, which can isolate inputs into one of two categories.
Single-layer neural networks can also be thought of as part of a class of
feedforward neural networks, where information only travels in one direction,
through the inputs, to the output. Again, this defines these simple networks in
contrast to immensely more complicated systems, such as those that use
backpropagation or gradient descent to function.

Unit-5 33
 8 Applications of Neural Networks
Have you ever wondered what forms the basis of May I know you page that
facebook directed you when you were busy scrolling through or how your
online signatures are verified ?

Remember the crime documentaries where graphologist analyzes murder’s

handwriting for finding the real culprit. Long gone are the days when all these
nitty gritty tasks were in human hands, now artificial intelligence has taken over
these assessments.
In the modern era neural networks are assisting humans to survive the new age
transitions in education, financial, aerospace and automotive sectors. But
before knowing how they are giving different sectors a push, it is first important
to understand the basic concept of neural networks and deep learning.

Deep Learning
Neural networks are a part of deep learning, which comes under the
comprehensive term, artificial intelligence. Neural networks are a set of
algorithms that are modelled after the human brain. These networks are also
known as artificial neural networks (ANN).
Sensory neurons, motor neurons and interneurons form the human
brain. Artificial neurons, form the replica of the human brain (i.e. a neural
network).

Artificial Neural Network (ANN)

Artificial Neural Network (ANN) is a collection of connected units (nodes).

These connected units are known as artificial neurons. These units closely
resemble the original neurons of a human brain. Every node is built with a set
of inputs, weights, and a bias value. Weights of the neural network are held
within the hidden layers.
Weights and biases are learning parameters of machine learning models, they
are modified for training the neural networks.

Applications of Neural Networks

Neural Networks are regulating some key sectors including finance, healthcare,
and automotive. As these artificial neurons function in a way similar to the

Unit-5 34
 human brain. They can be used for image recognition, character recognition
and stock market predictions. Let’s understand the diverse applications of
neural networks

1. Facial Recognition
Facial Recognition Systems are serving as robust systems of surveillance.
Recognition Systems matches the human face and compares it with the digital
images. They are used in offices for selective entries. The systems thus
authenticate a human face and match it up with the list of IDs that are present
in its database.
(Must Check: Facial Recognition Work in Deep Learning?)
Convolutional Neural Networks (CNN) are used for facial recognition and
image processing. Large number of pictures are fed into the database for
training a neural network. The collected images are further processed for
training.
Sampling layers in CNN are used for proper evaluations. Models are optimized
for accurate recognition results.

2. Stock Market Prediction

Investments are subject to market risks. It is nearly impossible to predict the
upcoming changes in the highly volatile stock market. The forever changing
bullish and bearish phases were unpredictable before the advent of neural
networks. But well what changed it all? Neural Networks of course…
To make a successful stock prediction in real time a Multilayer Perceptron
MLP (class of feedforward artificial intelligence algorithm) is employed. MLP
comprises multiple layers of nodes, each of these layers is fully connected to
the succeeding nodes. Stock’s past performances, annual returns, and non
profit ratios are considered for building the MLP model.
Check out this video to know how the LTSM model is built for making
predictions in the stock market.

3. Social Media
No matter how cliche it may sound, social media has altered the normal boring
course of life. Artificial Neural Networks are used to study the behaviours of
social media users. Data shared everyday via virtual conversations is tacked up
and analyzed for competitive analysis.

Unit-5 35
 Neural networks duplicate the behaviours of social media users. Post analysis
of individuals' behaviours via social media networks the data can be linked to
people’s spending habits. Multilayer Perceptron ANN is used to mine data from
social media applications.
MLP forecasts social media trends, it uses different training methods like Mean
Absolute Error (MAE), Root Mean Squared Error (RMSE), and Mean Squared
Error (MSE). MLP takes into consideration several factors like user’s favourite
instagram pages, bookmarked choices etc. These factors are considered as
inputs for training the MLP model.
In the ever changing dynamics of social media applications, artificial neural
networks can definitely work as the best fit model for user data analysis.

4. Aerospace
Aerospace Engineering is an expansive term that covers developments in
spacecraft and aircraft. Fault diagnosis, high performance auto piloting,
securing the aircraft control systems, and modeling key dynamic simulations
are some of the key areas that neural networks have taken over. Time delay
Neural networks can be employed for modelling non linear time dynamic
systems.
Time Delay Neural Networks are used for position independent feature
recognition. The algorithm thus built based on time delay neural networks can
recognize patterns. (Recognizing patterns are automatically built by neural
networks by copying the original data from feature units).
Other than this TNN are also used to provide stronger dynamics to the NN
models. As passenger safety is of utmost importance inside an aircraft,
algorithms built using the neural network systems ensures the accuracy in the
autopilot system. As most of the autopilot functions are automated, it is
important to ensure a way that maximizes the security.

5. Defence
Defence is the backbone of every country. Every country’s state in the
international domain is assessed by its military operations. Neural Networks
also shape the defence operations of technologically advanced countries. The
United States of America, Britain, and Japan are some countries that use
artificial neural networks for developing an active defence strategy.

Unit-5 36
 Neural networks are used in logistics, armed attack analysis, and for object
location. They are also used in air patrols, maritime patrol, and for controlling
automated drones. The defence sector is getting the much needed kick of
artificial intelligence to scale up its technologies.
Convolutional Neural Networks(CNN), are employed for determining the
presence of underwater mines. Underwater mines are the underpass that serve
as an illegal commute route between two countries. Unmanned Airborne
Vehicle (UAV), and Unmanned Undersea Vehicle (UUV) these autonomous
sea vehicles use convolutional neural networks for the image processing.
Convolutional layers form the basis of Convolutional Neural Networks. These
layers use different filters for differentiating between images. Layers also have
bigger filters that filter channels for image extraction.

6. Healthcare
The age old saying goes like “Health is Wealth”. Modern day individuals are
leveraging the advantages of technology in the healthcare
sector. Convolutional Neural Networks are actively employed in the healthcare
industry for X ray detection, CT Scan and ultrasound.
As CNN is used in image processing, the medical imaging data retrieved from
aforementioned tests is analyzed and assessed based on neural network
models. Recurrent Neural Network (RNN) is also being employed for the
development of voice recognition systems.
(Must Check: Learning Recurrent Neural Network and applications)

Voice recognition systems are used these days to keep track of the patient’s
data. Researchers are also employing Generative Neural Networks for drug
discovery. Matching different categories of drugs is a hefty task, but generative
neural networks have broken down the hefty task of drug discovery. They can
be used for combining different elements which forms the basis of drug
discovery.

7. Signature Verification and Handwriting Analysis

Signature Verification , as the self explanatory term goes, is used for verifying
an individual’s signature. Banks, and other financial institutions use signature
verification to cross check the identity of an individual.
Usually a signature verification software is used to examine the signatures. As
cases of forgery are pretty common in financial institutions, signature

Unit-5 37
 verification is an important factor that seeks to closely examine the authenticity
of signed documents.
Artificial Neural Networks are used for verifying the signatures. ANN are
trained to recognize the difference between real and forged signatures. ANNs
can be used for the verification of both offline and online signatures.
For training an ANN model, varied datasets are fed in the database. The data
thus fed help the ANN model to differentiate. ANN model employs image
processing for extraction of features.
(Related Blog: Hand Gesture Classification using Deep Learning with Keras)
Handwriting analysis plays an integral role in forensics. The analysis is further
used to evaluate the variations in two handwritten documents. The process of
spilling words on a blank sheet is also used for behavioural
analysis. Convolutional Neural Networks (CNN) are used for handwriting
analysis and handwriting verification.

8. Weather Forecasting
The forecasts done by the meteorological department were never accurate
before artificial intelligence came into force. Weather Forecasting is primarily
undertaken to anticipate the upcoming weather conditions beforehand. In the
modern era, weather forecasts are even used to predict the possibilities of
natural disasters.

Genetic Algorithm in Machine Learning

A genetic algorithm is an adaptive heuristic search algorithm inspired by
"Darwin's theory of evolution in Nature." It is used to solve optimization
problems in machine learning. It is one of the important algorithms as it helps
solve complex problems that would take a long time to solve.

Genetic Algorithms are being widely used in different real-world applications,

for example, Designing electronic circuits, code-breaking, image processing,
and artificial creativity.
In this topic, we will explain Genetic algorithm in detail, including basic
terminologies used in Genetic algorithm, how it works, advantages and
limitations of genetic algorithm, etc.

Unit-5 38
 What is a Genetic Algorithm?
Before understanding the Genetic algorithm, let's first understand basic
terminologies to better understand this algorithm:

Population: Population is the subset of all possible or probable solutions,

which can solve the given problem.

Chromosomes: A chromosome is one of the solutions in the population for

the given problem, and the collection of gene generate a chromosome.

Gene: A chromosome is divided into a different gene, or it is an element of

the chromosome.

Allele: Allele is the value provided to the gene within a particular

chromosome.

Fitness Function: The fitness function is used to determine the individual's

fitness level in the population. It means the ability of an individual to
compete with other individuals. In every iteration, individuals are evaluated
based on their fitness function.

Genetic Operators: In a genetic algorithm, the best individual mate to

regenerate offspring better than parents. Here genetic operators play a role
in changing the genetic composition of the next generation.

Selection

After calculating the fitness of every existent in the population, a selection

process is used to determine which of the individualities in the population will
get to reproduce and produce the seed that will form the coming generation.
Types of selection styles available

Roulette wheel selection

Event selection

Rank- grounded selection

So, now we can define a genetic algorithm as a heuristic search algorithm to

solve optimization problems. It is a subset of evolutionary algorithms, which is
used in computing. A genetic algorithm uses genetic and natural selection
concepts to solve optimization problems.

How Genetic Algorithm Work?

Unit-5 39
 The genetic algorithm works on the evolutionary generational cycle to generate
high-quality solutions. These algorithms use different operations that either
enhance or replace the population to give an improved fit solution.
It basically involves five phases to solve the complex optimization problems,
which are given as below:

Initialization

Fitness Assignment

Selection

Reproduction

Termination

1. Initialization
The process of a genetic algorithm starts by generating the set of individuals,
which is called population. Here each individual is the solution for the given
problem. An individual contains or is characterized by a set of parameters
called Genes. Genes are combined into a string and generate chromosomes,
which is the solution to the problem. One of the most popular techniques for
initialization is the use of random binary strings.

2. Fitness Assignment

Unit-5 40
 Fitness function is used to determine how fit an individual is? It means the
ability of an individual to compete with other individuals. In every iteration,
individuals are evaluated based on their fitness function. The fitness function
provides a fitness score to each individual. This score further determines the
probability of being selected for reproduction. The high the fitness score, the
more chances of getting selected for reproduction.

3. Selection
The selection phase involves the selection of individuals for the reproduction of
offspring. All the selected individuals are then arranged in a pair of two to
increase reproduction. Then these individuals transfer their genes to the next
generation.
There are three types of Selection methods available, which are:

Roulette wheel selection

Tournament selection

Rank-based selection

4. Reproduction
After the selection process, the creation of a child occurs in the reproduction
step. In this step, the genetic algorithm uses two variation operators that are
applied to the parent population. The two operators involved in the
reproduction phase are given below:

Crossover: The crossover plays a most significant role in the reproduction

phase of the genetic algorithm. In this process, a crossover point is
selected at random within the genes. Then the crossover operator swaps
genetic information of two parents from the current generation to produce a
new individual representing the [Link] genes of parents are
exchanged among themselves until the crossover point is met. These newly
generated offspring are added to the population. This process is also called
or crossover. Types of crossover styles available:

Unit-5 41
 One point crossover

Two-point crossover

Livery crossover

Inheritable Algorithms crossover

MutationThe mutation operator inserts random genes in the offspring (new

child) to maintain the diversity in the population. It can be done by flipping
some bits in the [Link] helps in solving the issue of
premature convergence and enhances diversification. The below image
shows the mutation process:Types of mutation styles available,

Flip bit mutation

Gaussian mutation

Exchange/Swap mutation

5. Termination
After the reproduction phase, a stopping criterion is applied as a base for
termination. The algorithm terminates after the threshold fitness solution is
reached. It will identify the final solution as the best solution in the population.

Unit-5 42
 General Workflow of a Simple Genetic
Algorithm

Advantages of Genetic Algorithm

The parallel capabilities of genetic algorithms are best.

It helps in optimizing various problems such as discrete functions, multi-

objective problems, and continuous functions.

Unit-5 43
 It provides a solution for a problem that improves over time.

A genetic algorithm does not need derivative information.

Limitations of Genetic Algorithms

Genetic algorithms are not efficient algorithms for solving simple problems.

It does not guarantee the quality of the final solution to a problem.

Repetitive calculation of fitness values may generate some computational

challenges.

Difference between Genetic Algorithms

and Traditional Algorithms
A search space is the set of all possible solutions to the problem. In the
traditional algorithm, only one set of solutions is maintained, whereas, in a
genetic algorithm, several sets of solutions in search space can be used.

Traditional algorithms need more information in order to perform a search,

whereas genetic algorithms need only one objective function to calculate
the fitness of an individual.

Traditional Algorithms cannot work parallelly, whereas genetic Algorithms

can work parallelly (calculating the fitness of the individualities are
independent).

One big difference in genetic Algorithms is that rather of operating directly

on seeker results, inheritable algorithms operate on their representations
(or rendering), frequently appertained to as chromosomes.

One of the big differences between traditional algorithm and genetic

algorithm is that it does not directly operate on candidate solutions.

Traditional Algorithms can only generate one result in the end, whereas
Genetic Algorithms can generate multiple optimal results from different
generations.

The traditional algorithm is not more likely to generate optimal results,

whereas Genetic algorithms do not guarantee to generate optimal global
results, but also there is a great possibility of getting the optimal result for a
problem as it uses genetic operators such as Crossover and Mutation.

Unit-5 44
 Traditional algorithms are deterministic in nature, whereas Genetic
algorithms are probabilistic and stochastic in nature.

Encoding Methods in Genetic Algorithm

Biological Background :
Chromosome: All living organisms consist of cells. In each cell, there is the
same set of Chromosomes. Chromosomes are strings of DNA and consist of
genes, blocks of DNA. Each gene encodes a trait, for example, the color of the
eye.
Reproduction: During reproduction, combination (or crossover) occurs first.
Genes from parents combine to form a whole new chromosome. The newly
created offspring can then be mutated. The changes are mainly caused by
errors in copying genes from parents. The fitness of an organism is measured
by the success of the organism in its life.
Operation of Genetic Algorithms :
Two important elements required for any problem before a genetic algorithm
can be used for a solution are

Method for representing a solution ex: a string of bits, numbers, character

ex: determination total weight.

Method for measuring the quality of any proposed solution, using a fitness
function.

Basic principles :
An individual is characterized by a set of parameters: Genes

The genes are joined into a string: Chromosome

The chromosome forms the genotype

The genotype contains all information to construct an organism: Phenotype

Reproduction is a “dumb” process on the chromosome of the genotype

Fitness is measured in the real world (‘Struggle for life’) of the phenotype.

Algorithmic Phases :

Unit-5 45
 Simple_Genetic_Algorithm()
{
Initialize the population;
Calculate Fitness Function;

while(Fitness Value != Optimal Value)

{
Selection; //Natural Selection, survival of fitt
est
Crossover; //Reproduction, propagate favorable c
haracteristics
Mutation;
Calculate Fitness Function;
}
}

Encoding using string :

Encoding of chromosomes is the first step in solving the problem and it
depends entirely on the problem heavily. The process of representing the
solution in the form of a string of bits that conveys the necessary information.
just as in a chromosome, each gene controls particular characteristics of the
individual, similarly, each bit in the string represents characteristics of the
solution.
Encoding Methods :

Unit-5 46
 Binary Encoding: Most common methods of encoding. Chromosomes are
string of 1s and 0s and each position in the chromosome represents a
particular characteristics of the solution.

Permutation Encoding: Useful in ordering such as the Travelling Salesman

Problem (TSP). In TSP, every chromosome is a string of numbers, each of
which represents a city to be visited.

Value Encoding: Used in problems where complicated values, such as real

numbers, are used and where binary encoding would not suffice. Good for
some problems, but often necessary to develop some specific crossover
and mutation techniques for these chromosomes.

Unit-5 47
 Genetic Algorithms
Genetic Algorithms(GAs) are adaptive heuristic search algorithms that belong
to the larger part of evolutionary algorithms. Genetic algorithms are based on
the ideas of natural selection and genetics. These are intelligent exploitation of
random searches provided with historical data to direct the search into the
region of better performance in solution space. They are commonly used to
generate high-quality solutions for optimization problems and search
problems.
Genetic algorithms simulate the process of natural selection which means
those species that can adapt to changes in their environment can survive and
reproduce and go to the next generation. In simple words, they simulate
“survival of the fittest” among individuals of consecutive generations to solve a
problem. Each generation consists of a population of individuals and each
individual represents a point in search space and possible solution. Each
individual is represented as a string of character/integer/float/bits. This string is
analogous to the Chromosome.

Foundation of Genetic Algorithms

Genetic algorithms are based on an analogy with the genetic structure and
behavior of chromosomes of the population. Following is the foundation of GAs
based on this analogy –

1. Individuals in the population compete for resources and mate

2. Those individuals who are successful (fittest) then mate to create more
offspring than others

3. Genes from the “fittest” parent propagate throughout the generation, that is
sometimes parents create offspring which is better than either parent.

Unit-5 48
 4. Thus each successive generation is more suited for their environment.

Search space
The population of individuals are maintained within search space. Each
individual represents a solution in search space for given problem. Each
individual is coded as a finite length vector (analogous to chromosome) of
components. These variable components are analogous to Genes. Thus a
chromosome (individual) is composed of several genes (variable components).

Fitness Score
A Fitness Score is given to each individual which shows the ability of an
individual to “compete”. The individual having optimal fitness score (or near
optimal) are sought.
The GAs maintains the population of n individuals (chromosome/solutions)
along with their fitness [Link] individuals having better fitness scores are
given more chance to reproduce than others. The individuals with better fitness
scores are selected who mate and produce better offspring by combining
chromosomes of parents. The population size is static so the room has to be
created for new arrivals. So, some individuals die and get replaced by new
arrivals eventually creating new generation when all the mating opportunity of
the old population is exhausted. It is hoped that over successive generations
better solutions will arrive while least fit die.
Each new generation has on average more “better genes” than the individual
(solution) of previous generations. Thus each new generations have
better “partial solutions” than previous generations. Once the offspring
produced having no significant difference from offspring produced by previous
populations, the population is converged. The algorithm is said to be converged
to a set of solutions for the problem.

Operators of Genetic Algorithms

Unit-5 49
 Once the initial generation is created, the algorithm evolves the generation
using following operators –
1) Selection Operator: The idea is to give preference to the individuals with
good fitness scores and allow them to pass their genes to successive
generations.
2) Crossover Operator: This represents mating between individuals. Two
individuals are selected using selection operator and crossover sites are
chosen randomly. Then the genes at these crossover sites are exchanged thus
creating a completely new individual (offspring). For example –

3) Mutation Operator: The key idea is to insert random genes in offspring to

maintain the diversity in the population to avoid premature convergence. For
example –

The whole algorithm can be summarized as –

1) Randomly initialize populations p

2) Determine fitness of population
3) Until convergence repeat:
a) Select parents from population
b) Crossover and generate new population
c) Perform mutation on new population
d) Calculate fitness for new population

Unit-5 50
 Example problem and solution using Genetic Algorithms
Given a target string, the goal is to produce target string starting from a random
string of the same length. In the following implementation, following analogies
are made –

Characters A-Z, a-z, 0-9, and other special symbols are considered as
genes

A string generated by these characters is considered as

chromosome/solution/Individual

Fitness score is the number of characters which differ from characters in

target string at a particular index. So individual having lower fitness value is
given more preference.
Note: Every-time algorithm start with random strings, so output may differ
As we can see from the output, our algorithm sometimes stuck at a local
optimum solution, this can be further improved by updating fitness score
calculation algorithm or by tweaking mutation and crossover operators.

Why use Genetic Algorithms

They are Robust

Provide optimisation over large space state.

Unlike traditional AI, they do not break on slight change in input or presence
of noise

Application of Genetic Algorithms

Genetic algorithms have many applications, some of them are –

Recurrent Neural Network

Mutation testing

Code breaking

Filtering and signal processing

Learning fuzzy rule base etc

Unit-5 51