Investigating the Effect of Different Hyperparameters of the

NEAT algorithm on its Performance

How do the mutation rate, crossover rate and maximum sight distance affect the

number of frames that it takes the NEAT algorithm to produce a solution of fitness level

equal to 1000 while playing the snake video game?

Extended Essay in Computer Science

Student Code: jtd366

Word Count : 3999

Session : May 2024

Table of Contents

1. Introduction & Background................................................................................................4

1.1 Theoretical Approach..................................................................................................... 6
1.4.1 Hypothesis on Mutation Rate................................................................................ 6
1.4.2 Hypothesis on Crossover Rate.............................................................................. 6
1.4.3 Hypothesis on Maximum Sight Distance..............................................................7
1.2 Contribution.................................................................................................................................7
2. Methodology......................................................................................................................... 8
2.1 Neural Networks..........................................................................................................................8
2.1.1 Layers.................................................................................................................... 8
2.1.2 Activation Function...............................................................................................9
2.1.3 Cost Function and Backpropagation..................................................................... 9
2.1.4 Types of Neural Networks...................................................................................10
2.1.5 The Experiment’s Neural Network.................................................................................. 10
2.2 Genetic Algorithms...................................................................................................... 11
2.2.1 Inspiration and Nature......................................................................................... 11
2.2.3 Fitness Calculation.............................................................................................. 12
2.2.4 Selection.............................................................................................................. 12
2.2.5 Crossover.............................................................................................................12
2.2.6 Mutation.............................................................................................................. 13
2.3 Neuroevolution of Augmented Topologies (NEAT).................................................... 14
2.3.1 Complexification................................................................................................. 14
2.3.2 Gene Tracking..................................................................................................... 14
2.3.3 Speciation............................................................................................................ 15
3. Experimental Framework................................................................................................. 15
3.1 The Problem................................................................................................................. 15
3.2 Independent Variables.................................................................................................. 16
3.3 Dependent Variable...................................................................................................... 16
3.4 Controlled Variables..................................................................................................... 16
3.5 Experimental Procedure............................................................................................... 18
4. Experimental Results......................................................................................................... 19
4.1 Default Hyperparameters Results.................................................................................19
4.2 Altered Mutation Rate Results..................................................................................... 22
4.2.1 Increasing the Mutation Rate.............................................................................. 22
4.2.2 Decreasing the Mutation Rate............................................................................. 25
4.3 Alter Crossover Rate Results....................................................................................... 28
4.3.1 Increasing the Crossover Rate.............................................................................28
4.3.2 Decreasing the Crossover Rate........................................................................... 31
4.4 Alter Maximum Sight Distance Results.......................................................................34
4.4.1 Increasing the Maximum Sight Distance............................................................ 34
4.4.1 Decreasing the Maximum Sight Distance........................................................... 37

2
 4.5 Tables......................................................................................................................40
5. Conclusions......................................................................................................................... 42
6. Limitations.......................................................................................................................... 43
7. Works Cited........................................................................................................................ 44

3
 1. Introduction & Background

Machine learning is a branch of artificial intelligence (AI) that has become a crucial

part of our daily lives and is a groundbreaking field in which there are constantly new

innovations and developments. There are endless common uses of machine learning

algorithms including speech recognition, self-driving cars, recommendation engines,

computer vision and fraud detection1. Because of the variety of tasks in which machine

learning is employed, a sequence of different approaches to it has been developed.

Two common ways to approach machine learning are Artificial Neural Networks and

Genetic Algorithms. The former take inspiration in the functionality of the biological neural

networks in the human brain in hopes of replicating the way in which humans learn.

However, the human brain, an impressively complex computational system, consists of

roughly 100 billion neurons, connected by an estimated 100 trillion synapses2 3 and the exact

way in which it works is still a mystery in many cases. Because of this, artificial neural

networks are simply models of the human brain since they cannot directly replicate it.

On the other hand, Genetic Algorithms are a search heuristic that is inspired by

Charles Darwin’s theory of natural evolution4. Genetic algorithms employ the principles of

natural selection (the process through which species adapt to their environments, the driving

force of evolution5) with the goal of developing randomly generated solutions to a problem

into optimized solutions. Thus, this approach also is a model of a biological process.

1
Sarker, I.H. Machine Learning: Algorithms, Real-World Applications and Research Directions. SN COMPUT.
SCI. 2, 160 (2021). [Link] Accessed 3 October 2023.
2
Herculano-Houzel, Suzana. “The remarkable, yet not extraordinary, human brain as a scaled-up primate brain
and its associated cost.” Proceedings of the National Academy of Sciences of the United States of America vol.
109 Suppl 1,Suppl 1 (2012): 10661-8. doi:10.1073/pnas.1201895109. Accessed 3 October 2023.
3
Cox, David; Daniel, and Thomas Dean. “Neural Networks and Neuroscience-Inspired Computer Vision - Cell
Press.” [Link], Current Biology, 22 Sept. 2014,
[Link]/current-biology/fulltext/S0960-9822(14)01039-2. Accessed 3 October 2023.
4
Mallawaarachchi, Vijini. “Introduction to Genetic Algorithms - Including Example Code.” Medium, Towards
Data Science, 1 Mar. 2020,
[Link]/introduction-to-genetic-algorithms-including-example-code-e396e98d8bf3#:~:text=A
genetic algorithm is a, offspring of the next generation. Accessed 3 October 2023.
5
National Geographic Society. “Natural Selection.” Education, National Geographic, 1 Aug. 2022,
[Link]/resource/natural-selection/. Accessed 3 October 2023.

4
 Because of their niche approaches to machine learning both aforementioned

techniques end up being useful in solving specific problems. However, by combining the two

approaches into one by having a genetic algorithm train a neural network a new approach to

machine learning (NEAT) is created. Neuroevolution of Augmented Topologies (NEAT: a

Neuro-Evolution (NE) algorithm mimicking the functionality of biological systems in nature

by presenting them as evolving virtual organisms that increase complexity over time6)

provides us with a machine learning algorithm that can be used to solve more and

increasingly complex problems.

Figure 1 : Visualization of the Genetic Algorithm changing the structure of the Neural

Network by either adding a connection between nodes or adding new nodes7

A good example of problems like that are videogames. Since both neural networks

and genetic algorithms are models of the natural processes they describe it is only logical to

6
Khazab, Mohammad PhD. “Neuro-Evolution of Augmenting Topologies Algorithm.” Neuro-Evolution of
Augmenting Topologies Algorithm, AI Tools & Tech Newsletter, 2 July 2022,
[Link]/pulse/neuro-evolution-augmenting-topologies-algorithm-mohammad-khazab/. Accessed 3
October 2023.
7
Omelianenko, Iaroslav. “Neuroevolution - Evolving Artificial Neural Networks Topology from the Scratch.”
Medium, Becoming Human: Artificial Intelligence Magazine, 17 Jan. 2020,
[Link]/neuroevolution-evolving-artificial-neural-networks-topology-from-the-scratch-d1ebc5540d8
4.

5
have them tackle models of real world problems and videogames have proven to be critical

testing grounds. In fact, research made in this field has given researchers insight into how

neural nets can be optimized to learn how to solve problems in the real world.

In this paper the typical snake game is used to train a NEAT algorithm upon. Through

training snakes to become fitter through chasing food the experiment aims to examine how

different hyperparameters8 of the NEAT algorithm (mutation rate, crossover rate, maximum

sight distance) affect its performance and more specifically the amount of frames that it takes

to produce a solution of fitness level equal to 1000 while playing the snake video game.

1.1 Theoretical Approach

1.4.1 Hypothesis on Mutation Rate

The mutation rate is a genetic algorithm hyperparameter that controls how frequently

a mutation is introduced to a chromosome (solution). Increasing the mutation rate is going to

introduce more chromosome diversity in the form of NEAT innovations (explained in 2.3.2).

Thus, I hypothesize that an increase in mutation rate will have a negative correlation with the

amount of frames that the algorithm requires to produce a chromosome of fitness 1000 (i.e. a

positive correlation with the algorithm’s efficiency) since the more innovations introduced,

the more the problem space9 is increased. An increased problem space entails a bigger chance

selection for the algorithm to search in for an optimal solution.

1.4.2 Hypothesis on Crossover Rate

The crossover rate represents the likelihood of two parents interbreeding to produce

offspring (the chance of crossover occuring). Unlike mutation, the crossover procedure

8
Different values in the code of the algorithm that affect how the Neural Network is developed
9
All the possible solutions that can occur

6
cannot introduce new innovations and instead creates a mix of the chromosomes of two

previous solutions. Thus since lowering the crossover rate is only indirectly related to

biodiversity, reducing how frequently chromosome recombination (crossover) occurs will

simply promote more stability of the algorithm. Thus, I hypothesize that a decrease in

crossover rate will have a positive correlation with the amount of frames it takes the

algorithm to produce an optimal solution (fitness 1000).

1.4.3 Hypothesis on Maximum Sight Distance

The maximum sight distance is a measurement of the distance the snakes can

“see/search” for food. A NEAT algorithm is fed information about the specificities of the

problem it’s trying to solve and the position of available food is one example. Since food is

the main way in which the snakes increase their fitness and stay alive (if a snake runs out of

food it “dies” out of starvation) being able to spot food from further away will give the NEAT

algorithm a greater chance to collect it (i.e. the snakes can track and eat food more easily).

Thus, I hypothesize that an increase of maximum sight distance will have a negative

correlation with the amount of frames required by an algorithm to produce a solution of

fitness 1000.

1.2 Contribution

Conducting research on this topic is very important since it provides a more mature

and thorough understanding of how different hyperparameters of the NEAT algorithm affect

its performance on certain tasks. Using information from this paper more optimized and

efficient algorithms can be created which lead to faster and better results. Also the

hyperparameter of maximum sight distance has no significant research conducted on it so the

results of the experiment will shed some light on its effect on the algorithm.

7
 2. Methodology

2.1 Neural Networks

2.1.1 Layers

Generally, in a feed forward neural network (FNN: a network where the information

flow is unidirectional, flows forward, from the input nodes, through the hidden nodes to the

output nodes, without any cycles or loops10) there are three layers.

The input layer consists of nodes and is responsible for bringing the input data into

the system. It is the beginning of the FNN and allows for the inputted data to be processed by

the subsequent layers.

The hidden layers are located between the input and output layers. Their nodes are

responsible for introducing non-linearity to the network through the activation function (as

explained in 2.1.2) and for applying the current weights to the inputted data thus changing it.

In the output layer’s nodes the final weights of the ANN are applied and the result for

the inputted data is produced11. In this layer the cost function is also applied in order for the

process of backpropagation to initiate (explained in section 2.1.3).

10
Zell, Andreas (1994). Simulation Neuronaler Netze [Simulation of Neural Networks] (in German) (1st ed.).
Addison-Wesley. p. 73. ISBN 3-89319-554-8. Accessed 3 October 2023.
11
A. K. Jain, Jianchang Mao and K. M. Mohiuddin, "Artificial neural networks: a tutorial," in
Computer, vol. 29, no. 3, pp. 31-44, March 1996, doi: 10.1109/2.485891. Accessed 3 October
2023.

8
Figure 2 : Visualization of the standard structure of a Neural Network12

2.1.2 Activation Function

The activation function is applied at the hidden and/or output layers of a neural

network. By inputting data to the activation function it is transformed to an output that is

going to be used by the nodes in the next layer. It is also responsible for introducing

non-linearity to the system in order for more complex tasks to be performed.

2.1.3 Cost Function and Backpropagation

The cost function in a supervised learning FNN produces a quantification of the error

of the network’s predicted output as opposed to the expected output from the training dataset.

This function is applied at the output layer and the error is used in the process of

backpropagation where according to the error the network’s weights are adjusted and thus the

network is trained.

12
Dabbura, Imad. “Coding Neural Network - Forward Propagation and Backpropagtion.” Medium, Towards
Data Science, 27 Sept. 2022,
[Link]/coding-neural-network-forward-propagation-and-backpropagtion-ccf8cf369f76.
Accessed 3 October 2023.

9
Figure 3 : Visualization of the backpropagation process in a Neural Network13

2.1.4 Types of Neural Networks

There are two main types of training in neural networks. In supervised learning the

ANN classifies any given input with a label based on its training. The ANN requires large

training datasets that include input-output pairs so that the ANN trains by classifying a certain

input and then calculating its error based on the expected output (as explained in 2.1.3).

Unsupervised learning networks are used for the task of clustering. They do not

require labeled training data sets and instead group inputs together based on their common

properties and features.

2.1.5 The Experiment’s Neural Network

This experiment deals with a feed forward unsupervised learning ANN of set

structure. It consists of an input layer of 48 nodes, 2 hidden layers of 16 nodes each and an

output layer of 2 nodes.

13
Johnson, Daniel. “Back Propagation in Neural Network: Machine Learning Algorithm.” Guru99, Guru99, 9
Sept. 2023, [Link]/[Link]. Accessed 3 October 2023.

10
2.2 Genetic Algorithms

2.2.1 Inspiration and Nature

Genetic Algorithms (GAs) are heuristic search and optimization algorithms that after

generations find a solution of sufficient fitness unlike neural networks which are models.

GAs follow a series of steps that emulate the process of natural selection14 as seen in figure 4.

Figure 4: The standard loop of a genetic algorithm, the different steps followed until

optimal solution is reached15

14
Mitchell, Melanie. “Genetic Algorithms: An Overview.” CITESEERX, Santa Fe Institute, 1995,
[Link]/document?repid=rep1&type=pdf&doi=b42cffa5a2ad63a31fcf99869e7cb8ef72b4
4374. Accessed 3 October 2023.
15
Battini, Deepak. “Can Biologically Inspired Operators Such as Mutation, Crossover and Selection Help
Improvise Machine Learning?” Tech-Quantum, Tech-Quantum, 24 Apr. 2021,
[Link]/optimize-machine-learning-using-genetic-algorithm/. Accessed 3 October 2023.

11
2.2.2 Population Initialization

The initialization of the population is the first step to the evolutionary cycle of a

genetic algorithm. Most commonly the population consists of randomly generated solutions

represented as binary strings. The population in the experiment is initialized in this way. All

the future generations will stem from the initial population.

2.2.3 Fitness Calculation

Once solutions exist, the fitness function is applied. This is an easy and fast way to

quantify the potential and the “merit” of each solution16. It is later used during the selection

procedure where according to the fitness value the evolution process will lead towards more

optimized solutions.

2.2.4 Selection

There are a variety of different methods to approach selection but generally

chromosomes (=solutions) in the population are selected for reproduction17. These

chromosomes are added to the “mating pool”. Later, chromosomes with larger fitness are

more likely to be selected to reproduce, and thus, propagate their genes to future generations.

2.2.5 Crossover

Crossover is used to produce the chromosomes of the next generation. Sequences of

two selected “parent” solutions are exchanged in order to produce two offspring which will

be part of the next generation as shown in figure 5.

16
A. Lambora, K. Gupta and K. Chopra, "Genetic Algorithm- A Literature Review," 2019 International
Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COMITCon), Faridabad, India,
2019, pp. 380-384, doi: 10.1109/COMITCon.2019.8862255. Accessed 3 October 2023.
17
Mitchell, Melanie. “Genetic Algorithms: An Overview.” CITESEERX, Santa Fe Institute, 1995,
[Link]/document?repid=rep1&type=pdf&doi=b42cffa5a2ad63a31fcf99869e7cb8ef72b4
4374. Accessed 3 October 2023.

12
Figure 5: A visual representation of single point crossover, a fairly common type of

crossover in GAs18

2.2.6 Mutation

During the mutation procedure small alterations are done to the chromosomes

produced by the crossover procedure (i.e. random flipping of bits) according to the mutation

rate which represents how likely such a mutation may occur. Mutation is necessary in order to

avoid premature convergence of the solutions (fixation on certain characteristics of

chromosomes that may prove suboptimal).

18
“Crossover in Genetic Algorithm.” GeeksforGeeks, GeeksforGeeks, 10 Mar. 2023,
[Link]/crossover-in-genetic-algorithm/. Accessed 3 August 2023.

13
2.3 Neuroevolution of Augmented Topologies (NEAT)

The Neat Algorithm emulates how biological systems function in nature through the

use of increasingly complex ANNs19. Essentially, NEAT employs a genetic algorithm to train

an artificial neural network. A NEAT algorithm performs three main procedures.

2.3.1 Complexification

A typical NEAT algorithm starts off with a minimal ANN structure but based on the

genetic algorithm’s outputs starts to increase the structure’s complexity by altering the

topology (e.g. adding hidden layer nodes/connecting nodes) and the connection weights of

the ANN so over generations the ANN becomes more sophisticated. This process is called

complexification20.

2.3.2 Gene Tracking

Each change that is made to the structure of the ANN (node creation, weight

adjustment etc) is stored by the NEAT algorithm as a historical marking/innovation at a

database. Each innovation has a unique innovation ID and they are critical for the crossover

and mutation procedures to function. Each time a new innovation is introduced its originality

is checked by comparing it to the database. Upon confirmation, the new innovation is given a

new unique identifier. This guarantees that despite potential variations in the structure and

weights of each genome, all related genes are identical21. Thus, when crossover or mutation

19
Khazab, Mohammad PhD. “Neuro-Evolution of Augmenting Topologies Algorithm.” Neuro-Evolution of
Augmenting Topologies Algorithm, AI Tools & Tech Newsletter, 2 July 2022,
[Link]/pulse/neuro-evolution-augmenting-topologies-algorithm-mohammad-khazab/. Accessed 3
October 2023.
20
Ibrahim, Mohamed & Sridhar, Rangarajan & Tv, Geetha & S S, Deepika. (2019). Advances in Neuroevolution
through Augmenting Topologies – A Case Study. 111-116. 10.1109/ICoAC48765.2019.246825. Accessed 3
October 2023.
21
“Neat.” Automatons Adrift, Automatons Adrift, [Link]/neat/. Accessed 3 October 2023.

14
occurs the new solution preserves the same innovation as its parents in each of its genes,

something which prevents the algorithm from becoming too computationally heavy.

2.3.3 Speciation

When a mutation/innovation is introduced in the population sample it is typical for it

to have an initially low fitness since it needs a few generations to be optimized. In order to

avoid complete disregard of new innovations the NEAT algorithm allocates them to a new

species, thus promoting biodiversity and avoiding premature convergence of the

chromosomes. This process is referred to as speciation22.

3. Experimental Framework

3.1 The Problem

The Artificial Neural Network structure is predetermined23. It is composed of an input

layer of 48 nodes, two hidden layers of 16 nodes each and an output layer of 2 sole nodes

(representing the snakes moving either left or right). The connections/links between the nodes

however are not preset. Thus, the genetic algorithm is only used to adjust the links between

nodes and their weights. Eight snakes are randomly generated as the initialization of the

population. Along with them, four pieces of “food” are scattered and move around the screen

for the snakes to eat. A snake has three ways of dying (an event that triggers the generation of

a new chromosome): running into each own body, running into a wall or running out of food

for a long period of time (a feature representing starvation). With each piece of food that a

snake eats its fitness is increased. The snakes cannot interact with each other but the same

food is available to them. The fact that there is competition for food (fitness) along with the
22
Andersen, Timothy & Stanley, Kenneth & Miikkulainen, Risto. (2002). Neuro-Evolution Through
Augmenting Topologies Applied To Evolving Neural Networks To Play Othello.
23
emgoz. "Neural-network-snake." Github, 19 Feb 2019, [Link]
Accessed 2 July 2023.

15
feature of fitter snakes being slower than the less fit ones, simulates natural selection, thus

resulting in the production of more competitive chromosomes. With each frame of the game

loop that passes, the code stores the fitness of the fittest chromosome. The simulation ends

when any of the chromosomes reaches a fitness value of one thousand.

3.2 Independent Variables

The independent variables are the variables being changed for the experimentation

and for the examination of their effect on the dependent variable. Those variables are

mutation rate, crossover rate and maximum sight distance. Each experiment for each variable

is executed thrice in order for the calculation of an average value and the mitigation of error.

3.3 Dependent Variable

The dependent variable in this experiment is the number of frames from the beginning

of the algorithm’s runtime until the point when a solution reaches a fitness level of 1000.

Frame count in this case is used as it provides more accuracy as opposed to time, since

introducing time as the dependent variable would produce the human error of manually

stopping the simulation. On the other hand, the frame rate during the experiment is constant

(locked at 60 frames per second) and since the code storing the data runs every frame,

accuracy of exactly in what frame fitness 1000 was achieved is introduced.

3.4 Controlled Variables

Table 1: Visual Representation of All Controlled Variables for the Experiments

Variable Description Specifications

Personal Computer Used I ran all the experiments on Device name TaxBook
the same HP laptop so that Processor AMD Ryzen
all experiments ran at the 5 4500U with Radeon
same framerate Graphics 2.38 GHz

16
 Installed RAM8.00 GB
(7.36 GB usable)
Device ID
26A9D0EB-4038-4259-BC0
B-EA5A0149065D
Product ID
00356-02682-65039-AAOE
M
System type 64-bit
operating system, x64-based
processor
Pen and touch No pen or
touch input is available for
this display

Operating System I used Windows 11 for all EditionWindows 11 Home

the experiments Edition

Same Algorithm Used I used the same algorithm Not Applicable

and set of scripts that was
taken from github a github
repository and slightly
modified by me for different
experiments and data
extraction24

Same Amount of Each experiment was Not Applicable

Repetitions repeated thrice

Same Variable Alteration for Once a variable was altered, Not Applicable
Each Repetition its altered value did not
change for the three
repetitions of the experiment

All of the variables in Table 1 were controlled in order to maintain the same

performance of the computer during all experiments and thus avoid skewed frame counts and

to have a standardized method of conducting each experiment and thus more reliable data.

24
emgoz. "Neural-network-snake." Github, 19 Feb 2019, [Link]
Accessed 2 July 2023.

17
3.5 Experimental Procedure

● A python script that reads a .txt file containing an account of maximum fitness of the

snakes for each frame, produces a graph of fitness and frame count and saves the

graph as an image file is created.

● All the code values not used in the experiment are set to defaults. The current

independent variable is altered.

● The name of the .txt file storing the data that will be created during the experiment is

named appropriately. *

● The algorithm is run until a chromosome reaches fitness of 1000, then the window is

terminated. *

● The data extracted during the experiment is inputted into the python script which is

executed. *

● The .png and .txt files for the experiment are stored in appropriate folders. *

● The frame in which fitness 1000 is reached is extracted from the .txt file.

● All steps marked with “*” are repeated thrice, the average frame count of solutions

reaching fitness 1000 is taken and stored.

● The steps are repeated for each experiment (independent variable).

Table 2: Different Values for Each Hyperparameter that were used in the Experiments

Default Value Increased Value Decreased Value

Mutation Rate 0.02 0.03 0.01

Crossover Rate 1 1.5 0.5

Maximum Sight 600 900 300

Distance

18
 The default value for each of the hyperparameters was the one set by the creator of the

code in the repository. In order to maintain an equal amount of increase and decrease of each

hyperparameter for all the experiments, I respectively added or subtracted half of the default

value.

4. Experimental Results

4.1 Default Hyperparameters Results

Figure 6: Visual line graph representation of the fitness per frame for the default

hyperparameters during the first experiment

19
 In the first experiment (Figure 6) the main problem with the default values is exposed.

There is a lot of instability in the progression of fitness since there are many peaks that

eventually drop back to a very low fitness level.

Figure 7: Visual line graph representation of the fitness per frame for the default

hyperparameters during the second experiment

Despite Figure 7’s portrayal of a more linear fitness progression the aforementioned

problem that the first experiment had remains.

20
Figure 8: Visual line graph representation of the fitness per frame for the default

hyperparameters during the third experiment

In Figure 8 the aforementioned instability issue is significantly less prevalent,

something that can be attributed to a favorable initialization of the algorithm

21
4.2 Altered Mutation Rate Results

4.2.1 Increasing the Mutation Rate

Figure 9: Visual line graph representation of the fitness per frame for the increased

mutation rate during the first experiment

22
Figure 10: Visual line graph representation of the fitness per frame for the increased

mutation rate during the second experiment

Figures 9 and 10 present a similar progression of fitness. In both experiments fitness

starts off very low for quite a while and with a lot of setbacks. This is due to many mutations

being tried that are not always favorable. However, when a favorable mutation is found, it is

fairly optimized which results in an exponential increase during the last frames.

23
Figure 11: Visual line graph representation of the fitness per frame for the increased

mutation rate during the third experiment

Figure 11’s results are similar to the default values experiments since fitness is

constantly set back until it reaches 1000. However, unlike the default values, these setbacks

are much more frequent, something attributed to unfavorably mutated producing the highest

fitness and then dying.

Overall, the increase of mutation rate forbade the algorithm from prematurely

converging to a suboptimal solution and, through the increase of biodiversity, the algorithm

had a larger problem space meaning that it had a larger amount of chromosomes to check for

24
an optimal solution. Thus, in the future it would be of interest to investigate whether taking a

slightly longer time to produce a solution of good fitness but also gaining a fairly

“stable”/optimal solution is a sacrifice worth making.

4.2.2 Decreasing the Mutation Rate

Figure 12: Visual line graph representation of the fitness per frame for the decreased

mutation rate during the first experiment

25
Figure 13: Visual line graph representation of the fitness per frame for the decreased

mutation rate during the second experiment

Figures 12 and 13 present similar data. Unlike before, the progression of fitness is

fairly linear with little and infrequent setbacks. This is caused by the random initialization

starting with a favorable problem space and the fact that introducing less mutations and thus

innovations was able to keep the algorithm more stable by allowing it to quickly find an

optimal solution (with the cost of only searching in a much narrower problem space due to

decrease in mutations).

26
Figure 14: Visual line graph representation of the fitness per frame for the decreased

mutation rate during the third experiment

Figure 14’s fitness progression is more unstable and introduces more setbacks,

something which is probably caused by an unfavorable initial population which the low

mutation rate rarely improved upon.

Overall, decreasing the mutation rate actually produced the best average results out of

all experiments (Table 6), thus becoming the most efficient parameter alteration at improving

the algorithm’s efficiency.

27
4.3 Alter Crossover Rate Results

4.3.1 Increasing the Crossover Rate

Figure 15: Visual line graph representation of the fitness per frame for the increased

crossover rate during the first experiment

28
Figure 16: Visual line graph representation of the fitness per frame for the increased

crossover rate during the second experiment

29
Figure 17: Visual line graph representation of the fitness per frame for the increased

crossover rate during the third experiment

Overall, in Figures 15, 16 and 17 there are frequent high peaks of fitness followed by severe

setbacks which were likely caused by crossover of chromosomes that produced unfavorable

offspring that died easily. As hypothesized, the increase in crossover rate made the algorithm

more of a random search by constantly recombining older chromosomes, thus increasing

instability.

30
4.3.2 Decreasing the Crossover Rate

Figure 18: Visual line graph representation of the fitness per frame for the decreased

crossover rate during the first experiment

31
Figure 19: Visual line graph representation of the fitness per frame for the decreased

crossover rate during the second experiment

32
Figure 20: Visual line graph representation of the fitness per frame for the decreased

crossover rate during the third experiment

Overall, the decrease in crossover rate as predicted promoted stability in the algorithm

and allowed for a seamless propagation of genes to the next generations, thus allowing new

generations to simply build upon older ones instead of becoming simple crossovers of them.

This, of course, is shown in Figures 18, 19 and 20 where they all present similar data of a

fairly linear progression with few and infrequent setbacks.

33
4.4 Alter Maximum Sight Distance Results

4.4.1 Increasing the Maximum Sight Distance

Figure 21: Visual line graph representation of the fitness per frame for the increased

maximum sight distance during the first experiment

34
Figure 22: Visual line graph representation of the fitness per frame for the increased

maximum sight distance during the second experiment

35
Figure 23: Visual line graph representation of the fitness per frame for the increased

maximum sight distance during the third experiment

Overall, the increase in the maximum sight distance produced the worst result out of

all other parameter set-ups, producing an average of 44945.3, a staggering 11000 frames

above the next worst experiment (Table 6). The bad nature of those results may be attributed

to the fact that increasing the sight distance gave a greater opportunity for smaller snakes

(that moved faster) to spot food from further away, thus “stealing” food from larger snakes,

an action which when repeated resulted in death from starvation. This is shown in Figures 21,

22 and 23 where they all present the most frequent and constantly occurring setbacks of

fitness out of all other experiments. These setbacks in this experiment indicate deaths from

36
starvation. In fact, this alteration of the hyperparameter was so suboptimal that in Figures 22

and 23 the frame count surpasses 50000.

4.4.1 Decreasing the Maximum Sight Distance

Figure 24: Visual line graph representation of the fitness per frame for the decreased

maximum sight distance during the first experiment

37
Figure 25: Visual line graph representation of the fitness per frame for the decreased

maximum sight distance during the second experiment

38
Figure 26: Visual line graph representation of the fitness per frame for the decreased

maximum sight distance during the third experiment

Overall, decreasing the maximum sight distance did not produce better results than

the default values with an average of roughly 3000 frames more than the default values.

However, decreasing the maximum sight distance produced the most varied results across

three experiments having both an experiment producing 15000 frames (Figure 25) and one

with 40000 frames (Figure 24). This virtual randomness can be explained by the random

initialization of the first population sample. Initializing an original population with a fast

reaction speed to seeing food paired with the fact that there is reduced competition since

snakes do not spot food from far away, created a solution of fitness 1000 very easily (Figure

39
25). However, initializing a population with a bad reaction speed to seeing food paired with

what was probably unfavorable mutations and crossovers created a bad environment for

snakes to increase their fitness, since they only “saw” food at the last moment and could not

react to it fast enough so that they eat it. This caused constant setbacks like those in Figures

24 and 26.

4.5 Tables

Table 3: Exact Data for the Number of Frames it Took the Algorithm to Reach Fitness

1000 for the First Repetition of Each Experiment

Frame in which Frame of Fitness Frame of Fitness

fitness 1000 was 1000 (Decreasing 1000 (Increasing
reached Variable) Variable)

Default 34420 N/A N/A

Mutation Rate N/A 14742 27748

Crossover Rate N/A 18003 38496

Maximum Sight N/A 43964 32571

Distance

The first experiment (Table 3) presented the decrease of mutation rate as the most

effective strategy for optimization while the decrease of maximum sight distance produced

unfavorable results.

40
Table 4: Exact Data for the Number of Frames it Took the Algorithm to Reach Fitness

1000 for the Second Repetition of Each Experiment

Frame in which Frame of Fitness Frame of Fitness

fitness 1000 was 1000 (Decreasing 1000 (Increasing
reached Variable) Variable)

Default 24083 N/A N/A

Mutation Rate N/A 11496 15670

Crossover Rate N/A 17012 34043

Maximum Sight N/A 15315 54219

Distance

Unlike Table 3, Table 4’s data presented the decrease of maximum sight distance as

the second most favorable change while its increase was the worst. Regardless, the decrease

of mutation rate again produced the lower framecount.

Table 5: Exact Data for the Number of Frames it Took the Algorithm to Reach Fitness

1000 for the Third Repetition of Each Experiment

Frame in which Frame of Fitness Frame of Fitness

fitness 1000 was 1000 (Decreasing 1000 (Increasing
reached Variable) Variable)

Default 17583 N/A N/A

Mutation Rate N/A 21695 24770

Crossover Rate N/A 13123 29403

Maximum Sight N/A 26043 48046

Distance

In the third experiment (Table 5) crossover rate’s decrease produced the lowest

framecount while the increase of maximum sight distance once again produced the worst

results.

41
Table 6: Representation of Mean Frame Count for Each Experiment

Mutation Rate Crossover Rate Maximum Sight

Distance

Increased 22729.3 33980.6 44945.3

Decreased 15977.6 16046 28440.6

Average Frame for Default Values: 25362

As seen in Table 6 the decrease of mutation rate produced the most optimized results

followed closely by the decrease of crossover rate. Contrarily, the increase of maximum sight

distance produced the highest frame count by roughly 10000 frames.

5. Conclusions

In this essay I made the hypotheses that an increase in mutation rate will have a

negative correlation with the number of frames, a decrease in crossover rate will have a

positive correlation with the number of frames and an increase of maximum sight distance

will have a negative correlation with the amount of frames. The two former failed while the

latter succeeded. Finally, it was discovered that decreasing the mutation rate and the

crossover rate both produced better results than their default values as opposed to the

maximum sight distance whose default value of 600 turned out to be more optimal than any

decrease or increase of that number. Thus, alterations in sight distance were the only

experiment that failed to produce better results than the default values. Despite the decrease

in frames produced by the decrease in mutation rate, increasing the mutation rate also

produced on average better results than the default values, indicating that the search process

became slower due to the algorithm looking for a more optimized solution. Thus, in future

research it would be interesting to compare the two solutions which reached fitness 1000 (one

from increasing mutation and one from decreasing) in order to examine whether the

42
increasing one is actually more optimal. The same claim cannot be made for crossover rate

whose increase seemed to exclusively hinder the algorithm’s efficiency as opposed to its

decrease that improved it. Altering the maximum sight distance in both cases produced less

optimized results. In this experiment the lowest amount of frames (most efficient search) was

produced by a decrease in mutation rate followed by a decrease in crossover rate, followed by

no change in maximum sight distance.

6. Limitations

One limitation of the experimental method concerns the maximum sight distance. The

results of the experimentation were affected by the competition of snakes (faster, smaller

snakes would reach food faster than the biggest one) which in both the increase and the

decrease experiments ended up taking a toll on the algorithm’s efficiency. This variable being

increased is a more computationally expensive process since more environmental information

(food/walls) is fed to the neural network of its snake. However, future research would

probably indicate that in an environment lacking competition with only a singular snake, the

NEAT algorithm would produce much more efficient results with an increase of maximum

sight distance. Another limitation of the method is that the algorithm was overly

unpredictable to securely identify patterns. For example, throughout the three experiments for

the default hyperparameter values the framecount spans from 17583 to 34420 frames. Since

the algorithm produced such varied results between experiments it would have been better to

execute each experiment more times (e.x. 5 times) in order to solidify the results.

43
 7. Works Cited

A. K. Jain, Jianchang Mao and K. M. Mohiuddin, "Artificial neural networks: a

tutorial," in Computer, vol. 29, no. 3, pp. 31-44, March 1996, doi: 10.1109/2.485891.

Accessed 3 October 2023.

A. Lambora, K. Gupta and K. Chopra, "Genetic Algorithm- A Literature Review,"

2019 International Conference on Machine Learning, Big Data, Cloud and Parallel

Computing (COMITCon), Faridabad, India, 2019, pp. 380-384, doi:

10.1109/COMITCon.2019.8862255. Accessed 3 October 2023.

Andersen, Timothy & Stanley, Kenneth & Miikkulainen, Risto. (2002).

Neuro-Evolution Through Augmenting Topologies Applied To Evolving Neural Networks To

Play Othello. Accessed 3 October 2023.

Battini, Deepak. “Can Biologically Inspired Operators Such as Mutation, Crossover

and Selection Help Improvise Machine Learning?” Tech-Quantum, Tech-Quantum, 24 Apr.

2021, [Link]/optimize-machine-learning-using-genetic-algorithm/.

Accessed 3 October 2023.

Cox, David; Daniel, and Thomas Dean. “Neural Networks and Neuroscience-Inspired

Computer Vision - Cell Press.” [Link], Current Biology, 22 Sept. 2014,

[Link]/current-biology/fulltext/S0960-9822(14)01039-2. Accessed 3 October 2023.

Dabbura, Imad. “Coding Neural Network - Forward Propagation and

Backpropagtion.” Medium, Towards Data Science, 27 Sept. 2022,

[Link]/coding-neural-network-forward-propagation-and-backpropagtion-cc

f8cf369f76. Accessed 3 October 2023.

emgoz. "Neural-network-snake." Github, 19 Feb 2019,

[Link] Accessed 2 July 2023.

44
 Herculano-Houzel, Suzana. “The remarkable, yet not extraordinary, human brain as a

scaled-up primate brain and its associated cost.” Proceedings of the National Academy of

Sciences of the United States of America vol. 109 Suppl 1,Suppl 1 (2012): 10661-8.

doi:10.1073/pnas.1201895109. Accessed 3 October 2023.

Ibrahim, Mohamed & Sridhar, Rangarajan & Tv, Geetha & S S, Deepika. (2019).

Advances in Neuroevolution through Augmenting Topologies – A Case Study. 111-116.

10.1109/ICoAC48765.2019.246825. Accessed 3 October 2023.

Johnson, Daniel. “Back Propagation in Neural Network: Machine Learning

Algorithm.” Guru99, Guru99, 9 Sept. 2023,

[Link]/[Link]. Accessed 3 October 2023.

Khazab, Mohammad PhD. “Neuro-Evolution of Augmenting Topologies Algorithm.”

Neuro-Evolution of Augmenting Topologies Algorithm, AI Tools & Tech Newsletter, 2

July 2022,

[Link]/pulse/neuro-evolution-augmenting-topologies-algorithm-mohammad-kha

zab/. Accessed 3 October 2023.

Mallawaarachchi, Vijini. “Introduction to Genetic Algorithms - Including Example

Code.” Medium, Towards Data Science, 1 Mar. 2020,

[Link]/introduction-to-genetic-algorithms-including-example-code-e396e9

8d8bf3#:~:text=A genetic algorithm is a, offspring of the next generation. Accessed 3

October 2023.

Mitchell, Melanie. “Genetic Algorithms: An Overview.” CITESEERX, Santa Fe

Institute, 1995,

[Link]/document?repid=rep1&type=pdf&doi=b42cffa5a2ad63a31fcf

99869e7cb8ef72b44374. Accessed 3 October 2023.

45
 National Geographic Society. “Natural Selection.” Education, National Geographic, 1

Aug. 2022, [Link]/resource/natural-selection/. Accessed 3 October

2023.

Omelianenko, Iaroslav. “Neuroevolution - Evolving Artificial Neural Networks

Topology from the Scratch.” Medium, Becoming Human: Artificial Intelligence Magazine,

17 Jan. 2020,

[Link]/neuroevolution-evolving-artificial-neural-networks-topology-from-the-scr

atch-d1ebc5540d84. Accessed 4 October 2023.

Sarker, I.H. Machine Learning: Algorithms, Real-World Applications and Research

Directions. SN COMPUT. SCI. 2, 160 (2021). [Link]

Accessed 3 October 2023.

Stanley, Kenneth O., and Risto Miikkulainen. “Evolving Neural Networks through

Augmenting Topologies.” MIT Press, MIT Press, 1 June 2002

[Link]/evco/article-abstract/10/2/99/1123/Evolving-Neural-Networks-through-Augme

nting?redirectedFrom=fulltext. Accessed 3 October 2023.

Zell, Andreas (1994). Simulation Neuronaler Netze [Simulation of Neural Networks]

(in German) (1st ed.). Addison-Wesley. p. 73. ISBN 3-89319-554-8. Accessed 3 October

2023.

“Crossover in Genetic Algorithm.” GeeksforGeeks, GeeksforGeeks, 10 Mar. 2023,

[Link]/crossover-in-genetic-algorithm/. Accessed 3 August 2023.

“Neat.” Automatons Adrift, Automatons Adrift, [Link]/neat/.

Accessed 3 October 2023.

46