CHAPTER ONE

INTRODUCTION

1.1. Background of the Study

In recent years, almost every member of the society has been using the Internet for daily life.

This is because it is almost impossible to do anything without the Internet including social

interactions, online banking, health related transaction, and marketing. Since the Internet has

been growing rapidly, criminals have started to commit crimes on the Internet rather than in real

world. Criminals are generally using malicious software to launch cyber-attacks to the victim

machines. Any software which intentionally executes malicious payloads on victim machines

(computers, smart phones, computer networks, etc.) is considered as malware. There are

different types of malware including virus, worm, Trojan horse, rootkit, and ransomware.

Each malware type and family is designed to affect original victim machine in different ways

such as damaging the targeted system, allowing remote code execution, stealing confidential

data, etc. These days, the classification of malware is getting harder because some malware

instances can present the characteristics of multiple classes at the same time. In the early days,

malware was written for simple purposes, thus, it was easier to detect. This kind of malware can

be defined as traditional (simple) malware. However, in recent times, the malware which can run

in kernel mode, and is more destructive and harder to detect than traditional malware can be

defined as new generation malware (next-generation). This kind of malware can easily bypass

protection software that is running in kernel mode such as firewalls, antivirus software, etc.

Generally, traditional malware consists of one process and does not use complicated techniques

to hide itself. On the other hand, new generation malware uses multiple different existing or new
1
processes at the same time, and uses some obfuscated techniques to hide itself and become

persistent in the system. New generation malware can launch more destructive attacks such as

targeted and persistent which have never been seen before, and more than one type of malware is

used during the attacks.

These days, the number, sophistication, and cost of malware inflicted on the world economy

have been increasing incrementally. According to scientific and business reports, approximately

1 million malware files are created every day, and cybercrime will damage the world economy

by approximately $6 trillion annually by 2021 (S. Morgan. 2019). Recent studies show that

mobile malware is on the rise. According to the McAfee mobile threat report, there is a huge

increase in backdoors, fake applications and banking Trojans for mobile devices (R. Samani &

G. Davis. (2019). Besides, the malware attacks related to the social media, healthcare industry,

cloud computing, internet of things (IoT), and cryptocurrencies are also on the rise. According to

cybersecurity ventures, ransomware malware will cost around $11.5 billion globally at the end of

2019 To protect legitimate users and companies from malware, malware need to be detected.

Malware detection is the process of determining whether a given program has malicious intent or

not. In early days, signature-based detection approach was used widely to detect malware.

However, this approach has some limitations such as it cannot detect unknown and new

generation malware. In process of time, researchers proposed new approaches including

behavioral-, heuristic-, and model checking-based detection. With these approaches, datamining

and machine learning (ML) algorithms are also started to be used widely in malware detection.

Recently, new approaches have been proposed such as deep learning-, cloud-, mobile devices-,

and IoT-based detection. For known and some of unknown malware, heuristic detection

approach performs well. On the other hand, for unknown and complicated malware; behavior-,
2
model checking-, and cloud-based approaches perform better. Deep learning-, mobile devices-,

and IoT-based approaches also emerge to detect some portion of known and unknown malware.

It has not been proved exactly that one detection approach is more effective than the others. This

is because each method has its own advantages and disadvantages, and in different situation one

method can detect better than another. Even though several new methods have been proposed by

using different malware detection approaches, no method could detect all new generation and

sophisticated malware. This shows that building an effective method to detect malware is a very

challenging task, and there is a huge demand for new studies and methods.

Since the first malware that appeared in the wild was a virus, most of the studies had been done

theoretically were based on the detection of virus. According to early studies, the detection of

virus is impossible and NP-complete (Z. Zuo et. al., 2005). According to F. Cohen 1992, the

detection of computer virus is an undecidable because detection process itself contains a

contradiction. If the detection problem is seen as a decision-making problem, D (decision-maker)

will decide whether P is a virus or not. According to Cohen, it cannot be decided whether P is a

virus because if P is a virus, it will be marked by D as a virus and will not be able to make

changes to other programs, as it will not act as a virus. If D decision maker did not identify P as a

virus, P will interact with other programs to spread and become infected.

This decision process involves contradiction, and therefore it is not possible to identify P as a

virus. According to M. Chess & R. White 1991, there is no program that detects all viruses

without false positives (FPs) because viruses are polymorphic and can be exist in different forms.

According to M. Adleman 1990, detecting a virus is quite intractable and almost impossible.

This is because according to Gödel 2003, numberings of the partial recursive functions, it is not

3
possible to create detecting mechanism. To reliably identifying a bounded-length mutating virus

is NP-complete explained in. According to the author, virus detector for certain virus strain can

be used to solve the satisfiability problem. Since satisfiability problem is known to be NP-

complete, so the detection of the malware is NP-complete. Zuo et al., 2005, claim that there exist

computer viruses whose detecting procedures have sufficiently large time complexity, and there

are undecidable viruses which have no minimal detecting procedure.

Techniques used for detecting malware can be categorized broadly into two categories: anomaly-

based detection and signature-based detection. An anomaly-based detection technique uses its

knowledge of what constitutes normal behavior to decide the maliciousness of a program under

inspection. A special type of anomaly-based detection is referred to as specification-based

detection. Specification-based techniques leverage some specification or rule set of what is valid

behavior in order to decide the maliciousness of a program under inspection. Programs violating

the specification are considered anomalous and usually, malicious. Signature-based detection

uses its characterization of what is known to be malicious to decide the maliciousness of a

program under inspection. As one may imagine this characterization or signature of the

malicious behavior is the key to a signature-based detection method’s effectiveness.

In light of this, the goal of this research is to compare techniques, instruments, and technologies

related to malware detection algorithms and their efficiencies. The study looks at case studies

and examples from the actual world to find best practices, obstacles, and possibilities in the

subject. In addition, the research will look at new developments and potential paths for malware

detection algorithms and their efficiencies, offering insights into how the world is changing to

counter new and developing threats.

4
1.2. Statement of the Problem

Despite the existence of various malware detection tools and techniques, malware remains a

significant threat to computer systems and networks. Traditional signature-based detection

methods are no longer effective against new and unknown malware threats. The increasing

complexity and diversity of malware have created a need for more advanced and effective

malware detection techniques.

The problem this study aims to address is the need for a more efficient and effective malware

detection algorithm that can detect and classify malware with high accuracy.

1.3. Aims and Objectives of the Study

The aim of this study is to evaluate a malware detection algorithm that can classify malware with

high accuracy.

The specific objectives of this study are:

1. To review existing malware detection techniques and algorithms.

2. To evaluate the performance of the proposed algorithm using a dataset of known malware

samples.

3. To compare the performance of the proposed algorithm with existing malware detection

algorithms.

5
1.4. Scope and Limitations of the Study

This study will focus on the evaluation of a malware detection algorithm that can detect and

classify malware with high accuracy. The study will be limited to the analysis of Windows-based

malware samples.

The limitations of this study include:

1. The study will only focus on Windows-based malware samples.

2. The study will not cover the detection of malware on mobile devices or other operating

systems.

3. The study will not address the issue of malware removal or mitigation.

1.5. Significance of the Study

This study is significant because it aims to evaluate a more efficient and effective malware

detection algorithm that can detect and classify malware with high accuracy. The study will

contribute to the existing body of knowledge in the field of malware detection and will provide a

foundation for future research in this area.

The study will also have practical implications for the development of more effective malware

detection tools and techniques, which will help to improve the security of computer systems and

networks.

6
1.6. Definition of Terms

Malware: Short for malicious software, refers to any type of software that is designed to harm or

exploit a computer system or network.

Malware detection: The process of identifying and detecting malware on a computer system or

network.

Malware classification: The process of categorizing malware into different types or families

based on their characteristics and behavior.

Algorithm: A set of instructions or rules that are used to solve a problem or perform a specific

task.

Signature-based detection: A method of malware detection that uses a database of known

malware signatures to identify and detect malware.

Anomaly-based detection: A method of malware detection that uses machine learning

algorithms to identify and detect malware based on its behavior.

7
 CHAPTER TWO

RELATED LITERATURE REVIEW

2.0. Introduction

This study explores into the existing literature surrounding the comparison of malware detection
algorithms and their efficiencies. This literature review aims to provide a comprehensive
overview of the key concepts, methodologies, tools, and technologies that have been explored in
previous research and scholarly works in this field by fusing and evaluating existing literature.
Furthermore, this chapter seeks to identify gaps, trends, and areas for further study, thereby
setting the foundation for the subsequent chapters of this study.

2.1. Review of Key Concepts