Academic Journal of Nawroz University (AJNU) 47

Evaluation of Open Source Web Application

Vulnerability Scanners
1 Hilmi S. Abdullah
1 IT Department, Amedi Technical Institute, Duhok Polytechnic University, Kurdistan Region - Iraq

ABSTRACT
Nowadays, web applications are essential part of our lives. Web applications are used by people for information
gathering, communication, e-commerce and variety of other activities. Since they contain valuable and sensitive
information, the attacks against them have increased in order to find vulnerabilities and steal information. For this
reason, it is essential to check web application vulnerabilities to ensure that it is secure. However, checking the
vulnerabilities manually is a tedious and time-consuming job. Therefore, there is an exigent need for web application
vulnerability scanners. In this study, we evaluate two open source web application vulnerability scanners Paros and
OWASP Zed Attack Proxy (OWASP ZAP) by testing them against two vulnerable web applications buggy web
application (bWAPP) and Damn Vulnerable Web Application (DVWA).
Keywords: Web Application Security, Open Web Application Security Project (OWASP), Vulnerability Scanner,
Penetration Testing.

1. Introduction scanners [4].

1 In recent years, web application hacking has increased Web applications vulnerability scanners perform the
dramatically. This is because the importance of the web checking process automatically. They generally have
applications in our daily lives. Ensuring the security of three components; first, the web crawler for gathering
web applications and finding their vulnerabilities is website data; second, the attacker component which
crucial as the majority of information and services on sends random and invalid input to the web application;
the internet are provided through web applications, and the last component is the analyzer which analyzes
such as e-commerce, social networking and e- the returned data, detects the vulnerabilities and
governance [1, 2]. There are several vulnerabilities generates the report. There are various available
which can result in data breach and shutdown of the scanners both commercial and free [4, 5]. In this study
web applications. Open Web Application Security we test and evaluate two open source web application
Project (OWASP) lists the top 10 web applications vulnerability scanners OWASP Zed Attack
security risks and vulnerabilities [3]. However, finding Proxy(OWSAP ZAP) and Paros.
the vulnerabilities manually is costly, time-consuming Section II provides the basic concepts of web
and difficult. Therefore, there is a need for vulnerability application vulnerabilities according to OWASP top 10
list as well as penetration testing methods in addition to
Academic Journal of Nawroz University an overview of how web application vulnerability
(AJNU) Volume 9, No 1 (2020).
Regular research paper : Published 17 Feb 2020 scanners work. Section III explains the experiment
Corresponding author’s e-mail : [Link]@[Link] environment including tested scanners and web
Copyright ©2018 Hilmi S. Abdullah.
This is an open access article distributed under the Creative applications in addition to the methodology. Section IV
Commons Attribution License.

doi : 10.25007/ajnu.v9n1a532
48 Academic Journal of Nawroz University (AJNU)

presents the results, analysis and assessment of the to perform injection and other attacks [3].
tested scanners. Finally, we provide the conclusion in • Using Components with Known Vulnerabilities: Using
section V. vulnerable components such as frameworks and
2. Background libraries might enable different attacks that lead to
2.1 Web Application Vulnerabilities data loss or server takeover [3].
There are variety of vulnerabilities that threatens web • Insufficient Logging and Monitoring: Logging of the
applications. OWASP lists the top 10 application events and persistent monitoring to discover any
security risks, following is the list for 2017 [3]. suspicious events is necessary [3].
• Injection: Injection happens when untrusted data is 2.2 Testing Methods
included in a query or a command to trick the Penetration testing of the web applications is necessary
interpreter and get unauthorized access to data [3]. prior to their launching and during their operation. The
• Broken Authentication: Attackers might steal test can be performed either automatically or manually
identities of other users and compromise [6].
passwords when authentication and session • Automated Testing: Is a technique of using software
management isn't perfect [3]. tools to scan web application pages to discover
• Sensitive Data Exposure: Attackers might get access vulnerabilities and generate reports at the end of
to sensitive data such as financial data if they are the test. There are several tools used for automated
not fully protected [3]. testing such as OWASP ZAP, Burp Suite, Paros,
• XML External Entities (XXE): These attacks will W3af, etc. [5]
result in denial of service or revealing confidential • Manual Testing: Sometimes automated testing is not
data by uploading malicious XML files which are enough to assess the vulnerabilities of the web
parsed by poorly configured XML parsers [3]. application and there is a need for human
• Broken Access Control: This vulnerability occurs intervention to perform the attacks as in social
when authenticated users’ permissions are not engineering. [5]
restricted properly, which enables the attackers to 2.3 Web Application Vulnerability Scanners
access sensitive data and change other users’ data Generally, web application vulnerability scanners
and permissions [3]. contain crawler, attacker and analyzer components [7].
• Security Misconfiguration: This security breach Firstly, the crawler component is responsible for
happens when the default configurations are finding the reachable pages of the scanned web
insecure. Continuous upgrade of servers, application as well as identifying its entry points such
frameworks and applications is necessary [3]. as HTML forms input and the parameters of GET or

• Cross-Site Scripting (XSS): The XSS vulnerability POST, etc.

enables the attackers to inject and run scripts on the Secondly, the attacker component is responsible for

victim's browser [3]. analyzing the data obtained by the crawler, then

• Insecure Deserialization: This vulnerability occurs generates values for each vulnerability type and sends

when the web application doesn't secure the form data to the web server to obtain the response.

deserialization process properly which can be used Finally, the analyzer component interprets and

doi : 10.25007/ajnu.v9n1a532
 Academic Journal of Nawroz University (AJNU) 49

analyzes the response from the web server. Then it and open source vulnerable web application
determines if a specific attack was successful. Next, it designed for testing and teaching purposes. It is
generates the report of the scan [7, 8]. Fig. 1 [8] shows written in PHP and uses MySQL database; version
the main stages of penetration testing. 2.2 is used for this experiment. It has three different
security levels low, medium and high. For this
experiment, the lowest one is used [9].
3.2 Operating System
To perform the test, Kali Linux is used as a virtual
machine in Oracle VirtualBox. Kali Linux is an open
source operating system based on Debian which is used
for penetration testing and digital forensics. For this
experiment version 3.28.0 is used. It contains several
built-in penetration tools such as OWASP ZAP, Paros,
etc. [10, 11].

Fig. 1. Main stages of penetration testing

3. EXPERIMENT ENVIRONMENT
To perform the experiment we used the operating
system Kali Linux, web application vulnerability
scanners and vulnerable web applications.
3.1 Vulnerable Web Applications
There are several available vulnerable web applications
developed intentionally for the purpose of learning and
testing. For this experiment, the following two Fig. 2. Kali Linux
vulnerable web applications have been selected which 3.3 Open Source Web Application Vulnerability
are available online and can be downloaded for free. Scanners

• DVWA: Damn Vulnerable Web Application There are various open source tools used for web

(DVWA) is a web application used by security application penetration testing. For this experiment

professionals and web developers for testing and OWASP ZAP and Paros are used, both are built-in Kali

teaching purposes. It is GNU General Public Linux as shown in Fig. 2.

License version 3; version 1.10 is used for this • OWASP ZAP: Is an open source web application
experiment. DVWA is PHP/MySQL web penetration testing tool. It is used by web

application and contains many vulnerabilities. It developers and security professionals to scan and

has four different security levels; the lowest level find the vulnerabilities of web applications. For this

has been selected for this experiment [4]. experiment version 2.7.0 is used [12, 13].

• bWAPP: buggy Web Application (bWAPP) is a free • Paros: Is a web application vulnerability scanner

doi : 10.25007/ajnu.v9n1a532
50 Academic Journal of Nawroz University (AJNU)

which is open source and cross-platform. Paros

Version 3.2.13 is used for this experiment [14, 15]. Tool ZAP PAROS ZAP PAROS
(DVWA) (DVWA) (BWAPP) (BWAPP)
Table I lists the general characteristics of the tested
Risk Level
open source scanners.
High 7 2 7 2
Medium 2 3 6 5
Table 1: General characteristics of the tested scanners
Low 4 0 8 3
OWASP ZAP Paros

License ASF2 GPL

Obviously there are several high risk (critical)
Version 2.7.0 3.2.13
vulnerabilities detected in the tested web applications
Scanning by both scanners. The majority of these vulnerabilities
Method Automated Automated
were Injection and Cross Site Scripting (XSS) which are
Status Up to date Outdated in the OWASP top 10 list for 2017.
Operating OWASP ZAP was able to detect the following critical
System Cross-Platform Cross-Platform vulnerabilities:
• SQL Injection
3.4 Methodology • Cross Site Scripting (Reflected)
We ran an automated scan for both scanners OWASP
• Cross Site Scripting (Persistent)
ZAP and Paros against the vulnerable web applications
• Remote OS Command Injection
DVWA and bWAPP which were installed on the local
• Path Traversal
host. For both scanners the default configuration were
• External Redirect
selected. Later on, when the scanning process finished, a
• Remote File Inclusion
report was generated about the detected vulnerabilities
On the other side, Paros was able to detect the
in the tested vulnerable web applications.
following critical vulnerabilities only:
4. Result and discussion
• SQL Injection
To evaluate OWASP ZAP and Paros, we compared the
• SQL Injection Fingerprinting
number and types of the detected vulnerabilities by
Overall, the performance of OWASP ZAP was better
each scanner. In addition, their features and ease of use
than Paros as can be seen in Fig. 3 which makes a
is assessed.
comparison between the tested scanners based on the
At the end of the scanning process we analyzed the
number of detected vulnerabilities.
reports generated by the tested scanners against the
tested vulnerable web applications. Both scanners
categorize the vulnerabilities per risk levels high,
medium and low. Table II presents the summary of the
detected vulnerabilities in the tested vulnerable web
applications.
Table 2: Detected vulnerabilities summary

doi : 10.25007/ajnu.v9n1a532
 Academic Journal of Nawroz University (AJNU) 51

2. Farah T., Shojol M., Hassan M. & Alam D. (2016).

25
Assessment of vulnerabilities of web applications of
No. of Vulnerabilities
20
OWASP ZAP Bangladesh: A case study of XSS & CSRF. Sixth
15 International Conference on Digital Information and
PAROS
10 Communication Technology and its Applications

5 (DICTAP), Konya, 2016, (pp. 74-78).

3. OWASP. (2018).The Open Web Application Security
0
DVWA BWAPP Project. Retrieved from [Link]

Vulnerable Web Applications 4. Makino Y. & Klyuev V. (2015). Evaluation of web

vulnerability scanners. Intelligent Data Acquisition and
Fig. 3. Comparison of the detected vulnerabilities Advanced Computing Systems:Technology and
Finally, considering the features of both scanners for Applications (IDAACS), IEEE 8th

the assessment, OWASP ZAP has more features InternationalConference vol. 1. IEEE, 2015, (pp. 399–402).

compared to Paros as well as it is more user-friendly 5. Nagpure S. & Kurkure S. (2017). Vulnerability
Assessment and Penetration Testing of Web Application.
and it is regularly updated unlike Paros which is
International Conference on Computing,
outdated. Furthermore, OWASP ZAP detected various
Communication, Control and Automation (ICCUBEA),
types of critical vulnerabilities while Paros was able to
Pune, 2017, (pp. 1-6).
detect only SQL injection.
6. Srinivasan S. & Sangwan R. (2017). Web App Security: A
5. CONCLUSION Comparison and Categorization of Testing Frameworks.
Web applications are used by people on everyday life IEEE Software, vol. 34, no. 1, IEEE, 2017, (pp. 99-102).
for various services like e-governance, shopping and 7. Suteva N., Zlatkovski D. & Mileva A. (2013). Evaluation

communication. However, the importance of web and testing of several free/open source web

applications attracts the attackers for different vulnerability scanners, 10th Conference for Informatics
and Information Technology, Bitola, Macedonia, 2013.
purposes. Therefore, there is an increasing need to
8. Jiménez R. (2016). Pentesting on web applications using
secure these web applications by penetration testing
ethical - hacking. IEEE 36th Central American and
using vulnerability scanners. In this study we tested
Panama Convention (CONCAPAN XXXVI), San Jose,
two open source scanners and compared their
2016, (pp. 1-6).
performance and features. Obviously, OWASP ZAP 9. BWAPP. (2018). A buggy Web Application. Retrieved
performed better, as it detected more vulnerabilities from [Link]
than Paros with a more diverse range of vulnerabilities. 10. Gaddam R. & Nandhini M. (2017). An analysis of various

In addition, OWASP ZAP has more features and is snort based techniques to detect and prevent intrusions

regularly updated which makes it superior to Paros. in networks proposal with code refactoring snort tool in
Kali Linux environment. International Conference on
6. References
Inventive Communication and Computational
1. Al-Khurafi, O. & Al-Ahmad. M. (2015). Survey of Web
Technologies (ICICCT), Coimbatore, 2017 (pp. 10-15).
Application Vulnerability Attacks. 4th International
11. Denis M., Zena C. & Hayajneh T. (2016). Penetration
Conference on Advanced Computer Science Applications
testing: Concepts, attack methods, and defense strategies.
and Technologies (ACSAT), Kuala Lumpur, 2015 (pp.
IEEE Long Island Systems, Applications and Technology
154-158).
Conference (LISAT), Farmingdale, NY, 2016 (pp. 1-6).

doi : 10.25007/ajnu.v9n1a532
52 Academic Journal of Nawroz University (AJNU)

12. Daud N.,Bakar K. & Hasan M. (2014) .A case study on

web application vulnerability scanning tools. Science and
Information Conference, London, 2014 (pp. 595-600).
13. OWASP ZAP. (2018). Zed Attack Proxy Project - OWASP.
Retrieved from
[Link]
ck_Proxy_Project
14. Engebretson P. (2013). The Basics of Hacking and
Penetration Testing. Waltham, MA: Syngress.
15. Goel J., Asghar M., Kumar V. & Pandey S. (2016).
Ensemble based approach to increase vulnerability
assessment and penetration testing accuracy.
International Conference on Innovation and Challenges
in Cyber Security (ICICCS-INBUSH), Noida, 2016 (pp.
330-335).

doi : 10.25007/ajnu.v9n1a532