Journal Pre-proof

EdgeSimPy: Python-based modeling and simulation of edge computing

resource management policies

Paulo S. Souza, Tiago Ferreto, Rodrigo N. Calheiros

PII: S0167-739X(23)00234-0
DOI: [Link]
Reference: FUTURE 6907

To appear in: Future Generation Computer Systems

Received date : 13 January 2023

Revised date : 17 May 2023
Accepted date : 13 June 2023

Please cite this article as: P.S. Souza, T. Ferreto and R.N. Calheiros, EdgeSimPy: Python-based
modeling and simulation of edge computing resource management policies, Future Generation
Computer Systems (2023), doi: [Link]

This is a PDF file of an article that has undergone enhancements after acceptance, such as the
addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive
version of record. This version will undergo additional copyediting, typesetting and review before it
is published in its final form, but we are providing this version to give early visibility of the article.
Please note that, during the production process, errors may be discovered which could affect the
content, and all legal disclaimers that apply to the journal pertain.

© 2023 Published by Elsevier B.V.

 Journal Pre-proof
REVISED Manuscript text UNmarked (Word or LATEX format) Click here to view linked References

EdgeSimPy: Python-Based Modeling and Simulation

of Edge Computing Resource Management Policies

of
Paulo S. Souzaa,∗, Tiago Ferretoa , Rodrigo N. Calheirosb
a School of Technology, Pontifical Catholic University of Rio Grande do Sul, Brazil
b School of Computer, Data and Mathematical Sciences, Western Sydney University, Australia

pro
Abstract
The increasing popularity of applications with tight latency requirements has motivated research on Edge Computing, which posi-
tions computing resources near data sources at the Internet’s edge. Despite the emergence of simulation tools that make prototype
validation less complex, time-consuming, and expensive, researchers and practitioners still face significant challenges when de-
veloping resource management strategies for the edge, as existing simulators fall short in providing a fine-grained model of edge
applications provisioning. To overcome this challenge, we propose EdgeSimPy, a simulation framework written in Python for mod-
eling and evaluating resource management policies in Edge Computing environments. EdgeSimPy features a modular architecture
that incorporates several functional abstractions for edge servers, network devices, and applications with built-in models for user
mobility, application composition, and power consumption that allow the simulation of various scenarios. Furthermore, we propose
re-
a novel conceptual model that accurately represents the entire lifecycle of edge applications and ensures seamless integration with
real application traces. In addition to submitting EdgeSimPy to an in-depth verification that checks the simulator implementation,
we discuss case studies that show EdgeSimPy in action in different large-scale scenarios.
Keywords: Simulation, Modeling, Edge Computing, Resource Management, Containers, Python.

1. Introduction of new resource management policies in real testbeds. Exam-

ples of such barriers are network instability and power outages.
lP
25
The emerging use cases of compute-intensive applications Such challenges favored the rise of several edge simulators, de-
with tight latency and bandwidth requirements have highlighted tailed in Section 3.1, that promise to close the gap between con-
the limitations of consolidated cloud data centers [1]. In re- ceptual research and prototyping. Despite such initiatives, re-
5 sponse, the IT industry has been moving towards Edge Comput- searchers and practitioners still face barriers when designing
ing [2], which distributes computing resources near data sources,30 resource management policies for the edge as existing simula-
enabling data processing at the network’s edge to avoid the tors fall short in providing fine-grained modeling of the edge
long round-trip times generated by traversing the Internet back- infrastructure management, which comprises a variety of pro-
haul [3]. cesses such as application provisioning, network flow schedul-
rna

10 While proximity to data sources grants Edge Computing ing, server maintenance, and others.
a natural advantage over the cloud, it also introduces signifi- 35 To overcome this challenge, this paper introduces EdgeS-
cant technical constraints. As the deployment of a large-scale imPy, a framework for modeling and simulating resource man-
edge data center in the middle of urban centers is typically un- agement policies for Edge Computing environments developed
feasible, edge infrastructures often comprise groups of devices in Python. EdgeSimPy is built on top of a modular architecture
15 with reduced computing power distributed across small physi- comprising several functional abstractions for edge servers, net-
cal spaces with limited power and cooling supply [4]. As such, 40 work devices, and applications. EdgeSimPy embodies a novel
ensuring the efficient use of resources, which is more critical conceptual model that replicates the application provisioning
than ever, also becomes challenging. Thus, efficient resource method of widely used platforms such as Docker1 , allowing
management policies need to be developed and tested to ensure seamless integration with repositories like DockerHub2 . In ad-
Jou

20 they achieve expected goals. dition to demonstrating the effectiveness of EdgeSimPy through
In addition to the cost and time required to validate resource 45 an in-depth verification that checks the correctness of the simu-
management policy prototypes, the distributed nature of edge lator implementation, we describe two case studies available in
infrastructures adds extra barriers to empirical experimentation the literature [5, 6] that show EdgeSimPy in action in different
large-scale scenarios.
∗ Corresponding The contributions of this paper are the following:
author
Email addresses: [Link]@[Link] (Paulo S. Souza),
[Link]@[Link] (Tiago Ferreto), 1 [Link]
[Link]@[Link] (Rodrigo N. Calheiros) 2 [Link]

Preprint submitted to Journal of LATEX Templates May 17, 2023

 Journal Pre-proof

50 • We provide a high-level interface that leverages the fea- shorter response times. Figure 1 illustrates the relationship be-
tures of well-known modeling solutions such as Mesa [7] tween cloud data centers, edge resources, and end devices.
and NetworkX [8] to ease the development of resource
management policies and simulator extensions (§4.1).
Cloud

of
• We propose a novel conceptual model that accurately rep-
55 resents the entire lifecycle of edge applications by repli-
cating the behavior of widely used platforms like Docker
(§4.3).
Network

...
• We implement multiple system models for simulating sev- Backhaul

...

...
eral features of edge environments, such as infrastruc-

pro
60 ture power consumption, application composition, ser-
vice workload variations, and user mobility (§4.2–4.4).
• We conduct an in-depth verification that checks the cor-
rectness of EdgeSimPy’s core features implementation
by comparing the simulation results to the expected out- Edge Edge Servers Edge Servers
65 puts (§5).
End Devices
• We demonstrate EdgeSimPy’s utility through two case
studies based on peer-reviewed papers [5] [6] that have
employed the simulator in different resource management
re- Figure 1: Interplay between cloud data centers, edge sites, and end devices.
scenarios (§6).
Similarly to cloud data centers, edge infrastructures rely
70 The remainder of this paper is organized as follows. Sec-
on virtualization technology, often through Virtual Machines
tion 2 reviews the concepts that shape the foundation of our
105 (VMs) or containers [13]. The fundamental idea behind virtu-
research, such as Cloud Computing, Edge Computing, and con-
alization is decoupling hardware (physical devices) from soft-
tainers. Then, Section 3 presents an overview of the edge simu-
ware (applications). Virtualization gives infrastructure opera-
lation landscape, highlighting the contributions of EdgeSimPy
tors fine-grained control over physical resources, which cre-
75 over existing simulators. Next, Section 4 details EdgeSimPy’s
ates several resource management possibilities, such as multi-
lP
architecture and its main components. Sections 5 and 6 check
110 tenancy (i.e., hosting multiple applications on the same server)
EdgeSimPy’s implementation correctness through an in-depth
and on-the-fly relocation of applications among servers.
verification and demonstrate its extensibility through two case
In addition to yielding a better use of resources, virtualiza-
studies. Section 7, discusses lessons learned during EdgeS-
tion allows developers to bypass repetitive tasks during appli-
80 imPy’s development, highlighting its limitations and improve-
cation deployment with template images [14]. At this point,
ment opportunities. Finally, Section 8 concludes the paper.
115 the differences between VMs and containers show up. Whereas
VM images are monolithic, most modern container solutions
2. Background follow Docker’s3 lead, splitting container images into read-only
rna

layers representing software instructions. This difference may

Over the last decade, Cloud Computing has become one of seem subtle, but it affects the entire lifecycle management of
the most attractive options for hosting applications over the In-120 applications (i.e., building, deploying, and terminating opera-
85 ternet [9]. Cloud computing relies on centralized computing re- tions) [15].
sources in data centers, reducing infrastructure management’s When a container needs to be spawned, a new writable layer
burden and allowing providers to offer cloud services at afford- is created on top of the container image’s filesystem. All changes
able prices for individuals and organizations. made to the container are stored in that top writable layer, so
With Cloud Computing acting as a catalyst for digital trans-125 that image layers remain unchanged regardless of application-
90 formation, the spotlight turned to use cases such as augmented level changes, which allows containers to share common con-
reality [10] and online gaming [11], which raised the bar re- tainer layers [16]. On the other hand, spawning a VM requires
Jou

garding the use of software applications in people’s daily lives. the creation of a complete copy of the base image, as no persis-
Eventually, it became evident that centralization around a few tence measure is taken to prevent applications from changing
cloud data centers results in them being geographically distant130 instructions from their base images. Consequently, provision-
95 from end devices, resulting in response times that conflicted ing a VM-based application is generally slower and generates
with the real-time requirements of those applications [12]. higher disk demand than a container-based application [17].
This limitation paved the way for the emergence of Edge
Computing [2], which adopts a decentralized model where ap-
3 [Link]
plications are hosted on resources (e.g., small servers and single-
100 board computers) at the edge of the network, which results in #images-and-layers

2
 Journal Pre-proof

In addition to the differences in template images, VMs and development and validation of resource management strategies
containers rely on different types of virtualization. In the VM for cloud data centers. Similarly, the dawn of Edge Computing
135 model, a control layer, known as a hypervisor, virtualizes the190 has motivated the development of several edge simulators. This
hardware, giving each VM a dedicated virtual CPU, memory, section starts with an overview of simulation tools for Edge
I/O, and network devices. In the container model, a control Computing (§3.1). Then, it highlights the differences and con-

of
layer, known as container runtime, virtualizes the host OS ker- tributions of EdgeSimPy over those existing solutions (§3.2).
nel instead of the hardware, so containers share the host OS
140 kernel while isolation is handled at the OS level. While con- 3.1. Edge Computing Simulators
tainerized applications do not need a standalone OS, containers195 Sonmez et al. [22] highlight the limitations of cloud and
display a lower virtualization overhead than VMs at the cost of network simulators when modeling the characteristics of edge
weaker isolation among co-hosted instances [13]. environments. While cloud simulators such as CloudSim lack

pro
There is no consensus about what type of virtualization should user mobility and wireless support, network simulators are not
145 be employed on edge infrastructures. Some researchers argue focused on modeling edge servers and users. Accordingly, the
that VMs and containers should be employed depending on the200 authors present EdgeCloudSim, a simulator that implements
deployment needs, enabling a broader spectrum of demands to user mobility, edge device power modeling, and network man-
be managed more efficiently [18]. In such a line of reasoning, agement, facilitating the prototyping of placement strategies in
containers can fit better when shorter provisioning times and Edge Computing environments.
150 small footprints are needed, while VMs can deliver better secu- Qayyum et al. [23] argue that edge simulators abstract net-
rity and isolation in multi-tenant deployments. 205 work infrastructure characteristics, restricting the options for
Despite the potential use cases for both types of virtualiza- designing new resource management strategies. The authors
tion, containers have been taking the lead as the prime architec- introduce a new simulator, FogNetSim++, which models the

155
re-
ture for deploying applications on edge computing devices, as
their small footprint and low virtualization overhead fit greatly
to the resource constraints of edge infrastructures while also210
meeting strict provisioning time constraints of edge applica-
power consumption of edge devices, supports various commu-
nication protocols (e.g., MQTT and CoAP), and simulates the
handover of mobile users among network devices. Finally, the
authors present a use case demonstrating FogNetSim++ effec-
tions [19]. tiveness in modeling placement and scheduling policies for the
At a glance, the virtualization of the edge infrastructure edge.
160 implies the reuse of consolidated resource management tech- Puliafito et al. [24] discuss the critical role of application
niques already adopted in Cloud Computing. However, for the215 migration in preserving low latency for users despite their mo-
performance expectations of latency-sensitive applications run-
lP
bility and how edge simulators lack the features needed to eval-
ning at the edge to be met, new properties such as location uate migration decisions. Based on these observations, the au-
awareness are required to be considered, highlighting the need thors propose MobFogSim, a simulator that models the appli-
165 for new resource management approaches. cation migration process based on user mobility (including at-
Although Edge Computing displays significant growth po-220 tributes such as speed and moving direction) and the coverage
tential, conducting experimental research is challenging for some area of access points spread in the environment.
reasons. First, the proximity between computing resources and Amarasinghe et al. [25] present a novel simulator called
end devices imposes challenges related to positioning the in- ECSNeT++, focused on deploying and processing stream-based
frastructure since deploying large-scale data centers in the mid-
rna

170
applications in Edge Computing environments. Unlike other
dle of urban centers might not be feasible. Consequently, edge225 simulators, ECSNeT++ enables fine-grained resource manage-
infrastructures are composed of small interconnected servers ment for streaming applications, where allocation policies can
dispersed in the environment, making it difficult to conduct re- determine how tasks are processed at the core level on edge de-
producible experiments, as multiple external factors can affect vices (where each CPU core has a processing queue), aiming
175 results (e.g., network instability and power supply outages). to reduce the network delay and power consumption of edge
In addition, experimental research may comprise several230 devices.
testing stages until prototypes are turned into consumer-ready Lera et al. [26] introduce YAFS, an edge simulator focused
products. Accordingly, conducting the entire experimentation on evaluating allocation decisions for composite applications
process in real testbeds implies investing time and budget in (i.e., applications composed of multiple components) on edge
180 computing and networking resources, power supply, and cool- infrastructures. According to the YAFS model, routing poli-
Jou

ing with no guarantees of returns. As a result, simulation tools235 cies coordinate the application communication across the net-
emerge as catalysts for early research on Edge Computing, as work, allowing modules of a given application to be allocated
evaluation in real testbeds is better suited for prototypes in more on different edge devices. The authors present several YAFS
advanced stages. use cases, including scenarios with dynamic scheduling of ap-
plication components, infrastructure failures, and user mobility.
185 3. Related Work 240 Jha et al. [27] discuss the complexity of the Cloud-Edge-IoT
ecosystem, which involves allocating resources across hetero-
Over the past decade, simulation tools like CloudSim [20] geneous devices using multiple network protocols (e.g., LoRa
and GreenCloud [21] were broadly adopted to accelerate the
3
 Journal Pre-proof

Table 1: Summary of built-in features supported by existing simulators and EdgeSimPy.

Simulator Server Power Network Power Network Routing User Mobility Application Composition Container Lifecycle
EdgeCloudSim [22] ✘ ✘ ✘ ✔ ✘ ✘
FogNetSim++ [23] ✔ ✔ ✔ ✔ ✘ ✘
MobFogSim [24] ✔ ✘ ✘ ✔ ✘ ✘
ECSNeT++ [25] ✔ ✔ ✔ ✔ ✔ ✘

of
YAFS [26] ✔ ✘ ✔ ✔ ✔ ✘
IoTSim-Edge [27] ✔ ✘ ✔ ✔ ✔ ✘
IoTSim-Osmosis [28] ✔ ✘ ✔ ✘ ✔ ✘
iFogSim2 [29] ✔ ✘ ✔ ✔ ✔ ✘
EdgeSimPy (this work) ✔ ✔ ✔ ✔ ✔ ✔

pro
Table 2: Summary of potential use cases supported by existing simulators and EdgeSimPy.
Simulator Service Migration Network Flow Scheduling Container Registry Management Maintenance Operations
EdgeCloudSim [22] ✘ ✘ ✘ ✘
FogNetSim++ [23] ✘ ✔ ✘ ✘
MobFogSim [24] ✔ ✘ ✘ ✘
ECSNeT++ [25] ✘ ✔ ✘ ✘
YAFS [26] ✘ ✘ ✘ ✘
IoTSim-Edge [27] ✘ ✔ ✘ ✘
IoTSim-Osmosis [28] ✘ ✔ ✘ ✘
iFogSim2 [29] ✔ ✔ ✘ ✘
EdgeSimPy (this work) ✔ ✔ ✔ ✔

245
re-
and Zigbee) and messaging protocols (e.g., CoAP and MQTT).
After highlighting the lack of edge simulators considering the
protocols of the Cloud-Edge-IoT ecosystem, the authors present280
the IoTSim-Edge simulator. The simulator allows the prototyp-
simulators lack native support for managing the lifecycle of
containerized applications. At first, extending existing simu-
lators may seem trivial as most of them are built with program-
ming stacks that facilitate the inclusion of new features. How-
ing of resource management strategies for the edge based on the ever, in this case, several changes may be needed to allow the
energy consumption of edge devices, application composition, evaluation of container management strategies at the edge, such
user mobility, and communication protocols. as including several new entities (e.g., container registries, im-
250 Alwasel et al. [28] make a case for Osmotic Computing, a285 ages, layers, etc.) and modeling the container provisioning pro-
paradigm that involves workload migration among cloud data cess from scratch, as it differs significantly from the VM model,
lP
centers and edge devices based on performance and security as discussed in Section 2.
events. The authors present a novel simulator, IoTSim-Osmosis, EdgeSimPy models aspects of edge computing that are also
focused on Osmotic Computing scenarios, which implements a supported by existing simulators, such as user mobility, energy
255 variety of models for data transmission, energy consumption,290 consumption (for computing and network devices), and appli-
and application performance. Finally, the authors present a use cation composition. Complementarily, it provides functional
case where IoTSim-Osmosis is used to model strategies that op- abstractions that are not covered by existing edge simulators
timize performance, energy consumption, and budget in Cloud- (e.g., container registries, container images, and container lay-
rna

Edge scenarios. ers), enabling the simulation of the whole lifecycle of container-
260 Mahmud et al. [29] highlight the lack of support for real295 ized applications (e.g., placement, scheduling, migration, up-
datasets on edge simulators, which limits the evaluation of re- date, and removal of operations). Table 1 summarizes the main
source management policies in edge environments comprising differences between EdgeSimPy and related simulators regard-
composite applications. Then, the authors introduce iFogSim2, ing built-in features.
a simulator with native support to real datasets that incorporates In addition to its built-in functionalities, EdgeSimPy aims
265 modeling and simulation of service migration in multi-tier in-300 to contribute to the community by providing more comprehen-
frastructures (e.g., Cloud-Edge), user mobility, service orches- sive support for specific use cases, as shown in Table 2. Al-
tration, and edge devices clustering. They present use cases though some simulators like Although specific simulators such
demonstrating iFogSim2 effectiveness in simulating several re- as MobFogSim and iFogSim2 support the modeling of service
source allocation scenarios at the edge. migration strategies, they focus predominantly on VM-based
Jou

305 migration models. For instance, although MobFogSim provides

270 3.2. Discussion a ContainerVM class, such an entity follows the VM migra-
As discussed in the previous section, the popularization of tion model, where containers are relocated from a source to a
Edge Computing motivated the development of several simula- destination server, overlooking the shareable structure of con-
tion tools that address aspects such as user mobility and infras- tainer images and the abstraction of container registries. In
tructure heterogeneity, which are not covered by cloud simula-310 contrast, EdgeSimPy’s abstractions for container-related enti-
275 tors despite being critical in edge scenarios. ties (i.e., container registries, container images, and container
Despite the significant research interest in simulation mod- layers) allow it to support the design of relocation strategies
eling focused on Edge Computing environments, existing edge
4
 Journal Pre-proof

for containerized applications. In the same way, these new ab-350 adhering to this predefined structure, EdgeSimPy can automati-
stractions implemented by EdgeSimPy make room for other cally identify entity input metadata and construct the simulated
315 use cases. For instance, as EdgeSimPy represents container scenario, even in cases where custom attributes and relation-
registries as domain-specific services, it supports the design of ships have been specified. Figure 3 depicts a sample EdgeS-
provisioning policies that manage such entities aiming at opti- imPy input file containing the metadata of a given application

of
mizations in the lifecycle of containerized instances. In addi-355 entity.
tion, EdgeSimPy facilitates modeling maintenance operations
320 for physical resources (e.g., servers, network devices) and ap- edgesimpy_dataset.json
plications, which is not supported by existing simulators. An {

in-depth analysis of EdgeSimPy’s suitability in two case stud- "Application": [

ies and a comparison with existing simulators in these scenarios

pro
"attributes": {

is provided in Section 6. Entity attributes with

"id": 1,

multiple data formats "privacy_requirement": "medium"

},

325 4. EdgeSimPy Architecture "relationships": {

"services": [

EdgeSimPy’s primary design goal is to support researchers {

"class": "Service", "id": 1

One-to-many relationship

interested in evaluating resource management strategies in Edge pointing to multiple entities },

Computing infrastructures. For example, EdgeSimPy supports ...

modeling different types of resource allocation decisions, such ],

"preferred_host": {

330 as placement, migration, scheduling, and maintenance, while One-to-one relationship

"class": "EdgeServer", "id": 1

considering the heterogeneity of the infrastructure (power con- pointing to a single entity

335
sumption, resource capacity), users (mobility, access profile),
and applications (composition, performance requirements). Fig-
ure 2 illustrates a sample EdgeSimPy simulation scenario in
which entities interact with each other and different resource
re- One-to-one relationship

pointing to a Python function

}
...

},

},

"comm_model": my_custom_function

management decisions affect the environment.

Figure 3: Sample EdgeSimPy input file.
Routing Scheduling Maintenance Service Migration/Placement

User Request Layer Pulling Server Update Provisioning Simulated entities can carry geospatial metadata, which fa-
cilitates the integration of datasets containing real or synthetic
lP
S3 CI3 CL4
coordinates to EdgeSimPy and allows the modeling of events
Edge Server 3 (ES3) such as user mobility. By default, EdgeSimPy uses the map
CL1
360 model proposed by Aral et al. [32], which divides the environ-
S1 CI1
User 1
L1 ES1
CL2
ment into hexagonal cells.
S2 CI2 Once the simulation starts, EdgeSimPy triggers a monitor-
L3

CL3
L4
L2

Registry
S3 CI3 CL4 ing mechanism that stores snapshots of the entity’s state at the
ES2 L5 ES3
Services Images Layers end of each time step. Simulation logs are stored in MessagePack4 ,
rna

CPU + RAM Disk 365 a binary serialization format designed to be a faster and smaller
alternative to JSON [33]. Instead of writing data to disk each
time step, EdgeSimPy stores the simulation output at config-
Figure 2: Sample EdgeSimPy simulation scenario. While “L” and “ES” denote urable intervals of time steps, reducing the I/O pressure during
the network links and edge servers, “S”, “CI”, and “CL” represent the services,
container images, and container layers. the simulation.
370 EdgeSimPy’s flexibility stems from a modular architecture
Before starting the simulation, EdgeSimPy expects an input (depicted as a block diagram in Figure 4), which divides the
file defining the simulated scenario. EdgeSimPy input files are functional abstractions into four layers (Core, Physical, Log-
written in the JavaScript Object Notation (JSON) format, which ical, and Management). Each abstraction is self-contained to
340 has a human-friendly file structure [30] and is widely adopted streamline the integration of new features and algorithms.
in various software domains [31], which facilitates EdgeSimPy375 The remaining of this section details the components that
Jou

integration with other tools. comprise each layer of EdgeSimPy’s architecture.

EdgeSimPy input files organize the metadata of each sim-
ulated entity into a well-defined structure comprised of two 4.1. Core Layer
345 distinct information groups: attributes and relationships. At- Edge computing environments can be seen as complex sys-
tributes refer to the internal characteristics of entities, such as tems with several entities that interact with each other in a non-
edge server capacity, network link bandwidth, application de-380 linear way. This ecosystem may assemble emergent phenom-
lay, among others. Relationships represent the associations be-
tween entities (e.g., a service’s host or a user’s applications). By
4 [Link]

5
 Journal Pre-proof

2. Executing
3. Activating Agents

1. Loading

Placement Migration Routing Scheduling Maintenance Dataset

Resource Management
and Advancing

Algorithms Simulation Clock

Management No

Container Container Container

of
Application Service Yes
Registry Image Layer 6. Displaying
5. Stopping 4. Monitoring

Results Criterion is Met? System State

Logical

Base
Network
Edge
User Figure 5: EdgeSimPy’s simulation workflow.
Station Switch Server
Physical

pro
files or Python dictionaries, spawning simulated entities accord-
Core
420 ingly (Figure 5, step 1). After loading the scenario, EdgeSimPy
EdgeSimPy starts an iterative process until a user-defined stopping criterion
is met. At each time step, EdgeSimPy executes user-defined
resource management policies, calls the activation regime for
Figure 4: EdgeSimPy architecture. updating the agents’ state, increments the simulation clock, and
425 collects logs about the system state, respectively (Figure 5, steps
2–4).
ena [34], which are the product of interactions between enti-
Once the stopping criterion is met, EdgeSimPy stops the
ties. An example of an emerging phenomenon at the edge is
simulation and displays the metrics and logs collected during
infrastructure saturation generated by a series of poor resource

385
allocation decisions. While detecting such type of emerging
re-
phenomena is key to avoiding erroneous allocation decisions,
understanding its causes through the attributes of individual en-
tities is challenging as it arises from the interaction between
430
the simulation (Figure 5, step 6). Thanks to EdgeSimPy’s de-
coupled architecture, it is possible to define custom stopping
criteria, resource management algorithms, and personalized rou-
tines for collecting and exhibiting simulation metrics.
multiple entities.
4.2. Physical Layer
One of the most popular techniques for studying emergent
390 phenomena is Agent-Based Modeling (ABM) [35] [36]. ABM- The Physical layer contains functional abstractions for users
based simulations represent the world through a bottom-up ap-435 and resources that comprise the edge infrastructure. Regard-
lP
proach, where independent entities (called agents) interact with less of their distinct functions, all components in the Physi-
each other and the environment over time according to commu- cal layer have a coordinates attribute, which carries the com-
nication and decision-making rules. Most ABM-based simula- ponent’s geospatial information. Physical entities that provide
395 tion systems adopt the Fixed-Increment Time Advance (FITA) networking capabilities leverage the features of NetworkX [8],
strategy [37], which advances the simulation clock in fixed in-440 a well-known graph library for manipulating complex networks
crements of time delta (∆). If multiple events are scheduled that ships several built-in methods (e.g., shortest path and com-
for the same time step t, they are considered to have occurred munity finding).
concurrently at the end of t. The remainder of this section discusses the particularities of
rna

400 FITA-based simulators must define two properties: the gran- each component in the Physical layer.
ularity of ∆ and the agents’ activation regime [37]. ∆’s granu-
larity affects the accuracy of the simulation output and the sim-445 4.2.1. Base Stations
ulation time—lower ∆ yields higher accuracy to the simulator Base Stations act as gateways in the edge network, provid-
as it reduces the number of events computed in the same time ing wireless connectivity for seamless communication between
405 step at the cost of longer runtime. The activation regime deter- users and edge servers. EdgeSimPy assumes that the base sta-
mines the order in which the simulator computes simultaneous tions cover the entire map area so that users always have con-
events at the end of each time step, which can affect the simu-450 nectivity regardless of location. As such, the set of coordinates
lation output, especially if simultaneous events influence each of the base stations comprehends the whole available area for
other. user transit, so users cannot be in a position that represents a dif-
ferent coordinate from all base stations, as they would not have
Jou

410 EdgeSimPy employs the ABM approach, representing en-

tities of edge scenarios as interactive agents. Features that com- network connectivity. Base stations on EdgeSimPy embody
prise the simulation core (i.e., agent modeling, activation regime,455 multiple customizable attributes, such as energy consumption
and time advance) are managed by Mesa [7], a well-known and wireless latency, enabling various scenarios to be modeled.
ABM framework that ships several built-in modules that en- In addition to providing wireless connectivity, EdgeSimPy
415 compass the building, analysis, and visualization of ABM sim- automatically handles user handoff between base stations based
ulations. EdgeSimPy’s simulation workflow is depicted in Fig- on user mobility patterns. Accordingly, EdgeSimPy allows for
ure 5. 460 a realistic simulation of users moving through the edge network
When EdgeSimPy is started, it loads input data from JSON and the associated changes in connectivity as they transition

6
 Journal Pre-proof

from one base station to another. In addition, base stations can host multiple services simultaneously. In addition to ca-
can be equipped with network switches for wired connectiv- pacity parameters such as CPU, RAM, and disk storage, edge
ity and network flow management and edge servers for hosting servers can also have performance parameters like Million In-
465 services, ensuring an adaptable model for edge environments. structions Per Second (MIPS), which allows for an alternative
EdgeSimPy can also be customized to support map mod-520 representation of server performance and resource allocation.

of
els where a single base station covers multiple coordinates, and It is possible to define the distribution of MIPS among applica-
attributes such as wireless latency are affected by the distance tions on a server based on custom criteria, allowing the model-
between the user’s and base station’s positions. This allows the ing of advanced phenomena such as resource contention among
470 modeling of various connectivity scenarios and the analysis of co-hosted applications.
their impact on the performance of edge applications. 525 Technical specifications such as hardware resources and re-
source usage affect the power consumption of edge servers dur-

pro
4.2.2. Network Switches ing simulation. EdgeSimPy incorporates three built-in generic
Network switches provide wired connectivity between in- power consumption models (LinearPowerModel, QuadraticPow-
frastructure components (e.g., base stations and edge servers) erModel, CubicPowerModel) [41], which assume that the power
475 and manage data flow in the network. These components ship530 consumption of edge servers grows according to their CPU us-
multiple configurable parameters, such as chassis types and vary- age following linear, quadratic, and cubic functions, respec-
ing numbers of ports with specific delay and bandwidth proper- tively. EdgeSimPy’s energy modeling enables the implementa-
ties. In EdgeSimPy, network switches are modeled as nodes in tion of advanced features, such as temporarily turning off edge
the graph representing the network topology. servers to save energy. As the properties of power models are
480 Although network switches are used as network nodes by535 fully encapsulated, EdgeSimPy supports custom power models
default, EdgeSimPy allows other entities to be modeled for this for edge servers.

485
re-
purpose, enabling a variety of networking scenarios to be sim-
ulated. For example, cars can act as topology nodes in vehicu-
lar networks, serving the environment as intermediate data ex-
change and communication entities. 540
As edge servers have static coordinates, they are immobile
by default. Nevertheless, EdgeSimPy can be extended to assign
mobility models to edge servers, allowing the representation of
mobile devices with computing capabilities, such as drones or
EdgeSimPy assumes that user requests are protected by Qual- Single-Board Computers (SBCs) connected to automobiles.
ity of Service (QoS) policies so that one user’s request does not
negatively affect others. On the other hand, more demanding 4.2.4. Users
data transfers between edge servers (e.g., service migrations) As part of the Physical layer, users have a coordinates at-
are modeled as network flows, sharing the bandwidth of net- tribute that defines their location on the map. Users can remain
lP
490

work links. The duration of a network flow depends on the545 in the same position during the entire simulation or move ac-
bandwidth of the links it spans over and the resource allocation cording to mobility models. By default, EdgeSimPy incorpo-
policy of the network switches. Whenever a network flow starts rates two mobility models, Random and Pathway [42], which
or ends, EdgeSimPy runs a flow scheduling algorithm to update can be easily replaced by other synthetic models or real mobil-
495 the occupation of the involved links, redistributing the available ity traces.
bandwidth, if applicable. 550 Users and applications are linked by a many-to-many re-
Since a network flow can use a path with links contain- lationship, which allows a user to access multiple applications
ing different bandwidth demands, EdgeSimPy normalizes the or even an application to be accessed jointly by multiple users.
rna

bandwidth available to a network flow to the lowest available Following this design principle, each user has properties that
500 bandwidth between the links in its path. The Max-Min Fairness define their delay and availability requirements for each ap-
algorithm [38] is used as the default flow scheduling algorithm,555 plication they access. As each entity is self-contained, adding
dividing network bandwidth proportionally to the demand of new user requirements such as security and budget is possible,
flows. As flow scheduling logic is fully encapsulated, it is pos- which opens room for prototyping custom allocation strategies.
sible to define custom flow scheduling algorithms without bur- Each user has their access pattern, specifying when they
505 den. will call their applications and how long each access will last.
During simulation, the network’s power consumption varies560 By default, EdgeSimPy incorporates two user access pattern
according to resource usage and the technical features of net- templates, Random and Circular. While the former arbitrarily
work switches. By default, EdgeSimPy includes the network defines when and for how long the user will access their appli-
Jou

power models proposed by Conterato et al. [39] and Riviriego cations, the latter establishes a pattern that repeats indefinitely.
510 et al. [40] However, the behavior of power models is fully en- Figure 6 illustrates four users using the Random and Circular
capsulated, allowing new models to be implemented without565 access patterns.
requiring changes to the simulator’s base features. As the user access patterns are defined through independent
classes, it is possible to easily define new patterns. This feature
4.2.3. Edge Servers allows EdgeSimPy to model different workloads, from stream-
Edge servers are used to host services. EdgeSimPy assumes ing to batch processing applications and serverless functions.
515 that the edge infrastructure is virtualized so that edge servers570 As a user’s access can be intermittent, allocation policies may
choose to deprovision applications during idle periods, which
7
 Journal Pre-proof

User Access Pattern

resources required by the service instance and therefore are un-
User 1 Random(start=2, duration=[1, 4], interval=[3, 5])
610 related to the service’s image. Each service also has a state
User 2 Circular(start=4, duration=[Infinity], interval=[0]) attribute, which defines whether it is stateless or stateful.
User 3 Circular(start=3, duration=[3], interval=[3]) Services leverage a layered file system model to separate
User 4 Circular(start=1, duration=[1], interval=[1, 2])
service components and ensure that changes in one container-

of
ized service do not affect other co-hosted services using the
615 same base image. This containerization approach provides an
User 1
efficient and isolated environment for hosting services in the
User 2
edge infrastructure. The layered file system model used by ser-
User 3
vices in EdgeSimPy is shown in Figure 7.
User 4

pro
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11 T12 T13 T14 T15 T16 T17 T18 T19 T20

Time
Container Layer
Custom

(Read/Write) Data (25 MB)

Figure 6: EdgeSimPy’s built-in user access patterns. Custom
(18 MB) Container Layer

Data (Read/Write)
Django (32 MB)

Image Layers
Image Layers

Python (39 MB) Shared

Java (47 MB)
can be beneficial in terms of resource saving or harmful if there (Read-Only) (Read-Only)
Layer
are long application provisioning times after the user’s request Debian (61 MB) Debian (61 MB)
(as in the case of serverless functions facing cold starts [43]).
Service A (Container) Service B (Container)
575 4.3. Logical Layer
re-
The Logical layer comprises functional abstractions for ap-
plications running on the edge infrastructure. Despite support-
ing VMs, EdgeSimPy adopts containerization as the default vir-
tualization model (see details in §2). Accordingly, abstractions
Host Server

Figure 7: Layered file system used by service in EdgeSimPy.

580 for container registries, container images, and container layers The layered file system used by services divides the ser-
are provided. The rest of this section describes the components620 vice image into two parts: the container and image layers. In
of the Logical layer. this setting, each service has its container layer with its run-
time files and user session data. As the container layer is not
shared with co-hosted services, it has read-write permissions.
lP
4.3.1. Applications
Conversely, image layers hold read-only permissions as they
Applications are modeled as abstract entities representing
625 provide static files, libraries, and dependencies, and are shared
585 data flows involving multiple services. This way, the applica-
among co-hosted services. This separation enables a stream-
tion services are allocated within the infrastructure rather than
lined relocation process and ensures that services can be effi-
the applications themselves. As EdgeSimPy models applica-
ciently transferred across edge servers without affecting other
tions as self-contained entities, they can receive custom attributes,
instances or interfering with shared resources.
such as priority and budget, which enables modeling specific
630 The state attribute plays a crucial role in how service is re-
590 scenarios.
located. Stateless services maintain no user session data or run-
rna

EdgeSimPy calculates the latency of a given based on the

time state, allowing them to be relocated without downtime, as
time it takes to visit the servers that host all its services. The
their container layer can be easily discarded on the target host
default application implementation in EdgeSimPy assumes that
and recreated on the destination host. In contrast, stateful ser-
the data flow starts at the application users and passes through
635 vices require the transfer of both image layers and the container
595 all the application services sequentially.
layer containing user session data, resulting in a brief downtime
In addition to the built-in application communication be-
while the service’s state is transferred to the destination host.
havior, EdgeSimPy supports custom communication patterns to
model multiple software architectures, from monoliths to mi-
4.3.3. Container Registries
croservices [44] and stream applications [45]. It is also possi-
Container registries are the main component when allocat-
600 ble to specify custom communication policies among the ser-
640 ing a service in the edge infrastructure, as the service’s con-
Jou

vices that compose a given application based on various crite-

tainer image is pulled from them to the destination host. A con-
ria, such as inter-service latency and the characteristics of the
tainer registry is a containerized service built on top of a registry
service hosts.
image that embeds image distribution and storage functionality.
Thus, a container registry also has its own CPU and memory re-
4.3.2. Services
645 quirements for performing image distribution processing.
605 Services in EdgeSimPy are modeled as container instances
EdgeSimPy supports the definition of custom policies for
within the infrastructure. While a service’s disk demand corre-
selecting from which container registries container images are
sponds to the size of the layers that comprise its container im-
pulled to the edge servers. This feature fosters the design of re-
age, its CPU and memory demand describe the computational
source management strategies that optimize service provision-
8
 Journal Pre-proof

650 ing by considering factors such as network usage and the loca- resource management strategies that could minimize applica-
tion of container registries. Additionally, as container images tion provisioning time by selectively choosing hosts already
follow a layered file system model, EdgeSimPy makes room possessing the necessary service layers.
for resource management strategies that leverage multiple con-
tainer registries to download the layers of a given image. 705 4.4. Management Layer

of
655 EdgeSimPy also allows defining how many layers of a par- In addition to modeling the behavior of several entities of
ticular image are downloaded simultaneously to a given host. Edge Computing scenarios, EdgeSimPy provides fine-grained
This level of control enables the design of resource manage- control over network and edge server resources. Consequently,
ment strategies that optimize service provisioning by balancing it facilitates prototyping various resource management policies,
multiple metrics of interest, such as network usage, provision-710 such as:
660 ing time, and resource availability.

pro
As container registries are modeled as domain-specific ser- • Service Placement: Defining the placement of applica-
vices that distribute container images across the edge infrastruc- tion services in the infrastructure represents a vital deci-
ture, EdgeSimPy enables dynamic provisioning of container sion to ensure the efficient use of resources. By support-
registries as it does with regular services. This feature allows ing the definition of custom user access patterns, EdgeS-
665 users to adapt the container registry deployment to changing715 imPy enables the modeling of online and offline place-
network conditions, resource requirements, or application de- ment strategies [46, 47]. In offline placement scenarios,
mands, ensuring efficient image distribution and optimized re- the allocation policy has a priori knowledge about all the
source utilization in the edge environment. applications it needs to provision. In online placement
scenarios, application provisioning requests occur at run-
4.3.4. Container Images 720 time, and provisioning policies must allocate application
re- services on demand.
670 Container images embed the basic functionality for services.
By default, each image has a tag attribute representing its ver- • Service Migration: Given the strict latency requirements
sion, which enables the modeling of scenarios where new image of applications running on the edge infrastructure, user
versions are released during simulation. Like applications, con- mobility may require relocation of services at runtime.
tainer images are modeled as abstract entities, so they have no725 To support this functionality, EdgeSimPy supports the
675 resource requirements by themselves. Instead, the disk demand modeling of allocation policies with fine-grained control
of a given container image results from the size of its layers. over the migration process. In this way, migration poli-
EdgeSimPy models container images according to the Open cies can define which edge servers should host the ser-
Container Initiative (OCI) 5 format, a well-known standard for
lP
vices, from which container registries the layers should
container images. By adhering to the OCI format, EdgeSimPy730 be pulled, and which network paths should be used in
680 can incorporate metadata from real-world container images from this process.
repositories like DockerHub, providing a more accurate repre-
sentation of services in the edge infrastructure. • Maintenance: Updates for applications and physical de-
In addition to providing compatibility with real image traces, vices are often released to add new features, fix bugs, or
EdgeSimPy’s container image format allows the definition of mitigate security vulnerabilities. As components of the
685 service provisioning constraints to model edge-specific scenar-735 Physical and Logical layers have versioning attributes,
ios. For instance, it is possible to define datasets with con- EdgeSimPy supports the modeling of maintenance sce-
rna

tainer image specifications that narrow the service provisioning narios, incorporating decisions such as the order in which
options based on the architecture and hardware capabilities of components are updated.
available hosts. • Network Flow Scheduling: In large-scale edge scenar-
740 ios, events such as user mobility may trigger the provi-
690 4.3.5. Container Layers sioning of multiple applications simultaneously. How-
Container layers represent the instructions aggregated into ever, the lack of network management can allow large
container images. Each container layer depicts a set of files flows to indiscriminately saturate the network, causing
added or modified concerning the previous layer. In EdgeS- the starvation of smaller flows and reduction of the over-
imPy, container layers can be identified by a digest attribute,745 all network performance [48] [49]. Accordingly, EdgeS-
enabling hosts to check the integrity of downloaded container imPy allows the definition of scheduling strategies for
Jou

695

layers. network flows that can control the priority of each flow
Each container layer carries attributes representing its soft- based on objectives modeled through built-in or custom
ware instruction and disk size. As container images in EdgeS- attributes.
imPy adhere to a layered filesystem model, co-hosted services
700 can share read-only image data, resulting in considerable disk750 The entities comprising EdgeSimPy’s architecture shape a
savings. This model makes room for the design of layer-aware robust platform for modeling various Edge Computing scenar-
ios, where different resource allocation policies can be proto-
typed and validated without burden. In addition, each compo-
5 [Link]
nent is designed to work independently, which facilitates the
9
 Journal Pre-proof

755 inclusion of new attributes and entities to the simulator, extend- Workflow 1 Workflow 2
ing the breadth of potential EdgeSimPy use cases. The next S1 S2 S3
section describes the verification process used to demonstrate User 1 User 2

the correctness of EdgeSimPy implementation.

S3

of
User 1 S1 S2
5. EdgeSimPy Verification L1 ES1
760 One of the main advantages of simulation is that it allows

L4
L2

L3
researchers to understand and explain real-world phenomena
with lower complexity and cost than empirical experimenta- L5 ES2
tion [50]. As modeling all the details and behaviors of a real-
L8

pro L6

L7
world system might be infeasible given the high complexity in-
765 volved, simulators usually make assumptions and abstractions ES3 L9
User 2
about the real world. While these can reduce the simulation
complexity, they inherently add inaccuracies to the model [51].
One of the most critical tasks in simulation studies is check- Figure 8: Overview of the infrastructure considered in EdgeSimPy’s verifica-
ing if a simulator delivers acceptable accuracy levels given the tion. Symbols “L”, “ES”, and “S” denote the infrastructure’s network links,
770 assumptions and abstractions it implements. This task gener- edge servers, and services.
ally involves two processes: validation and verification [52].
While validation determines whether the conceptual model ac-
curately represents the real world, verification checks whether State: 0 MB
State: 0 MB
State: 9 MB

CPU Demand: 1 core

CPU Demand: 1 core
CPU Demand: 1 core

775
re-
the conceptual model’s implementation is correct.
The conceptual model adopted in EdgeSimPy is based on
well-known abstractions representing user mobility, network
scheduling, and application provisioning, which have already
been formalized and discussed in past research [42] [53] [54].
Memory Demand: 1 GB

Layer 1(L1) - 12 MB
Image 1
Memory Demand: 1 GB

Layer 4(L4) - 9 MB
Layer 3(L3) - 9 MB
Layer 2(L2) - 6 MB
Layer 1(L1) - 12 MB
Image 2
Memory Demand: 1 GB

Layer 1(L1) - 12 MB
Image 1
Service 1 (S1) Service 2 (S2) Service 3 (S3)
As EdgeSimPy adopts such abstractions without further modi-
780 fication, model validation is out of the scope of this work. Application 1 Application 2
The remaining of this section presents a verification that
demonstrates the correctness of the implementation of EdgeS- Figure 9: Application specifications used in EdgeSimPy’s verification.
lP
imPy’s conceptual model. The verification process is conducted
using two well-known techniques, Animation and Tracing [50],
785 which display the simulated entities’ behavior over time, allow- In the first time step, all services start to be moved out of
ing us to check the simulator’s logic correctness. edge server ES1. This behavior is defined through a simple re-
source allocation strategy to demonstrate different events within
5.1. Scenario Description 810 the service provisioning process. Specifically, S1 is moved to
Figure 8 depicts the edge infrastructure used in EdgeSimPy’s edge server ES3 through links L3 and L6, while S2 and S3 are
verification. We consider three NVIDIA Jetson TX2 boards moved to edge server ES2 through link L4. The reallocation
rna

790 representing the edge servers. Each edge server has 4 CPU of the three services takes six time steps. For simplicity, each
cores, 8 GB of RAM, and 64 GB of storage (CPU and mem- time step corresponds to 1 second, and the Max-Min Fairness
ory values are obtained from Süzen et al. [55]). Edge servers815 algorithm [53] is used as the network flow scheduling policy.
can download at most three container layers at once, following The remainder of this section presents a discussion aiming
Docker’s default configuration to avoid network congestion6 . to track down events demonstrating EdgeSimPy reproduces the
795 The edge network comprises nine links with heterogeneous ca- expected behavior. For such, we draw a parallel between the
pacities (for simplicity’s sake, L1–L4 have a bandwidth capac- result of the EdgeSimPy simulation, presented in Figures 10-11
ity of 9 Mbps while L5–L9 have 2.5 Mbps). 820 and Table 3, and the conceptual model adopted within the sim-
In this scenario, two users move according to the Pathway ulator (i.e., provisioning services according to the lifecycle of
model [42], each accessing one of the two existing applications. containers and sharing bandwidth based on the Max-Min Fair-
Jou

800 As detailed in Figure 9, Application 1 has a stateless service ness algorithm [53]).
(S1), while Application 2 comprises a stateless service (S2) and
a stateful service (S3). Services S1 and S2 use the same base 5.2. Verification Discussion
image with a single layer (L1), while S2 has a base image with825 Figure 10 shows the progress of network flows spawned by
four layers (L1–L4). All services are initially on edge server service migrations. Service S1 is migrated across links L3 and
805 ES1, which also hosts a container registry demanding 1 CPU L6, which have different bandwidths (9 Mbps and 2.5 Mbps, re-
core, 1 GB of memory, and 10 MB of disk. spectively). In such a scenario, EdgeSimPy equalizes the band-
6 [Link] width available for service provisioning to the bandwidth of the
#concurrent-downloads

10
 Journal Pre-proof

830 link with the lowest capacity. Accordingly, layer L1, which cor-870 model called Edge-as-a-Service (EaaS), which harnesses the
responds to the base image of service S1, is transferred at 2.5 physical proximity of the edge in an infrastructure capable of
Mbps during time steps 1–5. delivering the speed necessary to meet the demand of latency-
sensitive applications [56] [57]. Just as cloud services have led
Service 1 Layer 1 BW:2.5. Left:9.5 BW:2.5. Left:7 BW:2.5. Left:4.5 BW:2.5. Left:2 BW:2. Left:0 a technological shift over the past decade, EaaS offerings dis-

of
Layer 1 BW:3. Left:9 BW:3. Left:6 BW:3. Left:3 BW:3. Left:0
875 play great potential to become the “next big thing” in the IT
Layer 2 BW:3. Left:3 BW:3. Left:0
industry.
Service 2
Layer 3 BW:3. Left:6 BW:3. Left:3 BW:3. Left:0 This case study explored the possibility of federated edges,
Layer 4 BW:3. Left:6 BW:6. Left:0 in which coalitions of EaaS providers are created to enhance
Service 3 State BW:9. Left:0
profit and meet ever-increasing application performance expec-
880 tations [58]. In such a scenario, microservice-based applica-

pro
T1 T2 T3 T4 T5
tions must be migrated according to user mobility while re-
Figure 10: Network flows used to transfer container layers and service states
specting privacy requirements of specific microservices.
among servers. For conciseness, “BW” denotes the bandwidth available for the Whereas infrastructure providers are willing to share re-
network flows, and “Left” denotes the remaining data that will be transferred in sources to reduce application latency bottlenecks, users exhibit
subsequent time steps. 885 distinct levels of trust with different providers, which restricts
allocation options for services with special privacy requirements.
Services S2 and S3 are moved from edge server ES1 to edge The performance evaluation compared a novel algorithm against
server ES2 through the same path, sharing the bandwidth of three migration strategies from the literature regarding the num-
835 link L4. As the base images of S2 and S3 are built on top of ber of migrations and SLA violations, which consider prede-
layer L1, ES2 does not pull L1 twice. Despite the layer-sharing890 fined latency thresholds of applications and the privacy require-
optimization, edge server ES2 can only pull three layers at once, re- ments of microservices.
which forces layer L4 to wait during time steps T 1 and T 2 until EdgeSimPy’s base architecture does not include a functional
the number of active downloads of ES2 decreases as the transfer abstraction for infrastructure providers. Therefore, to support
840 of layer L2 ends. this case study, a “provider” attribute is added to edge servers to
At time step T 4 , the Max-Min Fairness algorithm distributes895 identify their infrastructure providers, and a “trusted providers”
3 Mbps and 6 Mbps to the active flows of layers L1 and L4, re- attribute is added to users with the providers they trust. Dur-
spectively, instead of giving them an equal bandwidth share. ing simulation, both parameters are checked to arrange edge
That happens because Max-Min Fairness divides the available servers based on the level of trust between users and infrastruc-
845 bandwidth proportionally to the size of the network flows. As ture providers. In addition, a “privacy requirement” attribute
lP
layer L1’s flow only needs 3 Mbps to finish, the 1.5 Mbps left-900 is added to services to specify their privacy requirement level.
over is offered to L4’s flow, which can be transferred at 6 Mbps. The simulated scenario comprised an infrastructure with 60
The differences between stateless and stateful service pro- edge servers managed by two infrastructure providers and 240
visioning are noticeable in time step T 5 . Once all layers of S3’s services with heterogeneous capacity, privacy, and latency re-
850 base image are present in edge server ES2, S3 is stopped on its quirements. The obtained results demonstrated that certain mi-
source host ES1, and its state is transferred to ES3. EdgeSimPy905 gration decisions, i.e., prioritizing services with special privacy
keeps S3 unavailable during time step T 5 while its state is trans- requirements from applications with tight latency demands, can
ferred, as depicted in Figure 11. Further information about ser- reduce privacy leaks by up to 7.95% at federated edges without
rna

vice provisioning is presented in Table 3, which details resource sacrificing application latency [5].
855 demand and provisioned layers on the edge servers throughout
the simulation. 6.2. Case Study 2: Edge Server Maintenance
910 Edge infrastructures typically comprise computing resources
6. EdgeSimPy Case Studies deployed near end users, as it helps deliver low latency to ap-
plications [2]. At the same time, proximity also forces infras-
This section presents two case studies from research papers tructure dispersion, as deploying large-scale data centers close
that employed EdgeSimPy as a validation platform for resource to urban centers may not be feasible [1]. While the distribution
860 management in edge infrastructures. An in-depth discussion of915 of computing resources allows them to be close to end users, it
each case study can be found in the original publications [5] also introduces technical challenges related to IT operations.
Jou

(§6.1) and [6] (§6.2). Accordingly, this section focuses on de- In case edge resources cannot be deployed on a large scale
scribing the motivation, scenarios, and adjustments made to Ed- within centralized facilities, communication between nodes typ-
geSimPy for the simulations rather than discussing the results. ically relies on public networks, which increases the chance of
865 These case studies illustrate how EdgeSimPy can be easily ex-920 instability caused by outages and lower bandwidth [59]. In ad-
tended to comprehend various resource management scenarios. dition, outdoor-deployed edge devices are more prone to phys-
ical issues and security threats. In such a scenario, infrastruc-
6.1. Case Study 1: Application Migration
ture operators must invest in maintenance strategies to mitigate
As Edge Computing research matures, large-scale infras- eventual security threats and infrastructure failures.
tructure providers are starting to move toward a new service
11
 Journal Pre-proof

S3 S3 S3
User 1 S1 S2 User 2
S1 S2 S1 S2
L1 ES1 L1 ES1 L1 ES1

of
L4

L4

L4
L2

L3

L2

L3

L2

L3
User 2
L5 ES2 User 1 L5 ES2 L5 ES2

L8 L8 L8
L7

L7

L7
L6

L6

L6
ES3 L9 ES3 L9 ES3 L9
User 2 User 1

pro
(a) Time step T 1 (b) Time step T 2 (c) Time step T 3

S3
User 2 User 1 S1 S3
S1 S2
L1 ES1 L1 ES1 L1 ES1
L4

L4

L4
L2

L3

L2

L3

L2

L3
S2 S2 S3
L5 ES2 L5 ES2 User 1 L5 ES2
User 1 User 2

L8 re- L8 L8
L7

L7

L7
L6

L6

L6
S1
ES3 L9 ES3 L9 ES3 L9
User 2

(d) Time step T 4 (e) Time step T 5 (f) Time step T 6

Figure 11: Dynamics of each time step of EdgeSimPy’s verification. Dashed arrows represent the network paths used for communication between users and services.
lP
Table 3: Edge servers state throughout EdgeSimPy’s verification simulation.
Demand
Time Step Instance Services Waiting Queue Download Queue Layers
CPU RAM Disk
1 4 4096 46 Service 1, Service 2, Service 3 Registry, L1, L2, L3, L4
T1 2 2 2048 36 L4 L1, L2, L3
3 1 1024 12 L1
1 4 4096 46 Service 1, Service 2, Service 3 Registry, L1, L2, L3, L4
T2 2 2 2048 36 L4 L1, L3 L2
3 1 1024 12 L1
1 4 4096 46 Service 1, Service 2, Service 3 Registry, L1, L2, L3, L4
rna

T3 2 2 2048 36 L1, L4 L2, L3

3 1 1024 12 L1
1 4 4096 46 Service 1, Service 2, Service 3 Registry, L1, L2, L3, L4
T4 2 2 2048 36 L2, L3, L1, L4
3 1 1024 12 L1
1 2 2048 46 Service 1, Service 3 Registry, L1, L2, L3, L4
T5 2 2 2048 36 Service 2 L2, L3, L1, L4
3 1 1024 12 L1
1 1 1024 46 Registry, L1, L2, L3, L4
T6 2 2 2048 36 Service 2, Service 3 L2, L3, L1, L4
3 1 1024 12 Service 1 L1
Jou

925 This case study focused on an edge server maintenance sce- nance strategies: reduce maintenance time and application la-
nario. This case assumed that patches require edge servers to be935 tency bottlenecks. From a security perspective, maintenance
rebooted for the changes to take effect. Consequently, before an needs to be completed as soon as possible, as patches can corre-
edge server can be updated, the applications it hosts must be re- spond to critical security updates, so the infrastructure remains
located to another server to avoid downtime (such a process is vulnerable until all edge servers are updated. At the same time,
930 called server draining). As such, maintenance strategies need applications hosted on the infrastructure are latency sensitive,
to schedule the server update order and define new hosts for the940 so migration decisions performed to drain servers must keep
applications hosted by the servers that will be updated. applications close enough to their users to ensure that latency
This case study imposed two main objectives to mainte- remains low.

12
 Journal Pre-proof

The default implementation of edge servers in EdgeSimPy Despite the flexibility of creating classes for new entities to
comprises no attribute related to maintenance. Therefore, to suit specific needs, entity instances are bound by the attributes
945 support this case study, an “updated” attribute is added to edge1000 previously defined in their class definitions, meaning they can-
servers to denote their updated status (this attribute is False by not incorporate unique attributes not initially present in their
default). A “Patch” entity is also created with the time required class constructors. This constraint is also observed in Edge-

of
to complete the update. Edge servers and patches are bounded CloudSim [22], MobFogSim [24], IoTSim-Edge [27], IoTSim-
through a relationship attribute. Finally, an “update()” method Osmosis [28], and iFogSim2 [29], which extend the function-
950 is implemented inside the edge server entity, representing the1005 ality of CloudSim [20] and require extension of built-in classes
patching process. Once an edge server starts to be updated, it or creation of new ones to represent the necessary features.
is tagged as unavailable for its patch duration, and when the YAFS [26] displays the closest resemblance to EdgeSimPy
update period ends, the edge server’s update status is changed regarding extensibility, as it is also written in Python, which

pro
accordingly. In this case study, the simulation continues until is highly flexible regarding extending classes. Nevertheless, as
955 all edge servers are updated. 1010 YAFS lacks a well-defined format for importing and export-
This case study evaluation compared two novel algorithms ing entities, custom components must be built manually rather
against two strategies from the literature regarding maintenance than imported automatically through dataset files, as compati-
time, the number of migrations, latency requirement violations, ble serialization formats (e.g., JSON and XML) cannot recog-
and vulnerability surface (which quantifies how long edge servers nize advanced data structures used, for example, to define re-
960 remain outdated during maintenance). The simulated scenario1015 lationships between objects. Additionally, manual instructions
comprises an infrastructure with 40 edge servers hosting 90 must be passed to the simulator regarding how entities are up-
applications with heterogeneous capacity and latency require- dated over the simulation time and how entity metrics should
ments. The results showed that user-location-aware migration be gathered. Once importing custom entities becomes a manual

965

6.3. Discussion
re-
decisions can reduce latency requirement violations by 30.67%
on average without extending maintenance time [6]. 1020
process, YAFS suffers from the same issue as other simulators
based on Java and C++. Whereas this behavior may not be an
issue for small tests, it hinders the efficient execution of large-
scale batch executions.
This section motivates the adoption of EdgeSimPy to ad-
dress the needs of the described case studies, highlighting how 7. Lessons Learned
using it is more practical than other existing simulators.
970 The first case study focused on migrating composite appli- Several reflections have emerged during EdgeSimPy’s de-
velopment, providing valuable insights for researchers and prac-
lP
cations within an edge environment composed of multiple in-1025
frastructure providers. While many of the existing edge simu- titioners interested in developing simulation tools.
lators offer support for application composition (e.g., IoTSim- Selecting Python as the base language for EdgeSimPy has
Edge, YAFS, and iFogSim2), they lack functional abstractions enabled users to benefit from a broad ecosystem of libraries, es-
975 for infrastructure providers. In addition, they do not provide pecially those related to emerging fields such as Artificial Intel-
features for managing the lifecycle of containers, which is the1030 ligence. One key takeaway from our experience developing Ed-
preferred virtualization technology for composite applications. geSimPy that could benefit future simulator developers involves
The second case study focused on edge server maintenance. carefully considering the base programming language, espe-
cially regarding the popularity of the language within the com-
rna

Unlike EdgeSimPy, existing edge simulators do not provide the

980 features for modeling maintenance operations, which involves munity and the number of available libraries that users could
incorporating version attributes to differentiate updated from1035 leverage when using the developed simulator. This approach
outdated devices and functions to represent the update process. aims to broaden the utility of the simulator, enabling users to
Although EdgeSimPy’s built-in entity attributes had to be harness community-supported algorithms and thus potentially
extended for conducting the simulations, EdgeSimPy allowed reducing the need for manual implementation.
985 such changes to be included out of the box without requiring Throughout our internal testing, we also found that EdgeS-
modifications to its base features. This was possible as EdgeS-1040 imPy’s native support for Jupyter Notebooks7 considerably eases
imPy automatically identifies custom entity attributes, provided code sharing, as online platforms like Google Colaboratory8
its input format is followed. This feature mitigates the risk of and MyBinder9 facilitate real-time collaboration among EdgeS-
human-induced errors arising from changes to the simulator’s imPy users, eliminating the necessity for local installation of
Jou

990 base features. assets. With this in mind, we would advise researchers inter-
The ease with which EdgeSimPy allows the incorporation1045 ested in simulator development to select a base programming
of custom attributes and entities is an advantage over exist- language and design the simulator with user interactivity as a
ing simulators, which generally require creation or extension of primary requirement. Leveraging interactive computing plat-
classes to represent such attributes and entities. For example, forms can enhance the user experience through an environment
995 ECSNeT++ and FogNetSim++ are built on top of OMNeT++,
a discrete-event simulator written in C++. In OMNeT++, cus- 7 [Link]

tom entities can be modeled using C++ classes called modules. 8 [Link]
9 [Link]

13
 Journal Pre-proof

that supports real-time collaboration and learning, where users a flexible input format that allows users to define custom pa-
1050 can run experiments with different configurations, observe the rameters for simulated entities out-of-the-box, extending the
results in real-time, and adjust their approaches accordingly. 1105 simulator’s built-in capabilities without modifying its core fea-
Another lesson we can share with researchers interested in tures. EdgeSimPy’s source code can be accessed via GitHub10 .
simulator development is the importance of designing a frame- Furthermore, a tutorial library11 with several practical exam-

of
work with a decoupled architecture. In EdgeSimPy, we have ples has been developed to assist researchers and practitioners
1055 observed that its strength primarily comes from this highly de- in effectively integrating EdgeSimPy into their work. In fu-
coupled structure, which includes class-based modularization1110 ture work, we intend to extend EdgeSimPy networking capabil-
and standardized input format, which facilitate the implementa- ities to support the simulation of advanced routing policies that
tion of new entities and attributes, thereby supporting users with leverage the programmability of network switches at the edge.
specific simulation requirements. This degree of decoupling

pro
1060 further translates into a high level of configurability, where users
Acknowledgements
can adjust simulation parameters out-of-the-box, including the
simulation tick rate and system models (e.g., user mobility, power This work was supported by the PDTI Program, funded by
consumption of compute and networking devices, network band- 1115 Dell Computadores do Brasil Ltda (Law 8.248 / 91). The au-
width sharing control, etc.). thors acknowledge the High-Performance Computing Labora-
1065 Finally, it is worth noting that appropriate calibration of the tory of the Pontifical Catholic University of Rio Grande do Sul
simulator parameters is critical to obtaining an accurate repre- for providing resources for this project.
sentation of real-world experimental setups. The calibratable
parameters include the simulation tick rate and the configura-
References
tion of the system model to match the scenario being modeled.
1070 re-
For instance, this may involve defining a geographically accu-1120
rate map, setting up a realistic power consumption model, or
mimicking actual user mobility patterns. Additionally, the se-
lection of metrics to be collected during the simulation is an-
[1] M. Satyanarayanan, G. Klas, M. Silva, S. Mangiante, The seminal role
of edge-native applications, in: 2019 IEEE International Conference on
Edge Computing, IEEE, 2019, pp. 33–40.
[2] M. Satyanarayanan, P. Bahl, R. Caceres, N. Davies, The case for vm-
based cloudlets in mobile computing, IEEE Pervasive Computing 8
other crucial factor that must be carefully calibrated. 1125 (2009) 14–23.
[3] H. Zhao, S. Deng, Z. Liu, J. Yin, S. Dustdar, Distributed redundancy
scheduling for microservice-based applications at the edge, IEEE Trans-
1075 8. Conclusion and Future Work actions on Services Computing (2020).
[4] J. Wang, Z. Feng, S. George, R. Iyengar, P. Pillai, M. Satyanarayanan,
Towards scalable edge-native applications, in: ACM/IEEE Symposium
lP
Edge Computing is attracting attention as a paradigm that1130
on Edge Computing, 2019, pp. 152–165.
delivers low latency for applications by pulling processing from [5] P. Souza, A. Crestani, T. Ferreto, F. Rossi, Latency-aware privacy-
traditional cloud data centers to the network’s edge [2] [60]. preserving service migration in federated edges, in: International Con-
As the expectations on the potential benefits of Edge Com- ference on Cloud Computing and Services Science, 2022, pp. 288–295.
1080 puting grow, ensuring that Edge Computing transitions from1135 [6] P. Souza, T. Ferreto, F. Rossi, R. Calheiros, Location-aware maintenance
strategies for edge computing infrastructures, IEEE Communications Let-
promise to reality require developing efficient resource man- ters 26 (2022) 848–852.
agement techniques. [7] J. Kazil, D. Masad, A. Crooks, Utilizing python for agent-based model-
Aside from the development challenges, experimental re- ing: The mesa framework, in: International Conference on Social Com-
puting, Behavioral-Cultural Modeling and Prediction and Behavior Rep-
rna

search at the edge can be expensive and time-consuming as1140

resentation in Modeling and Simulation, Springer, 2020, pp. 308–317.
1085 it involves building and instrumenting an infrastructure with [8] A. Hagberg, P. Swart, D. S Chult, Exploring network structure, dynamics,
many interconnected devices. Furthermore, the distributed na- and function using NetworkX, Technical Report, Los Alamos National
ture of the edge makes room for several external factors (e.g., Lab.(LANL), Los Alamos, NM (United States), 2008.
[9] R. Buyya, S. N. Srirama, G. Casale, R. Calheiros, Y. Simmhan, B. Vargh-
network instability) to affect the reproducibility and reliability1145 ese, E. Gelenbe, B. Javadi, L. M. Vaquero, M. A. Netto, et al., A manifesto
of results, undermining the extraction of insights needed to ma- for future generation cloud computing: Research directions for the next
1090 ture prototypes. Consequently, simulation has been considered decade, ACM Computing Surveys 51 (2018) 1–38.
the primary approach to accelerate the validation of early-stage [10] R. Roman, J. Lopez, M. Mambo, Mobile edge computing, fog et al.: A
1150 survey and analysis of security threats and challenges, Future Generation
prototypes at a reduced cost [61]. Computer Systems 78 (2018) 680–698.
Existing Edge Computing simulators incorporate various [11] S. Shahzadi, M. Iqbal, T. Dagiuklas, Z. U. Qayyum, Multi-access edge
features for modeling user mobility, application composition, computing: open issues, challenges and future perspectives, Journal of
Jou

1095 and energy consumption of the edge infrastructure. However, Cloud Computing 6 (2017) 1–13.
1155 [12] Y. Mao, C. You, J. Zhang, K. Huang, K. B. Letaief, A survey on mobile
they fail to deliver fine-grained control over container provi- edge computing: The communication perspective, IEEE communications
sioning, which has been acknowledged as the primary choice surveys & tutorials 19 (2017) 2322–2358.
for deploying applications at the edge. [13] P. Sharma, L. Chaufournier, P. Shenoy, Y. C. Tay, Containers and virtual
To fill this gap, we presented EdgeSimPy, a novel simulator machines at scale: A comparative study, in: International Middleware
1160 Conference, ACM, New York, NY, USA, 2016, pp. 1–13.
1100 that models the lifecycle of containerized applications through
several functional abstractions that replicate the behavior of con-
tainer runtimes like Docker. In addition, EdgeSimPy features 10 [Link]
11 [Link]

14
 Journal Pre-proof

[14] K. Wang, J. Rao, C.-Z. Xu, Rethink the virtual machine template, ACM [36] E. Bonabeau, Agent-based modeling: Methods and techniques for simu-
SIGPLAN Notices 46 (2011) 39–50. lating human systems, Proceedings of the National Academy of Sciences
[15] C.-P. Wu, M. A. Suresh, D. Da Silva, Container lifecycle management 99 (2002) 7280–7287.
for edge nodes: poster, in: ACM/IEEE Symposium on Edge Computing,1235 [37] A. M. Law, W. D. Kelton, W. D. Kelton, Simulation modeling and analy-
1165 2017, pp. 1–2. sis, volume 5, Mcgraw-hill New York, 2015.
[16] J. Darrous, T. Lambert, S. Ibrahim, On the importance of container image [38] D. P. Bertsekas, R. G. Gallager, P. Humblet, Data networks, volume 2,

of
placement for service provisioning in the edge, in: International Confer- Prentice-Hall International New Jersey, 1992.
ence on Computer Communication and Networks, IEEE, 2019, pp. 1–9. [39] M. d. S. Conterato, T. C. Ferreto, F. Rossi, W. d. S. Marques, P. S. S.
[17] C. G. Kominos, N. Seyvet, K. Vandikas, Bare-metal, virtual machines1240 de Souza, Reducing energy consumption in sdn-based data center net-
1170 and containers in openstack, in: 2017 20th conference on innovations in works through flow consolidation strategies, in: ACM/SIGAPP Sympo-
clouds, internet and networks (icin), IEEE, 2017, pp. 36–43. sium on Applied Computing, 2019, pp. 1384–1391.
[18] K. Ha, Y. Abe, T. Eiszler, Z. Chen, W. Hu, B. Amos, R. Upadhyaya, [40] P. Reviriego, J. A. Hernández, D. Larrabeiti, J. A. Maestro, Performance
P. Pillai, M. Satyanarayanan, You can teach elephants to dance: Agile evaluation of energy efficient ethernet, IEEE Communications Letters 13

pro
vm handoff for edge computing, in: ACM/IEEE Symposium on Edge1245 (2009) 697–699.
1175 Computing, 2017, pp. 1–14. [41] A. Beloglazov, R. Buyya, Optimal online deterministic algorithms and
[19] B. I. Ismail, E. M. Goortani, M. B. Ab Karim, W. M. Tat, S. Setapa, J. Y. adaptive heuristics for energy and performance efficient dynamic con-
Luke, O. H. Hoe, Evaluation of docker as edge computing platform, in: solidation of virtual machines in cloud data centers, Concurrency and
IEEE Conference on Open Systems, IEEE, 2015, pp. 130–135. Computation: Practice and Experience 24 (2012) 1397–1420.
[20] R. N. Calheiros, R. Ranjan, A. Beloglazov, C. A. De Rose, R. Buyya,1250 [42] F. Bai, A. Helmy, A survey of mobility models, Wireless Adhoc Net-
1180 Cloudsim: a toolkit for modeling and simulation of cloud computing en- works. University of Southern California, USA 206 (2004) 147.
vironments and evaluation of resource provisioning algorithms, Software: [43] P. Silva, D. Fireman, T. E. Pereira, Prebaking functions to warm the
Practice and experience 41 (2011) 23–50. serverless cold start, in: International Middleware Conference, 2020, pp.
[21] D. Kliazovich, P. Bouvry, S. U. Khan, Greencloud: a packet-level sim- 1–13.
ulator of energy-aware cloud computing data centers, The Journal of1255 [44] F. Faticanti, F. De Pellegrini, D. Siracusa, D. Santoro, S. Cretti,
1185 Supercomputing 62 (2012) 1263–1283. Throughput-aware partitioning and placement of applications in fog com-
[22] C. Sonmez, A. Ozgovde, C. Ersoy, Edgecloudsim: An environment for puting, IEEE Transactions on Network and Service Management 17
performance evaluation of edge computing systems, in: International (2020) 2436–2450.

1190

environment, IEEE Access 6 (2018) 63570–63583.

re-
Conference on Fog and Mobile Edge Computing, 2017, pp. 39–44.
[23] T. Qayyum, A. W. Malik, M. A. Khan Khattak, O. Khalid, S. U. Khan,1260
Fognetsim++: A toolkit for modeling and simulation of distributed fog

[24] C. Puliafito, D. M. Gonçalves, M. M. Lopes, L. L. Martins, E. Madeira,

[45] M. D. de Assuncao, A. da Silva Veith, R. Buyya, Distributed data stream
processing and edge computing: A survey on resource elasticity and
future directions, Journal of Network and Computer Applications 103
(2018) 1–17.
[46] L. Zhao, L. Lu, Z. Jin, C. Yu, Online virtual machine placement for in-
E. Mingozzi, O. Rana, L. F. Bittencourt, Mobfogsim: Simulation of mo- creasing cloud provider’s revenue, IEEE Transactions on Services Com-
bility and migration for fog computing, Simulation Modelling Practice1265 puting 10 (2015) 273–285.
1195 and Theory 101 (2020) 102062. [47] H. Tian, J. Wu, H. Shen, Efficient algorithms for vm placement in cloud
[25] G. Amarasinghe, M. D. de Assunção, A. Harwood, S. Karunasekera, Ec- data centers, in: International Conference on Parallel and Distributed
snet++ : A simulator for distributed stream processing on edge and cloud Computing, Applications and Technologies, IEEE, 2017, pp. 75–80.
lP
environments, Future Generation Computer Systems 111 (2020) 401– [48] J. Shi, O. Gurewitz, V. Mancuso, J. Camp, E. W. Knightly, Measurement
418. 1270 and modeling of the origins of starvation in congestion controlled mesh
1200 [26] I. Lera, C. Guerrero, C. Juiz, Yafs: A simulator for iot scenarios in fog networks, in: International Conference on Computer Communications,
computing, IEEE Access 7 (2019) 91745–91758. IEEE, 2008, pp. 1633–1641.
[27] D. N. Jha, K. Alwasel, A. Alshoshan, X. Huang, R. K. Naha, S. K. Bat- [49] C. Toprak, C. Turker, A. T. Erman, Detection of dhcp starvation attacks
tula, S. Garg, D. Puthal, P. James, A. Zomaya, S. Dustdar, R. Ranjan, in software defined networks: A case study, in: International Conference
Iotsim-edge: A simulation framework for modeling the behavior of inter-1275 on Computer Science and Engineering, IEEE, 2018, pp. 636–641.
1205 net of things and edge computing environments, Software: Practice and [50] X. Xiang, R. Kennedy, G. Madey, S. Cabaniss, Verification and validation
Experience 50 (2020) 844–867. of agent-based scientific simulation models, in: Agent-Directed Simula-
[28] K. Alwasel, D. N. Jha, F. Habeeb, U. Demirbaga, O. Rana, T. Baker, tion Conference, volume 47, The Society for Modeling and Simulation
rna

S. Dustdar, M. Villari, P. James, E. Solaiman, R. Ranjan, Iotsim-osmosis: International San Diego, CA, USA, 2005, p. 55.
A framework for modeling and simulating iot applications over an edge-1280 [51] R. G. Sargent, Verification and validation of simulation models, in: Con-
1210 cloud continuum, Journal of Systems Architecture 116 (2021) 101956. ference on Winter Simulation, IEEE, 2010, pp. 166–183.
[29] R. Mahmud, S. Pallewatta, M. Goudarzi, R. Buyya, ifogsim2: An ex- [52] O. Balci, Principles and techniques of simulation validation, verification,
tended ifogsim simulator for mobility, clustering, and microservice man- and testing, in: Conference on Winter Simulation, 1995, pp. 147–154.
agement in edge and fog computing environments, Journal of Systems [53] F. Gebali, Analysis of computer and communication networks, Springer
and Software 190 (2022) 111351. 1285 Science & Business Media, 2008.
1215 [30] N. Nurseitov, M. Paulson, R. Reynolds, C. Izurieta, Comparison of json [54] L. A. D. Knob, C. H. Kayser, P. S. S. de Souza, T. Ferreto, Enforcing de-
and xml data interchange formats: a case study., in: International Con- ployment latency sla in edge infrastructures through multi-objective ge-
ference on Computer Applications in Industry and Engineering, 2009, pp. netic scheduler, in: IEEE/ACM International Conference on Utility and
157–162. Cloud Computing, 2021, pp. 1–9.
[31] F. Pezoa, J. L. Reutter, F. Suarez, M. Ugarte, D. Vrgoč, Foundations of1290 [55] A. A. Süzen, B. Duman, B. Şen, Benchmark analysis of jetson tx2, jet-
1220 json schema, in: International Conference on World Wide Web, 2016, pp. son nano and raspberry pi using deep-cnn, in: International Congress on
Jou

263–273. Human-Computer Interaction, Optimization and Robotic Applications,

[32] A. Aral, V. De Maio, I. Brandic, Ares: Reliable and sustainable edge pro- IEEE, 2020, pp. 1–5.
visioning for wireless sensor networks, IEEE Transactions on Sustainable [56] N. Chen, Y. Yang, T. Zhang, M.-T. Zhou, X. Luo, J. K. Zao, Fog as a
Computing 7 (2021) 761–773. 1295 service technology, IEEE Communications Magazine 56 (2018) 95–101.
1225 [33] J. Vanura, P. Kriz, Perfomance evaluation of java, javascript and php [57] A. Jindal, G. S. Aujla, N. Kumar, Survivor: A blockchain based edge-as-
serialization libraries for xml, json and binary formats, in: International a-service framework for secure energy trading in sdn-enabled vehicle-to-
Conference on Services Computing, Springer, 2018, pp. 166–175. grid environment, Computer Networks 153 (2019) 36–48.
[34] H. Sayama, Introduction to the modeling and analysis of complex sys- [58] C. Anglano, M. Canonico, P. Castagno, M. Guazzone, M. Sereno, A
tems, Open SUNY Textbooks, 2015. 1300 game-theoretic approach to coalition formation in fog provider federa-
1230 [35] C. M. Macal, M. J. North, Tutorial on agent-based modeling and simula- tions, in: International Conference on Fog and Mobile Edge Computing,
tion, in: Winter Simulation Conference, IEEE, 2005, pp. 14–pp. IEEE, 2018, pp. 123–130.

15
 Journal Pre-proof

[59] A. Aral, I. Brandić, Learning spatiotemporal failure dependencies for

resilient edge computing services, IEEE Transactions on Parallel and
1305 Distributed Systems 32 (2020) 1578–1590.
[60] M. Satyanarayanan, The emergence of edge computing, Computer 50
(2017) 30–39.
[61] M. Salama, Y. Elkhatib, G. Blair, Iotnetsim: A modelling and simulation

of
platform for end-to-end iot services and networking, in: International
1310 Conference on Utility and Cloud Computing, 2019, pp. 251–261.

pro
re-
lP
rna
Jou

16
 Journal Pre-proof

Highlights

of
● A high-level interface for developing management policies and simulator extensions.
● A novel conceptual model that represents the application lifecycle based on Docker.
● System models for simulating several features of Edge Computing environments.
● An in-depth verification that checks the correctness of EdgeSimPy's implementation.
● Case studies based on peer-reviewed papers that have employed EdgeSimPy.

pro
re-
lP
rna
Jou
 Journal Pre-proof

Author Biographies (Paulo, Tiago, and Rodrigo)

Author Biographies

of
Paulo S. Souza is a Ph.D. candidate in Computer Science at the Pontifical Catholic University
of Rio Grande do Sul (PUCRS), Brazil. He received his Master's degree in Computer Science
from the same institution in 2020. His research interest lies primarily in the fields of
computational resource management, Cloud Computing, Edge Computing, and algorithms.

pro
Tiago Ferreto is an Associate Professor in Computer Science at the Pontifical Catholic
University of Rio Grande do Sul (PUCRS), Brazil, where he obtained his Ph.D. degree in
Computer Science in 2010. His primary academic research interests are resource management,
Cloud Computing, Edge Computing, and virtualization.

Rodrigo N. Calheiros is an Associate Professor at the School of Computer, Data and

Mathematical Sciences, Western Sydney University, Australia. He works in the field of Cloud
re-
Computing and related areas since 2008, and since then he carried out R&D supporting
research in the area. His research interests also include Big Data, Internet of Things, Fog
Computing, and their application.
lP
rna
Jou
 Journal Pre-proof

Author Photo (Rodrigo) Click here to access/download;Author Photo;[Link]

of
pro
re-
lP
rna
Jou
 Journal Pre-proof

Author Photo (Tiago) Click here to access/download;Author Photo;[Link]

of
pro
re-
lP
rna
Jou
 Journal Pre-proof

Author Photo (Paulo) Click here to access/download;Author Photo;[Link]

of
pro
re-
lP
rna
Jou
 Journal Pre-proof

CRediT Author Statement

of
Paulo S. Severo: Conceptualization, Methodology, Software, Validation, Writing - Original Draft.
Tiago Ferreto: Conceptualization, Supervision, Methodology, Writing - Original Draft, Writing -
Review & Editing. Rodrigo N. Calheiros: Supervision, Methodology, Writing - Original Draft,
Writing - Review & Editing.

pro
re-
lP
rna
Jou
 Journal Pre-proof

Declaratio if ioterettt

of
☒ The authors declare that they have no known competng fnancial interests or personal relatonships
that could have appeared to infuence the work reported in this paper.

☐ The authors declare the following fnancial interests/personal relatonships which may be considered

pro
as potental competng interests:

re-
lP
rna
Jou