INSTITUTE OF INFORMATION AND COMMUNICATION TECHNOLOGIES (INSTIC)

COORDINATION OF THE TELECOMMUNICATIONS ENGINEERING COURSE

COMPUTER RESEARCH WORK IV

TOPIC: DISTRIBUTED SYSTEMS AND

PARALLELS

Luanda, 2023

Parque do Saber do MTTI, ISUTIC, Bairro dos CTT´s KM 7, Rangel-Luanda, Contactos: [Link]
 INSTITUTE OF INFORMATION AND COMMUNICATION TECHNOLOGIES (INSTIC)
COORDINATION OF THE TELECOMMUNICATION ENGINEERING COURSE

COMPUTER RESEARCH WORK IV

TOPIC: DISTRIBUTED SYSTEMS AND

PARALLELS

Year: 5o
Authors: Nº Processo:
Dário E. Prazeres 2130
Eduane V. Domingos 2142
Érica D. Bonito 1524
José N. Matazi 2184
Josué K. Mbunga 2186

Teacher: Esteves Tchyneke

Luanda, 2022

Science Park of MTTI, ISUTIC, CTT Neighborhood KM 7, Rangel-Luanda, Contacts: [Link]

General Index

General Objective ............................................................................................................................. IV

Specific [Link]
1. Introductiono.................................................................................................................................. 1
2. Systems Distributed ................................................................................................................ 2
2.1. Requirements two distributed systems................................................................................... 3
3. Systems Parallels ..................................................................................................................... 3
3.1. Requirements two parallel systemss ....................................................................................... 5
4. Challenges two distributed and parallel systemss .......................................................................... 5
5. As distributed and parallel architectures
6. A communication and synchronization in distributed and parallel systemss ...................................... 9
7. Conclusion 12
8. Bibliography 13

Page III
General Objective

Study the architecture of distributed and parallel systems and their implications in communication
and synchronization between the components, with the aim of understanding how these systems
they work and how they can be designed to meet different performance requirements
and availability.

Objetivos específicos

Analyze the main characteristics of distributed and parallel systems like yours.
architecture, communication, and synchronization;
Investigate the different architectures of distributed and parallel systems, including
client-server architectures, peer-to-peer and others;
Analyze the implications of choosing a specific architecture in communication and
synchronization of distributed and parallel systems, taking into account aspects such as
performance, scalability, and availability;
Study the communication and synchronization techniques used in distributed systems and
parallels, such as message exchange, shared memory, and locking.

Page IV
 1. Introduction

With the growth of data processing demands, it has become necessary to

development of systems capable of processing large volumes of information in a way
efficient and fast. In this sense, distributed and parallel systems emerged as a solution,
allowing various tasks to be executed simultaneously on different computers
interconnected in a network.

Distributed and parallel systems are two of the main ways to achieve high performance.
computational performance in large-scale data processing applications. While the
Distributed systems are characterized by the presence of multiple interconnected components.
In different geographical locations, parallel systems are characterized by the use of multiple
processors working together to execute tasks in parallel. In this work,
we will discuss in detail the main concepts related to each of these systems, as well as
its main applications in different areas of computing.

Page 1
 2. Distributed Systems

Distributed systems are characterized by the presence of multiple components.

hardware and software interconnected in different geographical locations, working together
to perform a specific task. These components can be personal computers,
servers, mobile devices, sensors, among others. The main objective of the systems
distributed is to enable the sharing of resources and the execution of tasks on a large scale, without
the need for a single centralized system to manage all operations.

Figure 1 - Distributed Systems

Among the main characteristics of distributed systems, the following stand out:

Heterogeneity: The components of a distributed system can be heterogeneous.

in terms of hardware and software, that is, they can have different processor architectures,
operating systems, programming languages, among others.

Autonomy: Each component of a distributed system is responsible for its own

operations, without the need for centralized control. This allows the system to be
more fault-tolerant and capable of continuing to operate even if some of its components
be inactive.

Transparency: Users of a distributed system do not need to worry about the

location or the state of the components being used. The system takes care of
manage communications and access to resources transparently.

Page 2
 Among the main applications of distributed systems, the following stand out:

Social networks: Social networks such as Facebook, Twitter, and LinkedIn are examples of systems
distributed systems that allow users from different geographic locations to connect and
share information in real time.

E-commerce systems: Online stores like Amazon and AliExpress are examples of
distributed systems that allow for large-scale commercial transactions to be carried out, with the
order processing, inventory management, and distribution of products in different
geographical regions.

Content management systems: Content management systems such as

WordPress and Joomla allow users in different geographical locations to create, edit and
share content on the web collaboratively.

2.1. Requirements of distributed systems

Transparency: distributed systems need to be transparent to users, or

Indeed, users should not have to know or worry about the location of resources and services.
that are being used.

Scalability: distributed systems need to be scalable, that is, they need to be capable
to handle a large number of users and resources without compromising performance.

Reliability: distributed systems need to be reliable, that is, they need to ensure that
integrity and availability of data and services, even in the event of failures or errors.

Security: distributed systems need to be secure, that is, they must ensure protection.
of data and services against unauthorized access and other threats.

3. Parallel Systems

Parallel systems are characterized by the use of multiple processors working.

set to perform tasks in parallel, with the aim of accelerating the processing of large
data volumes. These processors can be located on a single machine or across multiple ones.
interconnected machines, forming a high-performance computing cluster.
Page 3
 Parallel systems are especially useful in applications that involve intensive calculations,
such as numerical simulations, image processing, and pattern recognition.

Figure 2 - Parallel systems

Among the main characteristics of parallel systems, the following stand out:

Scalability: Parallel systems can be easily scaled by adding new

processors to increase processing capacity.

Granularity: Parallel systems can be implemented with different

granularities, that is, with different levels of task division among processors. The
fine granularity allows tasks to be divided into small pieces that can be
executed in parallel, while coarse granularity divides tasks into large chunks that
are executed in parallel.

Communication: The processors of a parallel system need to communicate with each other.
to exchange information and coordinate tasks. For this, it is necessary to use protocols of
efficient communication and synchronization mechanisms to avoid conflicts.

Among the main applications of parallel systems, the following stand out:

Image processing: Image processing involves the manipulation of large

real-time data volumes, which require high computational performance. The systems
parallels are widely used in this area to accelerate image processing in
surveillance applications, geospatial data analysis, among others.

Numerical simulations: Numerical simulations involve solving complex equations.

in various fields of science and engineering, such as physics, chemistry, biology, and mechanical engineering.

Page 4
Parallel systems are fundamental in this area to accelerate the processing of large
data volumes and reduce simulation time.

Pattern recognition: Pattern recognition involves the analysis of large

data volumes to identify patterns and trends. Parallel systems are widely
used in this area to accelerate data processing in artificial intelligence applications,
financial data analysis, among others.

3.1. Requirements of parallel systems

Scalability: parallel systems need to be scalable, that is, they need to be able to
lidar with a large number of processors and tasks, without compromising performance.

Synchronization: parallel systems need to have efficient synchronization mechanisms among

the processors, to avoid conflicts and ensure data consistency. Load balancing:
parallel systems need to have good load balancing, that is, they need to distribute the
tasks in a balanced manner among the processors, to prevent some processors from becoming
overloaded and others idle.

4. Challenges of distributed and parallel systems

Communication: distributed and parallel systems need to have efficient mechanisms for
communication between processors and resources, to ensure the exchange of information and coordination
the tasks.

Resource management: distributed and parallel systems need to have a good

resource management, that is, they need to allocate and deallocate resources efficiently, to
avoid waste and ensure the best possible use of available resources.

Programming: the programming of distributed and parallel systems is complex and requires
advanced knowledge in parallel and distributed programming techniques, which can be a
challenge for many developers.

Testing and debugging: distributed and parallel systems present specific challenges
regarding testing and debugging, due to its distributed and parallel nature, which can make
more difficult to identify and correct errors and flaws.

Page 5
 5. Distributed and parallel architectures

Distributed and parallel architectures are fundamental for efficient processing.

scalable of large volumes of data in various areas, from scientific computing to the
large-scale transaction processing. These architectures allow for various tasks
are to be executed simultaneously on different interconnected computers in a network,
increasing processing capacity and reducing response time.

A arquitetura distribuída: utiliza diversos computadores independentes interconectados em

network to perform a task. This approach allows processing to be distributed among
the different computers, increasing scalability and fault tolerance. The computers
can be organized in different network topologies, such as ring, star, or tree, and can be
used to perform different types of tasks, from scientific computing to
large-scale transaction processing.

Parallel architecture: consists of using multiple processors working together.

to perform a task, with the aim of reducing processing time. This approach
can be used at different levels, from performing simple tasks in a single
computer to the use of supercomputers to process large volumes of data.

One of the main advantages of distributed and parallel architectures is the possibility of
use of computer clusters for large-scale data processing. A cluster is
a group of interconnected computers that work together to perform a task. The
the use of clusters can be applied in different areas, such as data analysis, simulation
complex processes and high-performance computing.

Figure 3 - A cluster

Page 6
 In addition, distributed and parallel architectures can be used together to achieve
even more benefits, such as increased processing capacity and reduced time of
response. For example, it is possible to use clusters of interconnected computers on a network to
implement distributed parallel architectures, allowing tasks to be distributed among the
different processors.

In the context of cloud computing, distributed and parallel architectures are fundamental.
to ensure the scalability and efficiency of systems. Cloud services like Amazon
Web Services (AWS) or Microsoft Azure offer various resources for implementation
of these architectures, such as virtual machine instances, distributed databases and
batch processing tools.

There are several distributed and parallel architectures that can be used in computing.
current. Some of the most common are:

Client-server architecture: In this architecture, the services are distributed between two
main components: the client and the server. The client makes requests to the server, which
processes requests and returns results to the client. This architecture is widely
used in web systems and in distributed databases.

Figure 4 - Client-Server Architecture

Peer-to-peer architecture: In this architecture, all nodes in the network are equal and have the

ability to act as both client and server. The nodes can share resources, data and
services with each other, without the need for a centralized server. This architecture is commonly
used in file sharing systems and in decentralized social networks.

Page 7
 Figure 5 - Peer-to-peer architecture

Batch processing architecture: In this architecture, tasks are executed in

batches, that is, a set of tasks is processed all at once. This type of architecture is common
in large-scale data processing systems, such as image processing,
processing of financial data and analysis of research data.

Real-time processing architecture: In this architecture, tasks are processed

in real time, that is, the results are delivered instantly after the task is executed. This
architecture is common in industrial process control systems, monitoring systems
de tráfego e em sistemas de análise de dados em tempo real.

These are just a few examples of distributed and parallel architectures used in
modern computing. The choice of the most suitable architecture for a given system
it depends on several factors, such as the nature of the data to be processed, the response time
desired and the resources available for the implementation of the system.

As arquiteturas distribuídas e paralelas são utilizadas em uma grande variedade de aplicações

in various areas, such as:

Large-scale data processing systems: These systems are used for

process large volumes of data, such as social media data, medical records, data from
sales and finance. Distributed and parallel architectures allow these systems to process
data in parallel, significantly reducing the time needed to process large volumes
of data.

Industrial process control systems: These systems are used in processes

industries to control machines and equipment, monitor performance and adjust processes
in real time. Distributed and parallel architectures allow these systems to process
real-time information, ensuring a quick and accurate response to process events.

Page 8
 Artificial intelligence systems: These systems are used to analyze data.
complex, such as images, speech, text, and sensor data. Distributed architectures and
parallels allow these systems to process large volumes of data in parallel, reducing
significantly the time needed to train and execute machine learning models.

Online gaming systems: These systems are used to provide online gaming services,
such as multiplayer games and real-time games. Distributed and parallel architectures allow
that these systems process information in real time, providing a gaming experience
smooth and uninterrupted.

These architectures are fundamental to ensuring scalability, efficiency, and reliability.

of a wide variety of systems and applications in modern computing.

However, the implementation of distributed and parallel architectures involves various challenges,
how the communication and synchronization between the components, resource management and the
guarantee of reliability. Communication in distributed and parallel systems can be carried out
through different mechanisms, such as remote procedure calls, message passing
ou memória compartilhada. É necessário garantir que a comunicação seja realizada de maneira
efficient and safe, to avoid problems such as data loss or transmission conflicts
Information. Synchronization is also a critical aspect in the implementation of these architectures.
since it is necessary to ensure that the different components of the system are synchronized to
avoid concurrency problems, such as data inconsistency or information loss.

In summary, distributed and parallel architectures are a field of great importance in

current computing and present several advantages for processing large volumes of
data. However, its implementation involves several challenges, making it necessary an
careful planning to ensure that the system is efficient, scalable, and reliable.

6. Communication and synchronization in distributed systems and

parallels

Communication and synchronization are two critical aspects in distributed and parallel systems.
As these systems are composed of various processes running on different machines, it is
it is essential that these processes communicate with each other and coordinate to ensure the correct
functioning of the system.

Page 9
 Communication in distributed and parallel systems involves sending and receiving
messages between the processes or nodes that make up the system. Communication in systems
distribution can be done in several ways, including:

Message-based communication: In this communication model, the different

system components send and receive messages among themselves. Each message contains
Information about the task that needs to be performed or the result that needs to be returned.

Communication based on remote procedure call (RPC): In this model of

communication, the different components of the system use remote procedure calls to
invoke functions in other components of the system. This allows one component to request the
execution of a task in another component.

Object-based distributed communication: In this communication model, the different

system components interact with each other as if they were working with local objects. This
allows system components to access and manipulate objects in other components,
as if they were in the same system.

Synchronization is equally important in distributed and parallel systems, since

Different processes need to coordinate to ensure data consistency and avoid conflicts.
of access. There are several synchronization techniques used in distributed and parallel systems:

Blocking: Blocking is a synchronization technique that involves resource allocation.

shared exclusively for a process or node. This means that other processes or nodes
those who wish to access the same resource must wait until it is released by the process that holds it
is currently being used. Blocking is a simple technique, but it can lead to problems of
performance if many processes are competing for the same resource.

Traffic lights: Traffic lights are a more sophisticated synchronization mechanism that involves the
use of an object that controls access to a shared resource. The semaphore object can be
configured to allow or deny access to a shared resource depending on its state
currently. Traffic lights are useful for dealing with situations where multiple processes need to access the
the same resource, but only one process can access it at a time.

Barriers: Barriers are a synchronization technique used to ensure that multiple

processes reach a certain synchronization point before proceeding to the next
execution stage. A barrier is a synchronization point that all processes must

Page 10
reach before continuing execution. Barriers are useful to ensure that processes do not
proceed to the next stage of execution before everyone is ready.

Shared memory: Shared memory is a type of communication in which

the processes share a common memory region that can be read and written by all
processes that have access to it. Synchronization is necessary to ensure that processes do not
read or write in the same memory location at the same time, which can lead to results
incorrect

Message exchange: Message exchange is an asynchronous communication mechanism used

in distributed systems where processes send and receive messages to communicate with each other
with others. Synchronization is necessary to ensure that messages are processed in
the correct order is that the processes wait for the responses before proceeding with the execution.

These mechanisms are used to ensure that different processes run in sync and
we cooperate to complete a specific task.

Another important issue in distributed and parallel systems is scalability, that is, the
system's ability to handle a large number of processes running simultaneously.
As the number of processes increases, it is essential that the system is able to scale.
efficient and balanced way to ensure proper performance.

Moreover, security is another challenge in distributed and parallel systems. As these

systems are composed of various processes on different machines, it is necessary to ensure that the
data should be protected against unauthorized access or privacy violations.

In summary, communication and synchronization are critical aspects in distributed systems.

parallels, and the choice of the appropriate communication method and synchronization mechanism can
significantly affect the performance and scalability of the system. In addition, it is important
consider security issues when designing and implementing these systems.

Page 11
7. Conclusion

Distributed and parallel systems are an efficient solution to deal with the
processing large volumes of data and with the increasing demand for processing
in real time. However, the implementation of these systems presents several challenges, such as
the communication and synchronization between components, resource management, and tolerance
the failures.

To ensure the efficiency and safety of these systems, it is important that they are adopted.
efficient mechanisms for communication and synchronization, and that the resources are managed from
appropriate manner. Furthermore, it is essential that the system be able to handle failures of
hardware and software without interrupting the processing, to ensure the availability of the
services.

In summary, distributed and parallel systems are an efficient solution for dealing with
the processing of large volumes of data and with the demand for real-time processing
real. However, its implementation requires special care to ensure efficiency and the
security of the services offered.

Page 12
 8. Bibliography

Distributed systems: concepts and

project. Porto Alegre: Bookman.
Distributed systems: principles and
paradigms. São Paulo: Prentice Hall.
Mateos, C. F., García-Martínez, R., García-Galán, S., & Ríos-Navarro, A. (2019). One
Overview of Distributed and Parallel Systems in High Performance Computing
for Big Data: Methodologies and Applications (pp. 23-42). Springer, Cham.
Ozsu, M. T., & Valduriez, P. (2011). Principles of Database Systems
distributed. São Paulo: Pearson Prentice Hall.
Foster, I. (2015). Designing and Building Parallel Programs: Concepts and
Tools for Parallel Software Engineering. Bookman Publisher.
Stonebraker, M. (2010). The architecture of a parallel database system. In
Database Management Systems (pp. 535-551). Bookman.
Coulouris, G., Dollimore, J., & Kindberg, T. (2013). Distributed Systems: Concepts and
Project. Bookman Publishing.

Distributed Systems:
Principles and Paradigms. Pearson Prentice Hall.

Page 13