2.

2 notes
• The term “parallel” implies a tightly coupled system while “distributed”
refers to
• a wider class of system including those who are tightly coupled.
• the term “parallel computing” refers to a model where the computation
is divided
• among several processors sharing the same memory.
• architecture: homogeneity of components: each processor is of the same
type and it has the same capability of the others.
• The shared memory has a single address space, which is accessible to all
the processors.
• parallel systems now include all those architectures that are based on the
concept of shared memory.

• distributed computing
• allows the computation to be broken down into units and
• executed concurrently on different computing elements
• whether these are processors on different nodes, processors on the same
computer, or cores within the same processor.
• elements might be heterogeneous in terms of hardware and software
features.
• Examples are computing grids or the internet computing systems.
2.3 notes
2.3.1
• Processing of multiple tasks simultaneously on multiple processors is
called parallel processing.
• The parallel program consists of multiple active processes (tasks)
simultaneously solving a given problem.
• A given task is divided into multiple subtasks using divide-and-conquer
technique,
• and each one of them is processed on different CPUs.
• Programming on multi–processor system using divide-and-conquer
technique is called parallel programming.

• Parallel processing provides a cost-effective solution- many applications

today require more computing power.
• The workload can now be shared between different processors.
• This results in higher computing power and performance than a single
processor system.
• The development of parallel processing is being influenced by many
factors:
1. computational requirements are ever increasing The technical computing
problems, which require high speed computational power
2. sequential architectures are reaching physical limitation as they are
constrained by the speed of light and thermodynamics laws.
3. significant development in networking technology is paving a way for
heterogeneous computing.
4. the technology of parallel processing is mature and can be exploited
commercially

2.4 Notes
2.4.1
A distributed system is a collection of independent computers that appears to its
users as a single coherent system - given by Tanenbaum

A distributed system is one in which components located at networked

computers communicate and coordinate their actions only by passing messages
- given by coulouris

2.4.2
• A distributed system is the result of the interaction of several components
that traverse the entire computing stack from hardware to software.
• give to the users the illusion of a single coherent system.
• At the very bottom layer, computer and network hardware constitute the
physical infrastructure;
1. these components are directly managed by the operating system for-
inter-process communication, process scheduling and management, etc.
2. Taken together, these two layers become- a platform on which specialised
software is deployed- to turn a set of networked computers into a distributed
system.
3. The use of well-known standards at the operating system allows easy
harnessing of heterogeneous components and their organisation into a coherent
and uniform system.
4. At the operating system level, inter-process communication services are
implemented on top of standardized communication protocols such as TCP/IP
and UDP.

• The middleware layer leverages such services.

1. to build a uniform environment for the development and deployment of
distributed applications.
2. This layer supports the programming paradigms for distributed systems.
3. By relying on the services offered by the operating system,
4. the middleware develops its own protocols, data formats, and
programming language or frameworks
5. for the development of distributed applications.

• The top of the distributed system stack is represented by the applications

and services
1. designed and developed to use the middleware.
2. often expose their features in the form of graphical user interfaces
accessible locally or through the Internet via a Web browser.
3. A very good example is constituted by Infrastructure-as-a-Service (IaaS)
providers such as Amazon Web Services (AWS),
4. which provides facilities for creating virtual machines, organizing them
together into a cluster,
5. and deploying applications and systems on top of it.

Cloud-computing Distributed system

top-> Application (Saas)
• Social Networks, Scientific Computing, Enterprise Applications
Middle-> Middleware (Paas)
• Frameworks for Cloud Application Development
Bottom-> Hardware and OS (Iaas)
• Virtual hardware, networking, OS images, and storage.
• hardware and operating system layer make up the bare bone
infrastructure of one or more datacenters
• where racks of servers are deployed and connected together through
highspeed connectivity.
• This infrastructure is managed by the operating system
• The core logic is then implemented in the middleware
• that manages the virtualization layer that is deployed on the physical
infrastructure in order to maximize its utilization and to
• provide a customizable runtime environment for applications.

2.4.3 (pg 58-61)

Architectural styles for distributed computing

• System architectural styles cover the physical organization of components

and processes over a distributed infrastructure.
• They provide a set of reference models for the deployment of such
systems
• two fundamental reference styles: client-server and peer-to-peer.

1. Client-Server:
• This architecture is very popular in distributed computing.
• the client-server model features two major components: a server and a
client.
• These two components interact with each other through a network
connection by using a given protocol.
• The communication is unidirectional: the client issues a request to the
server, and server after processing the request returns a response.
• There could be multiple client components issuing requests to the server
that is passively waiting for them.
• This model is suitable in many-to-one scenarios.
• where the information and the services of interest can be centralized and
accessed through a single access point: the server.
• For the client design, we identify two major models:
• Thin-client model:
1. the load of data processing and transformation is put on the server side
2. retrieving and returning the data it is being asked for, on the client side.
3. client has a light implementation.
• Fat-client model:
1. the client component is also responsible for processing and transforming
the data before returning it back to the user
2. while the server features a relatively light implementation mostly
concerned with the management of access to the data.
• We can identify three major components in the client-server model:
presentation, application logic, and data storage.
• Presentation, application logic, and data maintenance can be seen as
conceptual layers, which are more appropriately called tiers.
• multi-tiered architectures. Two major classes exist:
1. Two-tier architecture:
• This architecture partitions the systems into two tiers, which are located
one in the client component and the other on the server.
• The client is responsible for the presentation tier by providing a user
interface
• the server concentrates the application logic and the data store into a
single tier.
• This architecture is suitable for systems of limited size and suffers from
scalability issues.
• In particular, as the number of users increases, the performance of the
server might dramatically decrease.
• Another limitation is caused by the dimension of the data to maintain,
manage, and access.
2. Three-tier Architecture/N-tier Architecture:
• separates the presentation of data, the application logic, and the data
storage into three tiers.
• This model is generally more scalable than the previous one because it is
• possible to distribute the tiers into several computing nodes, , thus
isolating the performance bottlenecks.
• also more complex to understand and manage.
• Ex: medium-size Web application that relies on a relational database
management system for storing its data.
• Within this scenario, the client component is represented by a Web
browser that embodies the presentation tier,
• while the application server encapsulates the business logic tier, and a
database server machine maintains the data storage.
• Application servers that rely on third party (or external) services in order
to satisfy the requests of clients are examples of N-tiered architectures.
The client-server architecture has been the dominant reference model for
designing and deploying distributed systems.

This model is generally suitable in case of a many-to-one scenario, where the

interaction is unidirectional and started by the clients. This model suffers from
scalability issues, therefore it is not appropriate in very large systems.

2. Peer-to-peer:
• This model introduces a symmetric architecture where all the
components, called peers, play the same role
• and incorporate both the client and server capabilities of the previous
model.
• each peer acts as a server when it processes requests from other peers
• and as a client when it issues requests to other peers.
• Therefore, this model is quite suitable for highly decentralized
architecture, which can scale better along the dimension of the number of
peers.
• The disadvantage of this approach is that the management of the
implementation of algorithms is more complex if compared to the client-server
model.
• Ex: file sharing applications such as Gnutella, BitTorrent, and Kazaa.
• in Kazaa, not all the peers have the same role, and some of them are used
to group the accessibility information of a group of peers. Another interesting
example of peer-to-peer architecture is represented by the Skype network.