Total Marks: 04
Obtained Marks:
ParallelComputing
and Distributed
Assignment
Section-7B# 3
Submitted To: Dr Farhat
______________________________________________________________________________
Student Name: Karar Hussain
______________________________________________________________________________
Reg Number: 2112375
_______________________________________________________________________________
Q # 1 What are the main differences between a parallel system and a
distributed system?
A parallel system is designed to perform multiple computations simultaneously
using multiple processors that share a common memory space. These
processors typically reside within the same physical machine, such as a multi-
core CPU or a supercomputer. Communication between processors is fast and
occurs through shared memory, which enables high-speed data access. Parallel
systems are best suited for tasks that require intensive computation, such as
scientific simulations, graphics rendering, or real-time data processing. However,
their scalability is limited by hardware constraints, and they tend to be less
fault-tolerant—if one processor fails, it may affect the entire system.
In contrast, a distributed system consists of multiple independent computers, or
nodes, that are connected over a network and work together to achieve a
common objective. Each node in a distributed system has its own private
memory, and communication between nodes is done through message passing,
which can introduce latency. Distributed systems are inherently more scalable,
as more nodes can be added easily, and they are generally more fault-tolerant—
failures in one node usually don’t compromise the entire system. These systems
are commonly used in cloud computing, web services, and large-scale data
processing applications. In summary, while both systems aim to improve
performance through concurrent processing, they differ significantly in
architecture, communication methods, and fault tolerance.
Q # 2 Identify some distributed applications in the scientific and commercial
application areas. For each application, determine which of the motivating
factors listed in this Lecture are important for building the application over a
distributed system.
Distributed systems are widely used in both scientific and commercial
applications due to their scalability, fault tolerance, and ability to handle large
amounts of data. Here are some examples of distributed applications in both
areas, along with the motivating factors that make distributed systems ideal for
these applications:
Scientific Applications
Climate Modeling and Weather Prediction
Motivating Factors:
Scalability: Weather prediction models require significant computational
resources to simulate complex atmospheric conditions. Distributed systems can
easily scale by adding more nodes to handle larger datasets and more complex
simulations.
Fault Tolerance: Given the critical nature of weather predictions, distributed
systems provide redundancy—if one node fails, others can continue to process
the data.
Parallelism: Many parts of the model can be run in parallel (e.g., different
geographic regions or different atmospheric layers), making distributed systems
ideal for dividing the workload.
Data Storage and Access: Massive datasets from satellites, sensors, and previous
predictions must be stored and accessed quickly and reliably, which is facilitated
by distributed databases.
Genomics and Bioinformatics
Motivating Factors:
Data Distribution: Genomic data (e.g., DNA sequences) is large and often stored
across different locations. Distributed systems allow for efficient data retrieval
and processing across various servers.
Scalability: Genome sequencing can involve massive datasets that require large-
scale computation, which can be efficiently handled by a distributed system that
can scale as needed.
Fault Tolerance: Genomic data analysis involves critical research. A distributed
system ensures that if one node fails, the analysis can continue without losing
data.
Parallelism: Tasks like sequence alignment or variant calling can be broken down
into smaller tasks and run simultaneously across multiple nodes.
High-Energy Physics Simulations (e.g., CERN)
Motivating Factors:
Scalability: High-energy physics experiments generate vast amounts of data,
often requiring significant computational power. A distributed system can scale
by leveraging thousands of machines to simulate and process the data.
Fault Tolerance: Given the high cost and complexity of experiments like those at
CERN, distributed systems provide fault tolerance and ensure that data is not
lost if one part of the system fails.
Parallelism: Many experiments can be split into smaller tasks, allowing parallel
processing to expedite the results.
Data Storage and Access: These applications involve large data repositories (e.g.,
particle collision data), which require distributed storage systems for efficient
access and redundancy.
Commercial Applications
E-commerce Platforms (e.g., Amazon, eBay)
Motivating Factors:
Scalability: E-commerce platforms must handle large volumes of transactions
and user interactions, especially during peak periods like holidays. Distributed
systems provide the scalability needed to handle such loads by distributing the
workload across multiple servers.
Fault Tolerance: E-commerce sites must remain available 24/7, even during
traffic surges. Distributed systems provide redundancy, so if one node fails,
others can take over without affecting service.
Data Distribution: Product catalogs, user profiles, and transaction data are often
stored across different geographic locations. A distributed system allows for
efficient data storage and retrieval across regions.
Load Balancing: Distributed systems can balance incoming traffic across multiple
servers, ensuring that no single server becomes a bottleneck.
Cloud Computing Services (e.g., AWS, Microsoft Azure)
Motivating Factors:
Scalability: Cloud services must dynamically scale based on customer demand.
Distributed systems allow the underlying infrastructure to scale horizontally by
adding more servers or resources as needed.
Fault Tolerance: Cloud services often support critical business applications, so
ensuring uptime and reliability is crucial. Distributed systems offer redundancy
and failover mechanisms to prevent service interruptions.
Load Balancing: Distributed cloud systems can balance workloads across
multiple data centers, ensuring that no single location becomes overwhelmed
with requests.
Data Distribution: Cloud services often need to store massive amounts of data in
different locations, ensuring data accessibility and reducing latency for users
across the globe.
Financial Services (e.g., Stock Trading Platforms)
Motivating Factors:
Scalability: High-frequency trading platforms require the ability to process
thousands of transactions per second. Distributed systems allow the processing
load to be distributed across multiple servers, ensuring that the system can
handle large volumes of data.
Fault Tolerance: Financial systems must ensure reliability and high availability to
maintain market integrity. Distributed systems provide redundancy to avoid
downtime and prevent data loss.
Data Distribution: Financial transactions and market data must be stored and
processed in various locations. Distributed systems allow the seamless
integration of data from multiple sources and regions.
Parallelism: Market data analysis can be done in parallel across multiple nodes,
making the system more efficient and capable of real-time data processing.
Online Gaming (e.g., MMORPGs)
Motivating Factors:
Scalability: Online multiplayer games often have millions of players
simultaneously. Distributed systems allow the game’s backend to scale
horizontally by adding more servers to accommodate a growing player base.
Fault Tolerance: Given the real-time nature of online gaming, servers must
remain operational without interruptions. Distributed systems provide fault
tolerance by replicating data and services across multiple nodes.
Load Balancing: Distributed systems can balance player connections and game
state across multiple servers, reducing latency and preventing overload on
individual servers.
Data Distribution: Game data, such as player progress, inventories, and world
states, are distributed across various nodes to ensure quick access and
synchronization across all players.
Q # 3 For the following, determine and explain whether it is an example of
distributed
computing
A. Using Excel on a standalone PC
B. Web Surfing
C. Instant Messaging
D. Using email on your Department Computer and send it to yourself
E. Using [Link] to download audio/Videos
A. Using Excel on a Standalone PC
Explanation: When you're using Excel on a standalone computer, everything
happens on that single machine. The program runs on your local processor,
accesses the data stored locally, and doesn’t communicate with any other
machines. There’s no sharing of resources or data between multiple computers
in this case.
Conclusion: Not Distributed Computing. This is a typical single-computer
operation, with no involvement of a network or multiple systems working
together.
B. Web Surfing
Explanation: When you browse the web, your computer communicates with
web servers across the internet. The websites you visit are stored on servers
located in various places, often in a distributed network. For instance, when you
visit a page, your browser makes requests to servers, which respond by sending
back the requested data (like text, images, and videos). The infrastructure
behind web surfing is typically distributed, meaning many servers handle
different aspects of the data.
Conclusion: Yes, Distributed Computing. Web surfing involves your computer
interacting with a network of distributed servers to access content.
C. Instant Messaging
Explanation: Instant messaging allows you to send and receive messages in real-
time over the internet. While the IM application itself may be installed on your
device, the messages are sent to and received from servers that are part of a
distributed system. These servers ensure that messages are delivered, often
storing them temporarily until they can be sent to the intended recipient,
whether they are online or offline.
Conclusion: Yes, Distributed Computing. Instant messaging depends on
distributed servers that manage the communication between users.
D. Using Email on Your Department Computer and Sending It to Yourself
Explanation: When you send an email from your department computer to
yourself, it’s handled by an email server. These servers are part of a distributed
network that helps store and deliver emails. The email is sent through a series of
intermediate servers (like SMTP, IMAP, or POP3) and is stored in different places
before you retrieve it on your device. Even if the email is sent to yourself, it still
involves communication with multiple machines and servers.
Conclusion: Yes, Distributed Computing. Email systems rely on a distributed
infrastructure to send, store, and retrieve messages.
E. Using [Link] to Download Audio/Videos
Explanation: When you use YouTube, you're accessing videos stored on
YouTube's distributed network of servers. These servers are located in different
regions and work together to store and deliver the video content to users.
YouTube also uses a system called Content Delivery Networks (CDNs) to ensure
that videos load quickly by distributing the content closer to users’ locations.
This involves multiple servers working together, and communication happens
between your device and these servers.
Conclusion: Yes, Distributed Computing. YouTube relies on a distributed system
of servers to deliver videos and audio to users.