Distributed Computing
What is Distributed Computing?
• Distributed computing is a model in which processing tasks are
divided among multiple computers (called nodes) connected through
a network. These nodes work together to achieve a common goal,
appearing to the user as a single, unified system.
Real Time Example
• Imagine a group project where each student is assigned a specific part
of the work. Everyone works on their own piece, and finally, the
results are combined into one final report. That’s what distributed
computing does for complex computational problems.
Graphical Explanation
Example in Computing
• One example of a distributed computing system is a cloud computing
system, where resources such as computing power, storage, and
networking are delivered over the Internet and accessed on demand.
In this type of system, users can access and use shared resources
through a web browser or other client software.
Key Characteristics of
Distributed Systems
• Multiple Nodes: Many independent computers (servers, PCs, or
devices)
• Resource Sharing: Hardware, software, and data are shared across
nodes.
• Concurrency: Multiple tasks are executed simultaneously.
• Scalability: Systems can expand easily by adding more nodes.
• Fault Tolerance: If one node fails, others can continue working.
• Transparency: Users see the system as a single entity, not multiple
machines.
• Peer-to-Peer Architecture: Devices or systems in a distributed system can
act as both clients and servers, as they can both request and provide
services to other devices or systems in the network.
• Horizontal Scaling: Scaling a distributed computing system typically
involves adding more devices or systems to the network to increase
processing and storage capacity. This can be done through hardware
upgrades or by adding additional devices or systems to the network.
Goals of Distributed Computing
• Performance Improvement: Tasks complete faster by parallel execution.
• Reliability: Redundancy across nodes ensures minimal downtime.
• Scalability: Ability to handle growing workloads.
• Resource Utilization: Efficient use of CPU, storage, and memory spread
across multiple nodes.
Types of Distributed Computing
Systems
• Cluster Computing
• Grid Computing
• Cloud Computing
• Peer-to-Peer (P2P) Systems
Cluster Computing
A cluster is a group of computers (called nodes) that are closely connected
(usually through a Local Area Network LAN) and work together as a single,
powerful system.
Working Principle:
• A cluster typically has a master node that distributes tasks among slave/worker nodes.
• All nodes run the same operating system and are tightly coupled, meaning they are
physically located close together.
• If one node fails, another can take over (fault tolerance).
• Examples:
• Google Search Engine backend clusters.
• University research clusters for physics simulations.
• High-Performance Computing (HPC) labs for weather forecasting.
Graphical Explanation
Grid Computing
Grid computing connects computers from different locations, often over
the internet, to work on a common task. Unlike clusters, grids are loosely
coupled and can use heterogeneous systems (different OS, hardware, and
locations).
Working Principle:
• Idle resources from multiple systems across organizations or homes are pooled
together.
• A central server breaks down tasks and distributes them to different nodes.
• Results are collected and combined.
Examples:
• SETI@home: Analyzing radio signals for extraterrestrial life.
• Folding@home: Protein folding research for medical purposes.
• CERN’s Large Hadron Collider data analysis.
Graphical Explanation
Cloud Computing
Cloud computing provides on-demand access to computing resources
(like servers, storage, networking, and applications) over the internet.
Users pay only for what they use.
Working Principle:
Cloud providers maintain large data centers with thousands of servers. Users access
resources remotely through virtualization and APIs. Resources can be scaled up or
down based on demand.
Service Models:
IaaS (Infrastructure as a Service): Provides virtual servers, storage (e.g., AWS EC2).
PaaS (Platform as a Service): Provides a development platform for apps (e.g.,
Google App Engine).
SaaS (Software as a Service): Provides ready-to-use applications (e.g., Gmail,
Microsoft 365).
Graphical Explanation
Peer-to-Peer (P2P) Systems
In P2P systems, each computer (peer) acts both as a client and a server.
There is no central control, and all nodes share resources directly.
Working Principle:
• Each peer can upload and download files or provide and consume services.
• Nodes communicate directly with each other instead of relying on a central
server.
• The network grows stronger as more peers join.
Examples:
• BitTorrent: File sharing network.
• Blockchain: Bitcoin, Ethereum cryptocurrency systems.
• Skype (early versions): Used P2P for calls and messaging.
Graphical Explanation
Real World Applications
Online Banking: Transactions are processed by distributed servers
worldwide.
E-commerce: Amazon uses distributed systems for inventory,
payments, recommendations.
Social Media: Facebook distributes data across thousands of servers to
serve billions of users.
Scientific Research: Climate change models, DNA sequencing, and
space exploration use distributed computing.
Entertainment: Netflix, YouTube, and Spotify use distributed servers for
smooth streaming.
Advantages of Distributed
Computing
Speed: Tasks complete faster using parallel processing.
Cost-effective: Commodity hardware can be used instead of expensive
supercomputers.
High Availability: If one server fails, others take over.
Flexibility: Supports different applications (scientific, business, social).
Scalability: Easily handle increasing workloads. Collaboration: Enables
teamwork across locations.
Challenges of Distributed
Computing
Security Issues: Data spread across multiple servers can be attacked.
Example: Cloud data breaches.
Network Dependency: If the network is slow or fails, performance
suffers.
Synchronization Issues: Keeping data consistent across servers is difficult.
Example: Banking transactions must remain accurate on all servers.
Complexity: Designing and managing distributed systems is harder than
centralized ones.
Debugging and Monitoring: Failures are harder to trace when multiple
machines are involved.
"A positive attitude causes a chain reaction of positive
thoughts, events and outcomes. It is a catalyst and it
sparks extraordinary results" - Wade Boggs.
Thank You