Unit-2

Cloud Computing Architecture

Architecture of cloud computing is the combination of both SOA (Service Oriented
Architecture) and EDA (Event Driven Architecture). Client infrastructure, application,
service, runtime cloud, storage, infrastructure, management and security all these are the
components of cloud computing architecture.

The cloud architecture is divided into 2 parts, i.e.

1. Frontend
2. Backend
The below figure represents an internal architectural view of cloud computing.

Architectural Styles of Cloud Computing

Cloud computing architecture refers to the structure of components and services that make up
a cloud system. It defines how cloud resources are organized, managed, and delivered to
users. The architectural styles of cloud computing can be categorized based on deployment,
service delivery, and system design approaches.

1. Service-Oriented Architecture (SOA)

  Concept:
SOA is a design style where cloud services are delivered as independent, reusable
components (services) that communicate through standard protocols (like HTTP,
XML, SOAP, REST).
 Features:
o Promotes interoperability among heterogeneous systems.
o Each service performs a specific business function.
o Enables scalability and flexibility in the cloud.
 Example:
AWS Lambda functions invoked through an API Gateway follow the SOA principle.

2. Layered Architecture

Cloud architecture is often organized into layers, each handling a specific aspect of service
delivery.

Main layers:

1. Cloud Infrastructure Layer – Physical servers, storage, and networking.

2. Virtualization Layer – Abstracts and manages physical resources using virtual
machines or containers.
3. Platform Layer – Provides runtime environments, databases, and development tools
(e.g., AWS Elastic Beanstalk, Google App Engine).
4. Application Layer – End-user applications like Gmail, Salesforce, or Office 365.
5. Management & Security Layer – Handles monitoring, billing, authentication, and
compliance.

3. Microservices Architecture

 Concept:
The application is divided into small, independent services that communicate via
lightweight APIs.
 Advantages:
o Easier scalability and maintenance.
o Independent deployment of components.
o Resilience — failure in one service doesn’t affect others.
 Example:
Netflix and Amazon use microservices for their scalable cloud-based systems.

4. Event-Driven Architecture (EDA)

 Concept:
Systems react to events (changes in data or system states) rather than direct requests.
  Used In:
Real-time analytics, IoT, and serverless platforms (e.g., AWS Lambda triggers on S3
uploads).
 Benefits:
o High responsiveness.
o Asynchronous processing.
o Better scalability for dynamic workloads.

5. Multi-Tenant Architecture

 Concept:
A single instance of a cloud application serves multiple customers (tenants) while
keeping their data isolated.
 Advantages:
o Efficient resource utilization.
o Cost-effective.
o Simplifies updates and maintenance.
 Example:
SaaS platforms like Salesforce or Microsoft 365.

6. Resource Pooling and Virtualized Architecture

 Concept:
Uses virtualization to pool physical resources (servers, storage, network) and allocate
them dynamically to users.
 Benefits:
o Elasticity and scalability.
o Efficient use of resources.
o Foundation for IaaS platforms (e.g., AWS EC2, Google Compute Engine).

7. Hybrid Cloud Architecture

 Concept:
Combines public and private clouds for flexibility and control.
 Example:
Sensitive data stored in a private cloud, while analytics run on a public cloud.
 Benefits:
o Balance between cost and security.
o Supports dynamic workloads.

8. Serverless Architecture

 Concept:
Developers focus on code, while the cloud provider automatically manages servers
and scaling.
  Examples:
AWS Lambda, Azure Functions, Google Cloud Functions.
 Advantages:
o No server management.
o Pay-per-execution model.
o Highly scalable and cost-efficient.

Summary Table

Architecture Style Key Feature Example

Service-Oriented (SOA) Modular, service-based AWS API Gateway
Layered Structured into layers Cloud stack model
Microservices Independent small services Netflix, Amazon
Event-Driven Triggered by system events AWS Lambda
Multi-Tenant Shared instance, isolated data Salesforce
Virtualized Resource pooling AWS EC2
Hybrid Cloud Public + Private mix Azure Hybrid Cloud
Serverless No server management Google Cloud Functions

Workflows: Coordination of Multiple Activities

A workflow is a sequence of interconnected tasks or activities that must be executed in a
specific order to achieve a particular goal.
In cloud computing or data processing, workflows coordinate multiple services,
applications, or computational steps — often across distributed systems.

Workflows are used to:

 Automate business processes

 Manage data pipelines
 Control the execution order of cloud-based tasks

Example: Cloud-Based Data Processing Workflow

Let’s take a simple data analytics workflow as an example:

Goal: Process uploaded sales data and generate a business report automatically.

Steps in the Workflow

1. Data Upload:
The user uploads a CSV file to a cloud storage (e.g., AWS S3 or Google Cloud
Storage).
 2. Data Validation:
A validation service checks file format and data consistency.
3. Data Cleaning:
Invalid or missing records are cleaned or corrected.
4. Data Analysis:
Analytical functions calculate key metrics (e.g., total sales, top-selling products).
5. Report Generation:
The results are formatted into a report (e.g., PDF or dashboard).
6. Notification:
The system sends an email or message to the user that the report is ready.

Diagram: Workflow Coordination

Below is a simple representation of this workflow:

┌──────────────────┐
│ User Uploads │
│ Sales Data │
└───────┬──────────┘
│
▼
┌──────────────────┐
│ Data Validation │
│ (Check format) │
└───────┬──────────┘
│
▼
┌──────────────────┐
│ Data Cleaning │
│ (Remove errors) │
└───────┬──────────┘
│
▼
┌──────────────────┐
│ Data Analysis │
│ (Compute metrics) │
└───────┬──────────┘
│
▼
┌──────────────────┐
│ Report Generation │
│ (PDF/Dashboard) │
└───────┬──────────┘
│
▼
┌──────────────────┐
│ Notification │
│ (Email/SMS sent) │
└──────────────────┘
Key Points:

 Each step is a task or activity in the workflow.

 The workflow ensures tasks are executed in sequence or in parallel, depending on
dependencies.
 Workflow coordination tools (e.g., AWS Step Functions, Apache Airflow, Azure
Logic Apps) automate this process.

Coordination Based on a State Machine Model in Cloud Computing

In cloud computing, many distributed applications consist of multiple components (servers,

services, or processes) that must coordinate with one another to stay consistent and reliable.

A state machine model provides a formal way to represent and manage coordination among
these distributed components by tracking their states and transitions.

What Is a State Machine?

A state machine (or finite state machine – FSM) is a mathematical model that defines:

 A set of states a system can be in.

 A set of transitions between states triggered by events or actions.

Each service or node in a distributed system can be represented as a state machine that:

 Starts in an initial state,

 Responds to inputs/events,
 Wait state
 Changes to a new state,
 Transition state
 Produces outputs (actions or messages).

Why Coordination Is Needed

In cloud systems, many components work together — for example:

 Distributed databases,
 Microservices,
 Workflow managers,
 Cluster coordination systems.

Each component may fail or respond at different times.

Hence, a coordination service ensures that:
  All nodes agree on the current state,
 Operations occur in a consistent order, and
 The system recovers correctly after a failure.

5. Example: Apache ZooKeeper

A practical example of state-machine–based coordination is Apache ZooKeeper — a

coordination service used in cloud and distributed systems.

ZooKeeper:

 Maintains configuration and synchronization among distributed applications.

 Uses a replicated state machine model.
 Ensures all ZooKeeper servers hold identical state data through an atomic
broadcast protocol (Zab).

7. Diagram: State Machine Coordination

┌──────────────┐
│ Client │
└──────┬───────┘
│ Request
▼
┌──────────────┐
│ Leader Node │
└──────┬───────┘
Broadcast │ Operation
┌─────────┴─────────┐
▼ ▼
┌──────────────┐ ┌──────────────┐
│ Replica Node │ │ Replica Node │
└──────┬───────┘ └──────┬──────┘
│ │
Execute Operation Execute Operation
▼ ▼
Identical State Identical State

Case Study and Specialized Cloud Applications

1. A Case Study: The Gre – The Web Application

“The Gre” (Generic Resource Environment) is a cloud-based web application model that
demonstrates how web apps can efficiently utilize cloud infrastructure for scalability,
reliability, and cost-efficiency.

Key Characteristics:

 It is service-oriented, modular, and scalable.

 Designed to handle large-scale web traffic using cloud resources on demand.
 Provides a multi-tier architecture — separating the presentation, application logic,
and data layers.

Architecture Components:

1. Frontend Layer (Client Interface)

o Built using web technologies (HTML, JavaScript, CSS).
o Connects to the cloud-based backend via REST APIs.
2. Application Layer (Middleware)
o Hosted on cloud servers or PaaS platforms (e.g., AWS Elastic Beanstalk,
Azure App Service).
o Handles business logic, authentication, and routing.
3. Database Layer (Data Storage)
o Utilizes cloud databases (e.g., AWS RDS, Google Cloud SQL, MongoDB
Atlas).
o Automatically scales based on user demand.
4. Content Delivery & Caching
o Uses CDNs (Content Delivery Networks) like CloudFront or Akamai for
faster content delivery.

 Dynamic scaling during peak loads.

 Improved performance through caching and load balancing.
 Reduced operational cost via pay-as-you-go model.
 High availability and disaster recovery.

The Gre illustrates how a modern web application can be deployed, scaled, and managed
entirely in the cloud, achieving performance and flexibility that traditional hosting cannot.

State-machine model — The Gre — The Web Application

Below is a clear state-machine diagram and explanation you can use for the case study
“The Gre — The Web Application”.
It models how the web app coordinates request handling, data processing, caching, and error
recovery using a replicated/state-driven approach.

ASCII state-machine diagram (readable in plain text)

[Start]
|

[Idle / Waiting]
| (HTTP Request arrives)

[Authenticate] <--- (auth failure) --- [Error / Respond 401]

| (success)

[Authorize / RateLimit] --- (not allowed) --> [Error / Respond 403]

|

[Validate Input] --- (invalid) ---> [Error / Respond 400]

|

[Check Cache] ----(hit)----> [Serve From Cache] --> [Log & Notify] --> [End]
|
(miss)

[Fetch Data from DB / Services] --(data unavailable)--> [Retry / Fallback] --> [Error /
Respond 503]
|

[Process / Transform Data] --- (processing error) ---> [Error / Rollback]

|

[Render Response / Generate View]

|

[Update Cache (async)]

|

[Send Response to Client]

|

[Log / Metrics / Notify]

|

[Idle / Waiting] <-- loop back for next request

States, Events & Actions — brief explanation

 Idle / Waiting — service waiting for incoming requests.

 Authenticate — validate user's identity (token/session).
Event: AuthSuccess / AuthFail.
Action: verify token, set user context.
 Authorize / RateLimit — check permissions & rate limits.
Event: Allowed / Blocked.
 Validate Input — check request payload/params.
Action: schema validation, sanitization.
  Check Cache — look up fast cache (CDN, Redis).
Outcome: Hit (serve quickly) or Miss (fetch backend).
 Fetch Data from DB / Services — call databases, microservices, or external APIs.
Concern: retries, circuit-breaker, fallback.
 Process / Transform Data — apply business logic, aggregations, MapReduce-style
jobs if needed.
 Render Response / Generate View — construct JSON/HTML for client.
 Update Cache (async) — store computed result for future requests.
 Send Response to Client — final HTTP response.
 Log / Metrics / Notify — telemetry, audit trail, and notifications.
 Error / Respond (400/401/403/503/etc.) — error handling, rollback, and client-
facing messages.

2. Cloud for Science and Engineering

Scientific and engineering applications often require massive computational resources for
simulation, modeling, and data analysis.
Cloud computing offers an on-demand, scalable infrastructure for these tasks without
requiring expensive supercomputers.

Applications:

 Physics and Engineering Simulations: Finite Element Analysis (FEA),

Computational Fluid Dynamics (CFD).
 Astronomy and Space Research: Processing telescope data using distributed cloud
clusters.
 Genomics and Bioinformatics: Sequence alignment, protein structure prediction,
and data analytics.
 Climate and Weather Modeling: Running large-scale environmental simulations on
cloud-based HPC systems.

Advantages:

 Elastic scalability: Researchers can instantly scale computing nodes up or down.

 Reduced costs: No need for on-premise supercomputers.
 Collaboration: Scientists can share results and access datasets globally.
 Reproducibility: Cloud environments can be easily replicated for consistent
experiment results.

Tools/Platforms:

 Amazon EC2 for Science

 Google Cloud for Research
 Microsoft Azure for Scientific Computing

⚡ 3. High-Performance Computing (HPC) on the Cloud

Overview:

HPC involves performing complex computations at high speed using parallel processing
across multiple CPUs or GPUs.
Cloud computing provides HPC capabilities through virtualized clusters and specialized
instances optimized for heavy workloads.

Key Components:

1. Compute Nodes: Virtual machines or bare-metal instances optimized for CPU/GPU

workloads.
2. High-Speed Networking: Low-latency interconnects (e.g., AWS Elastic Fabric
Adapter, InfiniBand).
3. Parallel File Systems: Fast storage for large datasets (e.g., Lustre, Amazon FSx).
4. Job Scheduling: Managed through HPC schedulers like Slurm or AWS Batch.

Benefits:

 Scalable on demand — resources can be provisioned temporarily for peak

workloads.
 Cost-effective — pay only for the computation time used.
 Accessible globally — no need for physical supercomputer labs.
 Integration with AI/ML for scientific predictions and optimizations.

Examples:

 NASA using AWS for space mission simulations.

 CERN using OpenStack cloud for particle collision data analysis.

4. Cloud Computing for Biology Research

Overview:

Modern biology and genomics generate massive datasets (terabytes to petabytes).

Cloud computing enables storage, processing, and sharing of this data efficiently.

Use Cases:

1. Genomic Data Analysis:

o Aligning DNA sequences using cloud tools like AWS Genomics CLI or
Google Genomics API.
2. Proteomics and Molecular Modeling:
o Running simulations of protein folding using GPU instances.
3. Epidemiological Modeling:
o Using cloud data analytics to predict disease spread (e.g., COVID-19
modeling).
4. Biological Databases Hosting:
 o NCBI, ENA, and EMBL host cloud-accessible biological datasets.

Advantages:

 Collaboration across research institutions via shared datasets.

 Elastic compute for data-heavy analysis pipelines.
 Cost-efficient research, especially for smaller labs.
 Integration with AI/ML tools for faster discovery.

Example Platforms:

 Galaxy Project (open-source bioinformatics platform on the cloud)

 AWS Genomics Cloud, Google Cloud Life Sciences, DNAnexus

5. Social Computing, Digital Content, and Cloud Computing

Overview:

Social computing involves applications that facilitate interaction and collaboration among
users, such as social networks, blogs, wikis, and collaborative tools.
These platforms rely heavily on cloud infrastructure to handle dynamic workloads and vast
amounts of user-generated data.

Cloud’s Role:

 Provides scalable storage for user posts, media, and interactions.

 Offers real-time data processing for feeds, notifications, and analytics.
 Ensures high availability and low latency through content delivery networks
(CDNs).
 Enables AI-based personalization (recommendations, ad targeting).

Examples:

 Facebook & Instagram use distributed cloud data centers for handling billions of
interactions.
 YouTube & Netflix leverage cloud CDNs for video streaming.
 Twitter uses cloud analytics for trend prediction and spam detection.
 Google Workspace & Slack rely on cloud collaboration tools for team productivity.

Digital Content and Cloud:

 Content Storage: Cloud storage like AWS S3 or Google Drive stores terabytes of
videos, images, and documents.
 Streaming: Cloud servers dynamically deliver high-definition media to users.
 Content Security: Encryption and access control protect user data and intellectual
property.
MapReduce programming model to design a workflow that processes large-scale genomic
data for biological research. (10 marks)

1. Requirements & objectives

Design a scalable MapReduce workflow to process billions of short reads (FASTQ) to

produce biologically meaningful outputs (e.g., per-position variant counts / allele frequencies,
k-mer frequencies, or candidate variant calls). Objectives: handle terabytes of data, parallelize
across cluster, tolerate faults, and produce reproducible, annotated results.

2. High-level MapReduce workflow

1. Ingest & Preprocess (non-MapReduce / lightweight distributed step): load FASTQ

from object store (S3/HDFS), perform quality filtering and adapter trimming (tools
like fastp). Output cleaned reads stored back to HDFS/S3.
2. MapReduce Job 1 — Read Alignment (optional MapReduce or use distributed
aligner): map step calls an aligner (e.g., BWA-MEM wrapped in mapper) to align
reads to reference; emit (chromosome,position) → basecalls. Alternatively use
distributed aligners or Spark-based solutions for performance.
3. MapReduce Job 2 — Per-position base counting (the canonical MapReduce use):
o Map: for each aligned read, emit key = (chr, pos, refBase) or (chr,pos) with
observed base/quality.
o Combiner: locally aggregate counts to reduce shuffle.
o Reduce: aggregate counts across mappers to produce per-position allele
counts and compute allele frequencies; apply filters (min coverage, base
quality).
4. Post-processing / Variant Calling & Annotation: From aggregated counts call
candidate variants (simple heuristics or call with GATK using reduced data).
Annotate variants with databases (dbSNP, ClinVar) in another MapReduce or
downstream pipeline.
5. Indexing & Export: Save results as compressed VCF/Parquet and publish to analysis
portal.

(Flow: Preprocess → Align → MapReduce counts → Variant calling → Annotation)

4. Practical considerations — storage, tools, fault tolerance, performance (2 marks)

 Storage: raw FASTQ/alignments in HDFS or cloud object store (S3). Use

compressed formats (bgzip, Parquet) to reduce IO.
 Compute framework: Hadoop MapReduce can be used, but Apache Spark
(map/reduce-like operations with in-memory) often gives better performance for
iterative genomic tasks.
  Distributed aligners: Wrap BWA in mappers or use specialized distributed aligners
(e.g., ADAM, Spark-BWA).
 Fault tolerance: MapReduce automatically retries failed mappers/reducers. Use
checkpointing for long jobs and split into smaller genomic windows to reduce rework.
 Optimization: Use combiners, increase mapper memory for aligners, tune block size,
and use range partitioner to avoid reducer skew.
 Security & compliance: encrypt data at rest/in transit, implement IAM, and handle
sensitive human genomic data per regulations (consent, de-identification).

 Validation: compare called variants with benchmark datasets (e.g., GIAB) to

compute precision/recall. Perform QC metrics (coverage, duplication).
 Outputs: per-position count tables, VCF of variants, summary stats, and annotated
variant tables.
 Reproducibility: document container images (Docker/Singularity), pipeline
definitions (CWL/Nextflow), parameter sets, and use versioned reference genomes.

Using MapReduce (or Spark) with careful partitioning, combiners, and alignment strategy
allows scalable processing of large genomic datasets to produce per-position allele
information and variants for downstream biological interpretation. Proper tuning, validation,
and secure data handling complete a production-grade genomic MapReduce workflow.