Module – 2 AWS Compute & Storage
Services
Introduction to AWS Compute
• AWS Compute refers to the collection of services provided by Amazon Web Services (AWS)
that enable users to run applications and manage computing resources in the cloud.
• It is categorized under Infrastructure as a Service (IaaS), allowing businesses to provision
and manage virtual machines, containers, and serverless computing resources.
• AWS provides many flexible computing services to meet the requirements of business
organizations like
• Amazon Elastic Compute Cloud (EC2),
• Amazon Elastic Container Service (ECS),
• Amazon Elastic Container Service for Kubernetes (EKS),
• Amazon Lightsail,
• AWS Lambda and many more
• This infrastructure as a service can be considered as the processing power required by your
applications, to host applications or run computation-intensive tasks.
• In AWS, with the use of these computing services, users can dynamically provision
the number of resources they are using and then pay only for the computing
resources they have used for.
• This leads to the reduction of the upfront capital investment required.
• These compute resources are closely related to regular server components like CPU
and RAM.
• However, for regular server components, you need to manage and buy the
infrastructure, provide for backups and emergency recovery, and ensure enough
server capacity to handle traffic-intensive times.
• With AWS compute all this headache is handed over to the AWS team.
Introduction to AWS Compute
Traditional Servers AWS Compute Services
No physical servers; use AWS’s virtual
Buy and maintain physical hardware.
resources.
Pay for full capacity even when idle. Pay only for what you use (cost-efficient).
Handle backups, security, and scaling manually. AWS manages everything automatically.
Risk of over/under-provisioning during traffic
Auto-scaling adjusts resources as needed.
spikes.
Amazon Elastic Compute Cloud (EC2)
Amazon Elastic Compute Cloud (EC2) is a web service that provides resizable compute
capacity in the cloud.
• It allows users to run virtual servers, known as instances, on demand.
• EC2 is designed to make web-scale cloud computing easier for developers by
providing a reliable, scalable, and secure environment for hosting applications.
• Amazon EC2 reduces the time required to obtain and boot new server instances
(called Amazon EC2 instances) to minutes, allowing you to quickly scale capacity,
both up and down, as your computing requirements change.
• Amazon EC2 changes the economics of computing by allowing you to pay only for
capacity that you actually use.
AWS EC2- Key Components
AWS EC2- Key Components
Amazon EC2 instance types
Instance type and Naming Convention
General Purpose Instances
• EC2 instances are like different types of computers in the cloud, each designed
for specific workloads.
• These provide a balance of compute, memory, and networking resources.
• They are suitable for a variety of applications that don't require optimization
in any specific area, such as web servers, small databases, and development
environments.
Example: Web servers (company websites, blogs)
Small databases (MySQL for a small business)
Development/testing environments
Compute Optimized Instances:
• These are designed for applications that require high-performance processors, such as
batch processing, high-performance computing, and machine learning workloads.
They offer powerful CPUs for compute-intensive tasks.
• Example:
• Batch processing: handling huge amounts of data in one go (think of sorting
millions of files)
• High-performance computing (HPC): solving complex problems, like climate
modeling or drug simulations
• Machine learning: training AI models using tons of data
Memory Optimized Instances
• These are optimized for workloads that process large datasets in memory, like
high-performance databases, distributed web-scale in-memory caches, and
real-time big data analytics. They provide high memory capacity.
• Example:
• Big databases (e.g. banking or e-commerce systems)
• Super fast web apps that serve millions of users
• Live data analysis (e.g. watching how people shop in real time)
Storage Optimized Instances
• These are designed for workloads that require high, sequential read and
write access to large datasets on local storage, such as distributed file
systems, data warehousing, and NoSQL databases. They offer high
storage performance.
Accelerated Computing Instances
These utilize hardware accelerators like GPUs to perform certain functions more
efficiently, including machine learning, high-performance computing, and
graphics-intensive applications.
Example:
• Machine Learning: Training AI models super fast
• High-Performance Computing: Solving scientific problems (like simulating weather or
DNA)
• Graphics and Rendering: Creating 3D animations or virtual reality scenes.
High-Performance Computing (HPC)
Instances
• These are specifically optimized for high-performance computing
workloads, providing powerful CPUs and networking capabilities for
complex simulations and scientific modeling.
AWS Pricing Models
AWS Pricing Models
Amazon EC2 (Elastic Compute Cloud) offers different ways to pay for virtual
servers, depending on how long, how often, or how flexibly you use them.
EC2 pricing models
• On-Demand Instances
• Reserved Instances (RIs)
• Spot Instances
• Savings Plans
• Dedicated Hosts
On-Demand Instances
Reserved Instances (RIs)
Spot Instances
Savings Plan
Dedicated Hosts
EC2 Instance Lifecycle
EC2 Instance Lifecycle
EBS-Backed instance
EBS-Backed instance
Summary of Instance States
AWS Auto Scaling
AWS Auto Scaling
AWS Auto Scaling
Scaling Policies
Benefits of AWS Auto Scaling
Elastic Load Balancing (ELB)
Elastic Load Balancing (ELB
• Elastic Load Balancer (ELB) is an AWS service that distributes incoming
traffic (like website visitors or app users) across multiple servers (like EC2
instances, containers, or Lambda functions) to ensure no single server gets
overloaded.
• ELB ensures that your application can handle an increasing amount of traffic
by automatically scaling resources up and down based on demand.
• It also checks the health of your registered targets and routes traffic only to
the healthy ones.
Key Features of Elastic Load Balancing
AWS Storage Services
AWS Storage Services
AWS Storage Services
AWS Storage Services
Amazon Simple Storage Service (S3)
• Amazon S3 is an object storage service that provides highly scalable,
durable, and secure storage for any amount of data.
• It is designed to store and retrieve any type of data, making it suitable for a
wide range of applications, from data lakes to backup and restore solutions.
• Each bucket can hold an unlimited number of objects, and the objects can
range in size from 0 bytes to 5 Terabytes.
• Buckets are globally unique, meaning that no two buckets can have the same
name across all AWS accounts.
AWS S3: Key Features
AWS S3
Structure of an S3 Bucket
S3 Buckets & Objects Limitations
How S3 Works
S3 Use Cases
S3 Storage Classes
S3 Storage Classes
S3 Standard
S3 Standard Limitations
S3 Standard-Infrequent Access (S3 Standard-IA)
S3 Standard-IA - Limitations
S3 Intelligent-Tiering
S3 Intelligent-Tiering - Limitations
S3 One Zone-Infrequent Access (S3 One Zone-
IA)
S3 One Zone-IA - Limitations
S3 Glacier Instant Retrieval - Limitations
S3 Glacier Flexible Retrieval
S3 Glacier Flexible Retrieval - Limitations
S3 Glacier Deep Archive
S3 Outposts
S3 Lifecycle Policies
S3 Lifecycle Policies
S3 Lifecycle Policies
Example
You have log files in a "logs/" folder that you want to:
• Move to Infrequent Access after 30 days
• Archive to Glacier after 90 days
• Delete after 365 days
Example
{ {
"Rules": [ "Days": 90,
{ "StorageClass": "GLACIER"
"ID": "LogFileRule", }
"Filter": { ],
"Prefix": "logs/" "Expiration": {
}, "Days": 365
"Status": "Enabled", }
}
"Transitions": [ ]
{ }
"Days": 30,
"StorageClass": "STANDARD_IA"
},
Benefits of Using Lifecycle Policies
Amazon EBS (Elastic Block Store)
Amazon EBS (Elastic Block Store)
• Amazon EBS is a block-level storage service that allows users to create
storage volumes and attach them to EC2 instances.
• Unlike traditional file storage, EBS volumes behave like raw, unformatted
block devices, which means they can be used to create file systems, run
databases, or store application data.
• EBS volumes are designed to be highly available and durable, ensuring that
data remains intact even when the associated EC2 instances are stopped or
terminated.
Amazon Elastic Block Store (EBS)
IOPS and Throughput
Input/Output Operations Per Second (IOPS)
Measures the speed and efficiency of storage devices based on their ability to
handle input/output operations.
Performance Indicator: A higher IOPS value indicates better performance and
faster data access, making it essential for data-intensive applications.
Throughput
Throughput measures the amount of data that can be processed by a storage system
in a given amount of time, typically expressed in bytes per second (Bps) or
megabytes per second (MB/s).
Throughput = (Read IOPS + Write IOPS ) × Block Size
Throughput is influenced by both IOPS and the size of the data blocks being processed.
EBS Volume Types
EBS Volume Types - General Purpose and
Provisioned IOPS SSD
Performance
Volume Type Best For Max Size Key Features
(IOPS/Throughput)
Baseline: 3,000 IOPS
- Boot volumes - Cheaper than gp2
gp3 (General Burst: Up to 16,000 IOPS
- Dev/Test 16 TB - Adjustable IOPS &
Purpose SSD) Throughput: 125–1,000
- Medium DBs throughput
MB/s
- Boot volumes 3 IOPS/GB - Being replaced
gp2 (Legacy) 16 TB
- Small DBs Burst: Up to 3,000 IOPS by gp3
- Critical DBs (Oracle, Up to 256,000 IOPS - High durability (io2)
io1/io2 (Provisioned
SAP) Throughput: Up to 4,000 16 TB - Guaranteed
IOPS SSD)
- Low-latency apps MB/s performance
HDD-Backed Volumes (Cheaper, for Large
Sequential Data)
Performance Max
Volume Type Best For Key Features
(Throughput) Size
- Big Data
(Hadoop, - Low cost,
st1 (Throughput
Kafka) Up to 500 MB/s 16 TB high
Optimized HDD)
- Log throughput
processing
- Rarely
- Cheapest,
sc1 (Cold HDD) accessed data Up to 250 MB/s 16 TB
slowest
- Archives
EBS Snapshots
EBS Pricing
Amazon Glacier
Amazon Glacier
• Amazon Glacier, is a low-cost cloud storage service specifically designed for
long-term data archiving and backup.
• It is ideal for storing "cold data," which refers to data that is infrequently
accessed but needs to be retained for future use.
• This makes it ideal for storing data that you don't need to access often but want
to keep for compliance, historical analysis, or disaster recovery purposes.
Amazon Glacier
• S3 Glacier offers three retrieval options:
• Expedited: Fastest option, retrieving data in 1-5 minutes, but most expensive.
• Standard: Default option, retrieving data in 3-5 hours.
• Bulk: Least expensive option, retrieving large amounts of data in 5-12 hours
AWS S3 Glacier
Data in S3 Glacier is organized into ”Archives" and ”Vaults.“
An archive is a single file or a collection of files that you store as a single unit.
- Basic units of storage in S3 Glacier.
- Each archive can contain any type of data (documents, photos, videos, etc.).
- Archives can be up to 40 terabytes in size.
- Archives are immutable—once stored, they cannot be modified.
- Each archive is identified by a unique ID assigned by AWS.
Amazon Glacier:
Containers for storing archives.
- Vaults help you organize and manage your archived data.
- You can create multiple vaults for different purposes (e.g., one for logs,
another for backups).
- Vaults support access policies to control who can access or modify them.
- You can enable notifications through Amazon SNS (Simple Notification
Service) to alert you when a retrieval job is completed.
Amazon Glacier:
• S3 Glacier integrates seamlessly with other AWS services.
• For Example, you can set up lifecycle policies in S3 to automatically move data to Glacier
after a certain period, helping you optimise storage costs.
• It also supports server-side encryption for data at rest and in transit, ensuring your archived
data remains secure.
• When using S3 Glacier, it's crucial to understand its pricing model.
• While storage is very cheap, retrieval can be costly if not managed properly.
• You're charged for storage, data transfer out, and retrieval requests.
• There's also a minimum storage duration charge of 90 days, meaning if you delete data
before 90 days, you'll still be charged for the full 90 days.
Amazon EFS (Elastic File System)
Amazon EFS (Elastic File System)
• Amazon EFS is a fully managed, scalable, and highly available file storage
service designed for use with Amazon EC2 instances and other AWS services.
• EFS is particularly beneficial for applications that require shared access to file
systems, allowing multiple Amazon Elastic Compute Cloud (EC2) instances to
access the same data simultaneously.
• It provides a simple, serverless, and elastic file system that automatically grows and
shrinks as you add and remove files, eliminating the need to provision and manage
capacity.
• This capability makes EFS ideal for a variety of workloads, including content
management systems, big data analytics, and web serving.
Amazon EFS (Elastic File System)
Amazon EFS (Elastic File System)
How Amazon EFS Works
Amazon EFS (Elastic File System)
Advantages and Limitations
Use Cases for Amazon EFS