UNIT 2: DATA STORAGE AND DATA

AVAILABILITY
(08
)
 Syllabus

• Storage technology
- Hard Disk Device (HDD), Solid State Devices (SSD), memory devices
- Data access - block, files, object
- Data center End to End View – overview of complete stack including storage, network, host,cluster,
applications, virtual machines, cloud storage
- Storage virtualization technologies - RAID level, storage pooling, storage provisioning
- Advance topics in storage virtualization – storage provisioning, thin-provisioning
- Cloud storage – S3, glacier, storage tiering

• High Availability
- Introduction to high availability
- clustering, failover, parallel access
 • [Link]

• [Link]

• [Link]
 DATA STORAGE
• Files and documents are recorded digitally and saved in a storage system for future use.
Storage systems may rely on electromagnetic, optical or other media to preserve and
restore the data if needed.

• Data storage makes it easy to back up files for safekeeping and quick recovery in the event of
an unexpected computing crash or cyber attack.

• Data storage can occur on physical hard drives, disk drives, USB drives or virtually in the
cloud. The important thing is that your files are backed up and easily available should your
systems ever crash beyond repair.

• Some of the most important factors to consider in terms of data storage are reliability, how
robust the security features tend to be and the cost to implement and maintain the
infrastructure.

• Browsing through different data storage solutions and applications can help you arrive at the
choice that is the best fit for your business' needs.
 Types of Data Storage
Primary Storage Secondary Storage (Non-Volatile) Tertiary and Offline Storage Cloud Storage
(Volatile)
RAM (Random Hard Disk Solid State Optical Flash Drives Tape Drives: External Hard Public Cloud Private
Access Drives Drives Discs and Memory Drives Services Cloud
Memory) (HDD): (SSD Cards
Temporary Traditional Faster CDs, DVDs, Portable Used for Portable Services like A cloud
storage used magnetic storage and Blu-ray storage backup and versions of Google environme
by the storage with technoloy discs used devices using archival HDDs or SSDs Drive, nt
computer to large using flash for storing flash storage. It's used for Dropbox, operated
store data that capacities. memory. data, memory. cost-effective backup and and AWS S3 solely for a
is actively It's slower More media, and Common for for large additional offer storage single
being used or than SSD expensive software. transferring volumes of storage. over the organizatio
but cheaper than HDD They are data data that do internet. n
processed. It is
per GB. but less between not need to
fast but loses
provides common devices and be accessed Data is It can be
all data when quicker for regular for additional frequently. stored in hosted
power is lost. data data storage on remote internally
access and storage devices like servers or by a
better due to cameras and maintained third-party
performan limited smart by third provider.
ce. capacity phones. parties
and slower
access
times.
 Data Availability
• Data availability is the process of ensuring that data is available to end-users and
applications, when and where they need it.

•Availability has to do with the accessibility and continuity of information, thus

accessibility is a key component.

•It defines the degree or extent to which data is readily usable along with the
necessary IT and management procedures, tools, and technologies required to
enable, manage and continue to make data available.
 STORAGE TECHNOLOGY
• Hard Disk Device (HDD)
1. Serial Advanced Technology Attachment (Sata)
2. Small Computer System Interface(SCSI)
3. Serial Attached SCSI(SAS)

• Solid State Devices (SSD)

 HARD DISK DEVICE (HDD)
• The HDD was introduced in the year 1956 by IBM.
• HDDs store information magnetically in tracks on rotating disks.
HDDs use heads to read and write data onto the disks.
• The first personal computer contains a hard drive of less than 1
megabyte, while modern computers contain the hard drive of 1
terabyte.
• The desktop computers which have external hard drives are used
for backup purposes or additional storage.
• HDD is an electro-mechanical storage device, which is an
abbreviation of Hard Disk Drive.
• It uses magnetic storage for storing and retrieving the digital data.
It is a non-volatile storage device.
• Hard Disk Drive is installed internally in our computer systems,
which is connected directly to the disk controllers of the
COMPONENTS OF HDD motherboard.
An HDD consists of several components:
the read/write head • Hard Disk Drive is a storage device which stores the operating
actuator arm, system (OS), installed software, and the other computer files.
spindle motor,
platter, and enclosure
WORKING OF HARD DISK DRIVE
• Spindle Motor: The motor that spins the platters at a
constant speed.
• Platters: The disks coated with magnetic material where
data is stored.
• Read/Write Head: The component that reads data from
and writes data to the platters.
• VCM Actuator: Voice coil motor actuator, which moves the
read/write head to the correct position on the platter.
• IDE Connector: Interface for connecting the hard drive to
the computer.
• Pivot: The pivot point for the actuator arm.
• Base Plate: The base structure of the HDD that holds all the
components.
 disCharacteristics of Hard Disk Drives
Storage Capacity
High Capacity: HDDs typically offer larger storage capacities compared to SSDs at a lower cost per
gigabyte, making them ideal for bulk storage.
Performance
Read/Write Speed: HDDs generally have slower read/write speeds compared to SSDs due to the
mechanical nature of their operation. Typical speeds range from 100 to 200 MB/s.
Latency: Higher latency compared to SSDs because the read/write heads need to physically move to the
correct position on the platters.
Durability and Reliability
Mechanical Parts: Contains moving parts (spindle, platters, actuator arm), which are more susceptible to
wear and tear and physical damage from shocks or drops.
MTBF (Mean Time Between Failures): Typically lower than that of SSDs due to mechanical wear and
potential for failure.
Form Factor
Standard Sizes: Common form factors include 3.5-inch for desktop computers and 2.5-inch for laptops.
Weight: Generally heavier than SSDs due to the mechanical components.
Energy Consumption
Power Usage: Higher power consumption compared to SSDs, especially during start-up and active use, due
to the need to spin the platters.
Heat Generation
Thermal Output: HDDs tend to generate more heat compared to SSDs because of the friction and
movement of mechanical parts.
 disCharacteristics of Hard Disk Drives
Cost
Cost-Effective: Generally cheaper per gigabyte than SSDs, making them a cost-effective solution for large-scale storage
needs.

Interface
Connectivity: Common interfaces include SATA (Serial ATA) and SAS (Serial Attached SCSI), with SATA being the most
prevalent in consumer applications.

Noise
Operational Noise: Produces noise due to the spinning of platters and movement of the read/write head, which can be
noticeable in quiet environments.

Data Density
Areal Density: High areal density allows for large amounts of data to be stored on a single platter.

Lifespan
Lifespan Variability: The lifespan of an HDD can vary significantly based on usage patterns, environmental conditions,
and manufacturing quality.

Usage Scenarios
Ideal for: Bulk storage, archival purposes, and applications where cost is a more significant factor than speed, such as in
servers, desktops, and external storage solutions.
 THREE TYPES OF HARD DISK?
• Hard Disk Drives (HDDs) come in several types, differentiated by factors
like their interface, form factor, and usage. Here are three common types:
1. SATA Hard Drives(Serial Advanced Technology Attachment )

2. SAS (Serial Attached SCSI(Small Computer System Interface) Hard

Drives

3. Hybrid Drives (SSHD - Solid State Hybrid Drives)

 1. SATA (Serial ATA) Hard Drives

• Description: SATA is the most common interface used in consumer HDDs. It has
largely replaced the older PATA (Parallel ATA) standard.
• Advantages:
– Speed: Offers sufficient speed for everyday computing needs (up to 600 MB/s
with SATA III).
– Compatibility: Widely compatible with most desktop and laptop computers.
– Cost-Effective: Generally cheaper than SSDs and other types of HDDs.
• Usage: Commonly used in desktops, laptops, and external drives.
 2. SAS (Serial Attached SCSI) Hard Drives

• Description: SAS drives are designed for high-performance and enterprise

environments. They offer better speed and reliability compared to SATA drives.
• As a result, SSDs offer advantages such as reduced boot times, quicker file
• Advantages:
transfers, and advanced system overall performance.
– Speed: Higher data transfer rates (up to 12 Gbps with SAS-3).
• Activities that used to require numerous minutes on an HDD, which include
– Reliability: Built forin-depth
launching resource- 24/7 operation, often
applications used in servers
or searching for files, and databe
can now centers.
carried
out within seconds
– Scalability: withmultiple
Supports the speeddevices
and efficiency
on the of an SSD.
same bus, useful in enterprise
• environments.
These storage devices store the data in the semiconductor cells. It uses electric
charges
• Usage: to
Typically used in servers, enterprise storage systems, and high-performance
represent binary information.
computing environments.
• This technology enables more compact and lightweight storage solutions, making
SSDs suitable for various applications such as laptops, desktop computers, game
consoles, and data centers.
 3. Hybrid Drives (SSHD - Solid State Hybrid Drives)

• Description: Hybrid drives combine a traditional spinning hard drive with a small
amount of solid-state storage (SSD) to improve performance.
• Advantages:
– Speed: Faster than standard HDDs due to the SSD component, which caches
frequently accessed data.
– Capacity: Offers large storage capacities like traditional HDDs.
– Cost-Effective: More affordable than full SSDs while providing some
performance benefits.
• Usage: Suitable for users who need a balance between speed and capacity, such as
in gaming laptops and desktop computers.
 Solid State Devices (SSD)

A Solid-State Drive (SSD) is a type of storage device that uses flash memory to store data, offering
faster performance compared to traditional Hard Disk Drives (HDDs).
 HOW DOES SSD WORK?
1. Flash Memory (NAND Flash):
• Storage: SSDs use NAND flash memory, a type of non-volatile memory, meaning it retains data even when power is turned off.
• Cells: Data is stored in memory cells, which are grouped into pages. Pages are further grouped into blocks. The more bits stored per cell, the
higher the density but the slower the performance.
• Types of NAND:
– SLC (Single-Level Cell): Stores 1 bit per cell (fastest, most durable, expensive).
– MLC (Multi-Level Cell): Stores 2 bits per cell.
– TLC (Triple-Level Cell): Stores 3 bits per cell.
– QLC (Quad-Level Cell): Stores 4 bits per cell (highest density, least durable, cheapest).

2. Controller:
• The SSD’s controller is like a processor that manages data on the drive. It handles reading, writing, garbage collection, wear leveling, and error
correction.
• Wear leveling: Flash memory wears out over time due to repeated writes. The controller spreads out the data writes across different memory
cells to extend the drive's lifespan.
• Garbage collection: SSDs cannot directly overwrite existing data, so when data is deleted, the drive marks those blocks as "invalid" and will later
erase them in a process called garbage collection, making space available for new data.
 HOW DOES SSD WORK?
3. Reading and Writing Data:
• Reading: When data is read from the SSD, the controller identifies the location of the data and retrieves it from the NAND cells, offering almost
instant access due to the lack of moving parts.
• Writing: When writing, the controller writes new data to free pages, but if pages need to be rewritten (updated), the entire block containing
those pages must first be erased before new data can be written. This is why SSDs can slow down when handling a lot of writes.

4. No Moving Parts:
• Unlike HDDs, SSDs have no mechanical parts. This gives SSDs several advantages:
 Speed: Much faster read/write speeds than HDDs.
 Durability: More resistant to physical shock.
 Noise: Completely silent operation.
 Energy Efficiency: Uses less power, which is particularly beneficial for laptops.

5. NVMe (Non-Volatile Memory Express):

• Many modern SSDs use the NVMe protocol over the PCIe interface, allowing for even faster data transfer rates compared to older SATA-based
SSDs. NVMe takes advantage of the parallelism of flash memory and offers significantly lower latency and higher throughput.
 HOW DOES SSD WORK?
6. TRIM Command:

• The TRIM command helps improve SSD performance and longevity. When data is deleted, TRIM informs the SSD
that those blocks are no longer in use, allowing the SSD to manage the space more efficiently.

• In summary, SSDs store data using flash memory, which provides fast access to information. The controller
manages all data operations, and the lack of mechanical parts results in superior performance, durability, and
energy efficiency compared to traditional HDDs.
 TYPES OF SSD
SATA SSD (Serial ATA) NVMe SSD (Non- M.2 SSD U.2 SSD External SSD
Volatile Memory
Express)
Interface Uses the SATA Uses the PCIe interface, Can use either SATA or Uses the PCIe Can use USB (Type-C,
interface, which is the allowing for much faster NVMe protocols. interface, like NVMe. Type-A), Thunderbolt,
same used by data transfer rates than or even SATA
• SATAHDDs.
traditional SSDs SATA. interfaces.

Speed: • PCIearound
Typically SSDs:
500- Can exceed 3,000 MB/s Depends on whether it uses Similar to NVMe M.2 Varies depending on
600 MB/s or more, depending on SATA or PCIe; NVMe-based drives. the connection type,
• NVMe the number of PCIe
lanes used.
M.2 drives are significantly
faster
with Thunderbolt
being the fastest
SSDs:
Form 2.5-inch, mSATA (for M.2 or U.2. A small, rectangular form Larger than M.2, Portable and rugged
Factor laptops and ultra- factor, often used in thin often used in designs.
• M.2 devices).
portable SSDs: laptops and compact enterprise settings.
devices
Usage •Common
U.2 SSDs:
in consumer Popular in high- Ideal for ultra-thin devices Typically found in Used for portable
laptops and desktops, performance systems, and high-performance servers and data storage, backups, and
• External
offering a balance gaming PCs, and laptops. centers due to its as an external drive
between performance workstations due to its robustness and hot- for laptops and
SSDs:
and cost. speed advantage. swapping capability. desktops.
 THE INTERNAL STRUCTURE OF AN SSD INCLUDES SEVERAL KEY COMPONENTS:
• The internal structure of a Solid-State Drive (SSD) consists of several key components that work together to store and retrieve
data. Here’s a breakdown of these components:

1. NAND Flash Memory:

Purpose: This is the primary storage medium of the SSD, where all data is stored. NAND flash memory is a type of non-volatile
memory, meaning it retains data even when the power is turned off.
Structure: It consists of memory cells, which store data in binary form (1s and 0s). These cells are grouped into pages, and
pages are grouped into blocks.

2. SSD Controller:
Purpose: The controller is essentially the brain of the SSD. It manages data storage and retrieval, oversees wear leveling,
garbage collection, and error correction, and handles the communication between the SSD and the rest of the system.
Components: The controller contains its own processor (often ARM-based), firmware, and algorithms to manage the various
processes and ensure efficient operation.

3. DRAM Cache:
Purpose: Some SSDs include a DRAM cache, which is a temporary storage area used to speed up data access. When data is
written to the SSD, it is often first placed in the cache to improve performance before being written to the NAND memory.
Function: The cache helps reduce latency and improve the overall speed of read and write operations. However, not all SSDs
have DRAM caches—some lower-cost models may rely on a portion of the NAND flash as a substitute.
 THE INTERNAL STRUCTURE OF AN SSD INCLUDES SEVERAL KEY COMPONENTS:
4. Interface:
Purpose: The interface is how the SSD connects to the rest of the system. It determines the speed and type of data transfer.
Types:
SATA: A common interface used by many SSDs, similar to the one used by traditional hard drives.
PCIe/NVMe: A newer, faster interface that allows much higher data transfer rates compared to SATA.
M.2: A form factor and interface that can support both SATA and PCIe/NVMe protocols, often found in laptops and compact
devices.

5. Power Loss Protection Circuitry:

Purpose: Some SSDs include power loss protection circuitry to prevent data corruption in case of a sudden power loss.
Function: These components ensure that any data in transit is safely written to the NAND flash before the power is completely
lost. This is particularly important in enterprise and critical data storage applications.

6. Firmware:
Purpose: The SSD’s firmware is software embedded in the controller that manages the operation of the SSD, including data
management, error correction, and wear leveling.
Function: Firmware updates can improve the performance and reliability of the SSD by optimizing how it manages data and
interacts with the host system.

7. Temperature Sensors:
Purpose: High-end SSDs may include temperature sensors to monitor the drive’s temperature and prevent overheating.
Function: These sensors can trigger thermal throttling, reducing the drive's performance temporarily to prevent damage from
excessive heat.
 THE INTERNAL STRUCTURE OF AN SSD INCLUDES SEVERAL KEY COMPONENTS:

8. Capacitors (in some models):

Purpose: Capacitors can be included in some enterprise-grade SSDs to provide a temporary power supply in case of power
loss, allowing the SSD to save any data that was in the process of being written.
Function: This feature is critical for preventing data loss in environments where data integrity is essential, such as in servers and
data centers.

9. PCB (Printed Circuit Board):

Purpose: The PCB is the board that holds all of the SSD’s components together, including the NAND flash chips, controller, and
other components.
Design: It provides the electrical connections between
 Key SSD Components and Their Functions:

• Controller: The controller acts as the brain of the SSD, managing information flow between the flash memory chips and
the host device. It handles tasks that include error correction and garbage collection (reclaiming unused or deleted
information).
• Interface: The interface connects the SSD to the host machine and permits data transfer. Common interfaces include
Serial ATA (SATA), widely used in client SSDs, and PCIe (Peripheral Component Interconnect Express), which gives faster
speeds and is generally discovered in high- performance SSDs.
• When you keep information on an SSD, the controller divides it into small blocks and stores them within the flash
memory cells. Each cell can keep more than one bit of data, and the type of flash memory (like a single-level or multi-
level cell) relies upon how many bits it can keep.
• To read the data, the controller finds the right cells using the storage addresses given by the computer. It measures the
electric charges in the cells to figure out the data values. That is much quicker than the moving parts in HDDs so SSDs can
access data quickly.
• Erasing data works differently for SSDs compared to HDDs. Rather than overwriting data directly, SSDs want to erase
entire blocks before new data can be stored. The controller manages this method called "garbage collection."
• Because SSDs use flash memory and don't have any moving components, they may be great for many uses, from
personal computers to large data centers. They offer each speed and reliability, which are important in those
applications.
 MEMORY DEVICES

1. RAM
2. ROM
1. ROM
 1. RAM

• A RAM constitutes the internal memory of the CPU for storing data, program and program result.

• It is read/write memory. It is called random access memory (RAM).

• Since access time in RAM is independent of the address to the word that is, each storage location inside
the memory is as easy to reach as other location & takes the same amount of time. We can reach into the
memory at random & extremely fast but can also be quite expensive.

• RAM is volatile, i.e. data stored in it is lost when we switch off the computer or if there is a power failure.
Hence, a backup uninterruptible power system (UPS) is often used with computers.

• RAM is small, both in terms of its physical size and in the amount of data it can hold.

• RAM is of two types

• Static RAM (SRAM)

• Dynamic RAM (DRAM)

• Static RAM (SRAM)
• The word static indicates that the memory retains its contents as long as power remains applied.
However, data is lost when the power gets down due to volatile nature. SRAM chips use a matrix of 6-
transistors and no capacitors. Transistors do not require power to prevent leakage, so SRAM need not
have to be refreshed on a regular basis.
• Because of the extra space in the matrix, SRAM uses more chips than DRAM for the same amount of
storage space, thus making the manufacturing costs higher.
• Static RAM is used as cache memory needs to be very fast and small.

Dynamic RAM (DRAM)

• DRAM, unlike SRAM, must be continually refreshed in order for it to maintain the data. This is done by
placing the memory on a refresh circuit that rewrites the data several hundred times per second. DRAM
is used for most system memory because it is cheap and small.
• All DRAMs are made up of memory cells.
• These cells are composed of one capacitor and one transistor.
 2. ROM
• ROM stands for Read Only Memory. The memory from which we can only read but cannot
write on it. This type of memory is non-volatile. The information is stored permanently in
such memories during manufacture.
• A ROM, stores such instruction as are required to start computer when electricity is first
turned on, this operation is referred to as bootstrap.
• ROM chip are not only used in the computer but also in other electronic items like washing
machine and microwave oven.
• Following are the various types of ROM −
• MROM (Masked ROM)
• PROM (Programmable Read Only Memory)
• EPROM (Erasable and Programmable Read Only Memory)
• EEPROM (Electrically Erasable and Programmable Read Only Memory)
• MROM (Masked ROM)
• The very first ROMs were hard-wired devices that contained a pre-
programmed set of data or instructions. These kind of ROMs are known
as masked ROMs. It is inexpensive ROM.
• PROM (Programmable Read Only Memory)
• PROM is read-only memory that can be modified only once by a user.
The user buys a blank PROM and enters the desired contents using a
PROM programmer. Inside the PROM chip there are small fuses which
are burnt open during programming. It can be programmed only once
and is not erasable.
• EPROM (Erasable and Programmable Read Only Memory)
• The EPROM can be erased by exposing it to ultra-violet light for a duration of upto 40 minutes.
Usually, an EPROM eraser achieves this function. During programming an electrical charge is
trapped in an insulated gate region. The charge is retained for more than ten years because the
charge has no leakage path. For erasing this charge, ultra-violet light is passed through a
quartz crystal window (lid). This exposure to ultra-violet light dissipates the charge. During
normal use the quartz lid is sealed with a sticker.
• EEPROM (Electrically Erasable and Programmable Read Only Memory)
• The EEPROM is programmed and erased electrically. It can be erased and reprogrammed
about ten thousand times. Both erasing and programming take about 4 to 10 ms (millisecond).
In EEPROM, any location can be selectively erased and programmed. EEPROMs can be
erased one byte at a time, rather than erasing the entire chip. Hence, the process of re-
programming is flexible but slow.
 Data Access
The three primary types of data access methods are
1. Block storage
2. File storage
3. Object storage

These represent different approaches to managing and storing data, often used in cloud storage solutions
and networked storage systems.
• Object storage stores and manages all data in an unstructured format and in units called objects.
• Block storage takes any data, like a file or database entry and divides it into blocks of equal sizes. It then
stores the data block on underlying physical storage in a way that's optimized for fast access and retrieval..
• [Link]
 Three primary types of data access methods
•Sequential Access
•Data is accessed in a specific linear order.
•Commonly used in tape storage.
•To reach a specific piece of data, all preceding data must be read.
•Suitable for tasks like batch processing.

•Direct Access (or Random Access)

•Data can be read or written in any order.
•Used in hard disks, SSDs, and RAM.
•Offers faster retrieval as it doesn’t require reading data
sequentially.

•Indexed Access
•Uses an index to locate and access data.
•Combines features of sequential and direct access.
•Often used in databases and indexed files for efficient lookups.
Three File Storage Methods
 File Storage
• Description: Data is stored and accessed as complete files.

• Used In: Hard drives, USBs, NAS (Network Attached Storage).

• File Systems: NTFS (Windows), FAT32, ext4 (Linux), HFS+ (Mac).

• Advantages:
- Easy to use and understand.
- Suitable for documents, media files, etc.

• Disadvantages:
- Slower access speed compared to block storage.
- Not ideal for large databases.

• Use file storage for basic file management.

 File Storage
• Description:
– File storage manages data as a hierarchy of files and directories.
– Each file is accessed using a file path and the storage system manages the metadata for these files.
– Data is accessed in a specific linear order
– Commonly used in tape storage
– To reach a specific piece of data all preceding data must be read .
– Suitable for tasks like batch processing.
• Characteristics:
– Ease of Use: Familiar and easy to use for individuals and applications.
– Metadata Management: Manages file metadata, making it easy to organize and search.
– Network Accessibility: Commonly used in network-attached storage (NAS) systems, making it easy to share files across a network.
• Use Cases:
– File sharing and collaboration
– Home directories
– Content management
• Examples:
– NAS (Network Attached Storage)
– NFS (Network File System)
– SMB/CIFS (Server Message Block/Common Internet File System)
 NAS (Network Attached Storage)
• It is a file-based storage architecture that
makes stored data more accessible to
networked devices means multiple users or
client devices retrieve data from a single
storage system.
• As the multiple clients or users connected
through a Local Area Network access the data
from a centralized disk capacity by ethernet
that’s why it is referred to as Network
Attached Storage.
• Network Attached Storage(NAS) gives all the
connected devices of the network a single
access point for storage which is called NAS
Storage Server.
NFS (Network File System)
• The advent of distributed computing was marked by
the introduction of distributed file systems.
• Such systems involved multiple client machines and
one or a few servers.
• The server stores data on its disks and the clients may
request data through some protocol messages.

Advantages of a distributed file system:

• Allows easy sharing of data among clients.
• Provides centralized administration.
• Provides security, i.e. one must only secure the
servers to secure data.
 SMB/CIFS (Server Message Block/Common Internet File
System)
• The Common Internet File System (CIFS), additionally
called Server Message Block (SMB), is a community file-
sharing protocol used for gaining access to documents,
printers, and different assets on a community.
• CIFS is a file-sharing protocol that works in the
Application and Presentation Layer. It is mainly used to
provide shared access to all the local systems to the
remote files or other services like printing remotely, it
also provides access to the interprocess
communication services, without being available in the
same network.
 Block Storage

• Description:

• Data is stored in blocks. The system manages and assembles them to create files.
Used In: SAN (Storage Area Networks), high-performance databases.

• Advantages:
- Faster performance.
- More control over how data is accessed and managed.

• Disadvantages:
- Complex to manage.
- Requires specialized hardware or configuration.

• Use block storage for high-speed applications like databases.

 Block Storage
• Description:
– In block storage, data is stored in fixed-sized blocks. Each block is given a unique identifier and the
storage system retrieves these blocks without any file structure or metadata.
• Characteristics:
– High Performance: Optimized for low latency and high throughput.
– Flexibility: Allows for high-performance applications such as databases and virtual machine file systems.
– Scalability: Scales well with performance but can be complex to manage.
• Use Cases:
– Databases
– Virtual machines
– Transactional applications
• Examples:
– SAN (Storage Area Network)
– iSCSI (Internet Small Computer Systems Interface)
– Fibre Channel
 Storage Area Network (SAN).
• A dedicated, fast network that gives storage devices network access is
called a Storage Area Network (SAN).
• SANs are generally made up of several technologies, topologies, and
protocols that are used to connect hosts, switches, storage elements,
and storage devices.
• SANs can cover several locations.
• Data transfer between the server and storage device is the primary goal
of SAN.
• Additionally, it makes data transmission across storage systems
possible. Storage area networks are primarily used to connect servers to
storage devices including disk-based storage and tape libraries.
Fiber Channel (FC):
• A Fiber Channel is one of the maximum broadly used SAN storage connections.
• It presents excessive-velocity, low-latency connectivity between servers and
storage devices with the use of fiber optic cables.
• Fiber Channel helps factor-to-factor, arbitrated loop, and switched fabric
topologies.
• It gives excessive throughput, reliability, and scalability, making it suitable for
traumatic enterprise environments.
• Internet Small Computer System Interface(iSCSI):
• iSCSI is a storage protocol that transmits SCSI commands over TCP/IP networks,
permitting servers to get the right of entry to faraway storage devices using
fashionable Ethernet connections.
• ISCSI offers a value-effective alternative to Fibre Channel, leveraging current
Ethernet infrastructure and TCP/IP networks. It presents features such as block-
level garage access, multipathing, and CHAP authentication.
 Fibre Channel over Ethernet (FCoE):
• Fibre Channel over Ethernet encapsulates Fibre Channel frames into Ethernet packets,
allowing Fibre Channel site visitors to be transmitted over Ethernet networks.
• FCoE enables the convergence of storage and data networks, lowering infrastructure
complexity and fees.
• It leverages Ethernet’s sizable adoption and familiarity at the same time as preserving
Fibre Channel’s overall performance characteristics

Serial Attached SCSI(SAS):

• Serial Attached SCSI is a factor-to-point garage protocol designed to attach servers to
garage gadgets using high-pace serial connections.
• SAS gives overall performance akin to Fibre Channel but with less difficult cabling and
decrease expenses.
• It helps direct-connected garage (DAS) and may be used in SAN environments with SAS
switches or routers.
 Object Storage
• Description:
– Object storage stores data as objects.
– Each object contains the data itself, metadata, and a unique identifier. It is designed for storing large
amounts of unstructured data.
• Characteristics:
– Scalability: Can scale to handle massive amounts of data across distributed systems.
– Resilience: Built-in data replication and integrity checking.
– Metadata-Rich: Each object can have extensive metadata, making it easier to manage and retrieve
large datasets.
• Use Cases:
– Cloud storage
– Backup and archival
– Multimedia files (images, videos)
• Examples:
– Amazon S3 (Simple Storage Service)
– Open Stack Swift
– Google Cloud Storage
 Object Storage

• Description:

• Data is stored as objects that include the file, metadata, and a unique ID.
Used In: Cloud storage (e.g., Amazon S3, Google Cloud Storage).

• Advantages:
- Highly scalable.
- Great for unstructured data (videos, images, backups).
- Metadata makes searching and management easier.

• Disadvantages:
- Slower for frequent updates or file editing.
- Not suitable for traditional file systems.

• Use object storage for scalable cloud-based storage

 Amazon s3
• Amazon S3 stores data as objects within buckets.
• An object is a file and any metadata that describes the file.
• A bucket is a container for objects.
• To store your data in Amazon S3, you first create a bucket and specify a bucket name and AWS Region.
• Then, you upload your data to that bucket as objects in Amazon S3.
• Each object has a key (or key name), which is the unique identifier for the object within the bucket.
• S3 provides features that you can configure to support your specific use case.

• For example, you can use S3 Versioning to keep multiple versions of an object in the same bucket,
which allows you to restore objects that are accidentally deleted or overwritten. Buckets and the
objects in them are private and can only be accessed with explicitly granted access permissions. You
can use bucket policies, AWS Identity and Access Management (IAM) policies, S3 Access Points, and
access control lists (ACLs) to manage access.
Amazon S3
 Open Stack Swift
• The OpenStack Object Store project, known as Swift, offers cloud storage software
so that you can store and retrieve lots of data with a simple API.
• It's built for scale and optimized for durability, availability, and concurrency across
the entire data set.
• Swift is ideal for storing unstructured data that can grow without bound.
• Open Stack Swift is a distributed object storage system designed to scale from a
single machine to thousands of servers.
• It is optimized for multi-tenancy and high concurrency, and is ideal for storing
unstructured data that can grow without bound.
• Swift is used for redundant, scalable data storage using clusters of standardized
servers to store peta bytes of accessible data
 Google Cloud Storage

• Google Cloud Storage presents a variety of solutions tailored

to diverse data needs. Ranging from Machine Learning (ML) to
data analytics, each service relies fundamentally on storage
options, a vital component of Cloud Computing.
• Storage is key in facilitating effective data management and
accessibility both for individual applications and enterprise-
scale operations.
Comparison of Data Access Methods
 DATA CENTER END TO END
• A data center is a physical facility that organizations use to house their critical
applications and data.
• A data center's design is based on a network of computing and storage resources
that enable the delivery of shared applications and data.
• The key components of a data center design include routers, switches, firewalls,
storage systems, servers, and application-delivery controllers.
WHY ARE DATA CENTERS IMPORTANT TO BUSINESS?

• In the world of enterprise IT, data centers are designed to support

business applications and activities that include:
[Link] and file sharing
[Link] applications
[Link] relationship management (CRM)
[Link] resource planning (ERP) and databases
[Link] data, artificial intelligence and machine learning
[Link] desktops, communications and collaboration services
WHAT ARE THE CORE COMPONENTS OF A DATA
CENTER?
• Data center design includes routers, switches, firewalls, storage systems, servers,
and application delivery controllers.

• Because these components store and manage business-critical data and

applications, data center security is critical in data center design. Together, they
provide:
• Network infrastructure. This connects servers (physical and virtualized), data center
services, storage, and external connectivity to end-user locations.
• Storage infrastructure. Data is the fuel of the modern data center. Storage systems
are used to hold this valuable commodity.
• Computing resources. Applications are the engines of a data center. These servers
provide the processing, memory, local storage, and network connectivity that drive
applications.
WHAT IS IN A DATA CENTER FACILITY?

• Data center components require significant infrastructure to

support the center's hardware and software.
• These include power subsystems, uninterruptible power
supplies (UPS), ventilation, cooling systems, fire suppression,
backup generators, and connections to external networks
 WHAT ARE THE STANDARDS FOR DATA CENTER
INFRASTRUCTURE?
• The most widely adopted standard for data center design and data center infrastructure is. ANSI/TIA-942
(Telecommunication infrastructure standard for data center).

• It includes standards for ANSI/TIA-942-ready certification, which ensures compliance with one of four categories
of data center tiers rated for levels of redundancy and fault tolerance.

[Link] 1: Basic site infrastructure. A Tier 1 data center offers limited protection against physical events. It has
single-capacity components and a single, nonredundant distribution path.
[Link] 2: Redundant-capacity component site infrastructure. This data center offers improved protection
against physical events. It has redundant-capacity components and a single, nonredundant distribution path.
[Link] 3: Concurrently maintainable site infrastructure. This data center protects against virtually all physical
events, providing redundant-capacity components and multiple independent distribution paths. Each component
can be removed or replaced without disrupting services to end users.
[Link] 4: Fault-tolerant site infrastructure. This data center provides the highest levels of fault tolerance and
redundancy. Redundant-capacity components and multiple independent distribution paths enable concurrent
maintainability and one fault anywhere in the installation without causing downtime.
 DATA CENTER NEW DESIGN:
• It is also known as a spine-leaf design. It uses massive switches which are in the distribution
layer. (e.g., Cisco Nexus class switches) The distribution layer switches are known as spine
nodes.
• The access layer switches are known as leaf nodes. It’s now predictable how much hope a
packet will take. Takes less time to reach a packet from server to server. Too many Fiber-
optic cables. Handles north-south traffic as well as east-west traffic.
• Spine switches are connected with all the leaf switches in mesh
topology while the leaf switches are connected to the server
and connect end users.
• Spine switches handle layer 3 network traffic whereas leaf
switches can handle both layer 2 and layer 3 network traffic.
• Traditional three-tier architecture has three layers and spine-
leaf architecture has two layers.
• In the traditional three-tier architecture, the east-west traffic is
more than the north-south traffic while in spine-leaf
architecture east-west traffic flows from server to server within
the same data center.
• Access switches directly connect to the server in spine-leaf
architecture.
• Spine-leaf architecture distribution switches provide redundant
connections to access switches. Spine leaf architecture
provides fast data transfer between the spine switches and leaf
switches.
TYPES OF DATA CENTERS :
There are four primary types of data centers.
Which are :
[Link] and Corporate Data Center
[Link] Data Center
[Link] Data Center
[Link] Data Center
VIEW – OVERVIEW OF COMPLETE
STACK INCLUDING STORAGE
• Stack storage is the storage provided by Language Environment that
is needed for routine linkage and any automatic storage. It is a
contiguous area of storage obtained directly from the operating
system. Stack storage is automatically provided at thread initialization.
• A storage stack is a data structure that supports procedure or block
invocation (call and return). It is used to provide both the storage
required for the application initialization and any automatic storage
used by the called routine. Each thread has a separate and distinct
stack.
• The storage stack is divided into smaller segments called stack frames, which are
also known as dynamic storage areas (DSAs). A stack frame, or DSA, is
dynamically acquired storage composed of a register save area and an area
available for dynamic storage allocation for items such as program variables.
• Stack frames are added to the user stack when a routine is entered, and removed
upon exit in a last in, first out (LIFO) manner. Stack frame storage is acquired during
the execution of a program and is allocated every time a procedure, function, or
block is entered, as, for example, when a call is made to a Language Environment
callable service, and is freed when the procedure or block returns control
• The stack is allocated as one large area of contiguous storage, the size of
which is specified by the maximum size parameter of the STACK64
runtime option. Only a portion of the stack is initially available to the
application, the amount specified in the initial size parameter of the
STACK64 runtime option. The rest of the stack storage is "guarded,"
which prevents the application from storing into it.
• When the initial stack area becomes full, a stack increment is created by the
operating system by un guarding additional storage contiguous to the
currently available stack area. The amount of storage to unguard is
specified by the increment size parameter of the STACK64 runtime
option.
 NETWORK

• A network is a collection of computers, servers, mainframes, network

devices, peripherals, or other devices connected to allow data
sharing.
• An example of a network is the Internet, which connects millions of
people all over the world. To the right is an example image of a home
network with multiple computers and other network devices all
connected.
 HOST

• What is a host?
• A host is a computer or other device that communicates with other hosts on
a network. Also known as network hosts, hosts include clients and
servers that send or receive data, services and applications.
• Hosts typically do not include intermediary network devices like switches
and routers, which are instead often categorized as nodes. A node is a
broader term that includes anything connected to a network, while a host
requires an IP address. In other words, all hosts are nodes, but network
nodes are not hosts unless they require an IP address to function.
• Hosts use various protocols to communicate, including TCP and User
Datagram Protocol (UDP).

• On a TCP/IP network, each host has a host number that, together

with a network identity, forms its unique IP address.
• In the Open Systems Interconnection (OSI) model, protocols in the
transport layer, also known as Layer 4, are responsible for
communication between hosts.
 CLUSTER
• A computer cluster is a set of computers that work together so that they can be viewed as a single system.
Unlike grid
computers, computer clusters have each node set to perform the same task, controlled and scheduled by
software.
• The components of a cluster are usually connected to each other through fast local area networks, with each
node (computer used as a server) running its own instance of an operating system. In most circumstances, all
of the nodes use the same hardware {better source needed] and the same operating system, although in some
setups (e.g. using Open Source Cluster Application Resources (OSCAR)), different operating systems can be
used on each computer, or different hardware.
• Clusters are usually deployed to improve performance and availability over that of a single computer, while
typically being much more cost-effective than single computers of comparable speed or availability.
• Computer clusters emerged as a result of the convergence of a number of computing trends including the
availability of low- cost microprocessors, high-speed networks, and software for high-performance distributed
computing. They have a wide range of applicability and deployment, ranging from small business clusters with a
handful of nodes to some of the fastest supercomputers in the world such as IBM's Sequoia. Prior to the advent
of clusters, single unit fault tolerant mainframes with modular redundancy were employed; but the lower upfront
cost of clusters, and increased speed of network fabric has favoured the adoption of clusters. In contrast to
high-reliability mainframes clusters are cheaper to scale out, but also have increased complexity in error
handling, as in clusters error modes are not opaque to running programs
 VIRTUAL MACHINES

• Virtual Machine (VM) is a compute resource that uses software instead of a physical
computer to run programs and deploy apps.
• One or more virtual “guest” machines run on a physical “host” machine.
• Each virtual machine runs its own operating system and functions separately from the
other VMs, even when they are all running on the same host.
• This means that, for example, a virtual MacOS virtual machine can run on a physical
PC.
• Virtual machine technology is used for many use cases across on-premises and cloud
environments. More recently, public cloud services are using virtual machines to
provide virtual application resources to multiple users at once, for even more
cost efficient and flexible compute.
WHAT ARE VIRTUAL MACHINES USED FOR?

• Virtual machines (VMs) allow a business to run an operating system that behaves
like a completely separate computer in an app window on a desktop. VMs may be
deployed to accommodate different levels of processing power needs, to run
software that requires a different operating system, or to test applications in a
safe, sandboxed environment.
• Virtual machines have historically been used for server virtualization, which
enables IT teams to consolidate their computing resources and improve efficiency.
Additionally,
virtual machines can perform specific tasks considered too risky to carry out in a
host
environment, such as accessing virus-infected data or testing operating systems.
Since the virtual machine is separated from the rest of the system, the software
inside the virtual machine cannot tamper with the host computer.
• How do virtual machines work?
• The virtual machine runs as a process in an application window,
similar to any
other application, on the operating system of the physical machine.
• Key files that make up a virtual machine include a log file, NVRAM
setting file, virtual disk file and configuration file.
 CLOUD STORAGE

• Cloud storage allows you to save data and files in an off-site location that you
access either through the public internet or a dedicated private network connection.
• Data that you transfer off-site for storage becomes the responsibility of a third-party
cloud provider. The provider hosts, secures, manages, and maintains the servers
and associated infrastructure and ensures you have access to the data whenever
you need it.
• Cloud storage delivers a cost-effective, scalable alternative to storing files on on-
premise hard drives or storage networks. Computer hard drives can only store a
finite amount of data. When users run out of storage, they need to transfer files to
an external storage device.
• You pay for only exactly the capacity you use.
• How does cloud storage work?
• Like on-premise storage networks, cloud storage uses servers to
save data; however, the data is sent to servers at an off-site location.
Most of the servers you use are virtual machines hosted on a
physical server. As your storage needs increase, the provider creates
new virtual servers to meet demand.
 CLOUD STORAGE IS AVAILABLE IN
PRIVATE, PUBLIC AND HYBRID CLOUDS.
• Public storage clouds: In this model, you connect over the internet to a storage cloud that’s maintained by a
cloud provider and used by other companies. Providers typically make services accessible from just about any
device, including smartphones and desktops and let you scale up and down as needed.
• Private cloud storage: Private cloud storage setups typically replicate the cloud model, but they reside within
your network, leveraging a physical server to create instances of virtual servers to increase capacity. You can
choose to take full control of an on-premise private cloud or engage a cloud storage provider to build a
dedicated private cloud that you can access with a private connection. Organizations that might prefer private
cloud storage include banks or retail companies due to the private nature of the data they process and store.
• Hybrid cloud storage: This model combines elements of private and public clouds, giving organizations a
choice of which data to store in which cloud. For instance, highly regulated data subject to strict archiving
and replication requirements is usually more suited to a private cloud environment, whereas less
sensitive data (such as email that doesn’t contain business
secrets) can be stored in the public cloud. Some organizations use hybrid clouds to supplement their internal
storage networks with public cloud storage.
 STORAGE VIRTUALIZATION TECHNOLOGIES
• Storage virtualization is the pooling of physical storage from multiple storage devices into what
appears to be a single storage device -- or pool of available storage capacity. A central console
manages the storage.
• The technology relies on software to identify available storage capacity from physical devices and to
then aggregate that capacity as a pool of storage that can be used by traditional architecture servers
or in a virtual environment by virtual machines (VMs).

• RAID level
• storage pooling
• storage provisioning
 RAID( R edundant Array of Independent Disks)

• [Link]

• RAID or redundant array of independent disks is a data storage virtualization

technology that combines multiple physical disk drive components into one or more
logical units for data redundancy, performance improvement, or both.
• It is a way of storing the same data in different places on multiple hard disks or
solid-state drives to protect data in the case of a drive failure. A RAID system
consists of two or more drives working in parallel. These can be hard discs, but
there is a trend to use SSD technology (Solid State Drives).
• RAID combines several independent and relatively small disks into single storage of
a large size. The disks included in the array are called array members. The disks
can combine into the array in different ways, which are known as RAID levels.
 HOW RAID WORKS
• RAID works by placing data on multiple disks and allowing input/output operations to overlap in a balanced way,
improving performance. Because various disks increase the mean time between failures (MTBF), storing data
redundantly also increases fault tolerance.
• RAID arrays appear to the operating system as a single logical drive. RAID employs the techniques of disk mirroring
or disk striping.
• Disk Mirroring will copy identical data onto more than one drive.
• Disk Striping partitions help spread data over multiple disk drives. Each drive's storage space is divided into units
ranging from 512 bytes up to several megabytes. The stripes of all the disks are interleaved and addressed in order.
• Disk mirroring and disk striping can also be combined in a RAID array.
• In a single-user system where significant records are stored, the stripes are typically set up to be small (512 bytes) so
that a single
record spans all the disks and can be accessed quickly by reading all the disks at the same time.
• In a multi-user system, better performance requires a stripe wide enough to hold the typical or maximum size record,
allowing overlapped disk I/O across drives.
 LEVELS OF RAID

• Many different ways of distributing data have been standardized into various RAID
levels. Each RAID level is offering a trade-off of data protection, system
performance, and storage space. The number of levels has been broken into three
categories, standard, nested, and non-standard RAID levels.
• Standards RAID Levels
• Nested RAID levels
• Non-standard RAID levels

•
 RAID Level Difference
RAID 0 (Striping) RAID 1 (Mirroring) RAID 5 (Striping RAID 6 (Striping with RAID 10 (1+0, RAID 50 and
with Parity) Dual Parity) Mirroring and RAID 60
Striping)
Configuration Combines RAID 1 Combines RAID 5/6
(mirroring) and RAID 0 with RAID 0, creating
(striping). Data is a striped array of
mirrored and then RAID 5/6 arrays.
striped across drives.

Benefits: High performance and Higher performance

redundancy. Can and fault tolerance
tolerate multiple drive than RAID 5 or 6
failures as long as one alone
drive in each mirrored
pair remains functional.

Drawbacks Requires at least four Requires more disks

drives and only offers and is complex to
50% storage efficiency. implement

Best for High-performance Enterprise-level

systems that need both applications with high
speed and redundancy, performance and
like databases or web redundancy
servers. requirements.

Summary Speed but no High redundancy, no Balance of speed, Greater redundancy than Combines speed and
redundancy. speed improvement. storage, and RAID 5. redundancy, but with
redundancy. lower storage
efficiency.
• 1. RAID 0 (striped disks)
• RAID 0 is taking any number of disks and merging them into
one large volume. It will increase speeds as you're reading
and writing from multiple disks at a time. But all data on all
disks is lost if any one disk fails. An individual file can then
use the speed and capacity of all the drives of the array.
• The downside to RAID 0, though, is that it is NOT redundant.
The loss of any individual disk will cause complete data loss.
This RAID type is very much less reliable than having a
single disk
• There is rarely a situation where you should use RAID 0 in a
server environment. You can use it for cache or other
purposes where speed is essential, and reliability or data
loss does not matter at all.
• 2. RAID 1 (mirrored disks)
• It duplicates data across two disks in the array, providing full redundancy. Both disks
are store exactly the same data, at the same time, and at all times. Data is not lost as
long as one disk survives. The total capacity of the array equals the capacity of the
smallest disk in the array. At any given instant, the contents of both disks in the array
are identical.
• AID 1 is capable of a much more complicated configuration. The point of RAID 1 is
primarily for redundancy. If you completely lose a drive, you can still stay up and
running off the other drive.
• Either drive fails, you can then replace the broken drive with little to no downtime. RAID
1 also gives you the additional benefit of increased read performance, as data can read
off any of the drives in the array. The downsides are that you will have slightly higher
write latency. Since the data needs to be written to both drives in the array, you'll only
have a single drive's available capacity while needing two drives.
• 3. RAID 5(striped disks with single parity)
• RAID 5 requires the use of at least three drives.
It combines these disks to protect data
against loss of any one disk; the array's
storage capacity is reduced by one disk. It
strips data across multiple drives to increase
performance. But, it also adds the aspect of
redundancy by distributing parity information
across the disks.
•
• 4. RAID 6 (Striped disks with double parity)
• RAID 6 is similar to RAID 5, but the parity data are written to two
drives. The use of additional parity enables the array to continue to
function even if two disks fail simultaneously. However, this extra
protection comes at a cost. RAID 6 has a slower write performance
than RAID 5.
• The chances that two drives break down at the same moment are
minimal. However, if a drive in a RAID 5 system died and was
replaced by a new drive, it takes a lot of time to rebuild the swapped
drive. If another drive dies during that time, you still lose all of your
data. With RAID 6, the RAID array will even survive that second
failure also.
• Nested RAID levels
• Some RAID levels are referred to as nested RAID because they
are based on a combination of RAID levels, such as:
• 1. RAID 10 (1+0)
• This level Combines RAID 1 and RAID 0 in a single system, which
offers higher performance than RAID 1, but at a much higher cost.
• This is a nested or hybrid RAID configuration. It provides security by
mirroring all data on secondary drives while using striping across each set of
drives to speed up data transfers.
 • 2. RAID 01 (0+1)
• RAID 0+1 is similar to RAID 1+0, except the data organization method is slightly different.
Rather than creating a mirror and then striping the mirror, RAID 0+1 creates a stripe set and
then mirrors the stripe set.
• 3. RAID 03 (0+3, also known as RAID 53 or RAID 5+3)
• This level uses striping similar to RAID 0 for RAID 3's virtual disk blocks. This offers higher
performance than RAID 3 but at a higher cost.
• 4. RAID 50 (5+0)
• This configuration combines RAID 5 distributed parity with RAID 0 striping to improve RAID 5
performance without reducing data protection.
 WHAT IS A STORAGE POOL?

• A storage pool is capacity aggregated from disparate physical storage resources in a shared storage environment.

• IT administrators can configure storage pools in various sizes. Storage pools can provide a number of benefits in areas such as performance, management and data
protection.

• Admins can provision pools to include any amount of capacity and use any combination of physical storage space in a storage area network. In virtual server
environments, admins can use dedicated pools to store virtual machines to ensure critical VMs can access the proper amount of storage.

• Vendors often provide the capability for admins to provision storage pools through a management interface. For example, the Storage Spaces feature in Microsoft
Windows Server provides pooling capabilities. Tools such as IBM Tivoli Storage Manager and Google Cloud Backup and Disaster Recovery Service also enable this type
of provisioning.

• A storage pool is a collection of storage volumes. A storage volume is the basic unit of storage, such as allocated space on a disk or a single tape cartridge. The server
uses the storage volumes to store backed-up, archived, or space-managed files.

• The server provides three types of storage pools that serve different purposes: primary storage pools, copy storage pools, and active-data pools. You can arrange
primary storage pools in a storage hierarchy. The group of storage pools that you set up for the Tivoli® Storage Manager server to use is called server storage.

• Primary storage pools

• When a user tries to restore, retrieve, recall, or export file data, the requested file is obtained from a primary storage po ol, if possible. Primary storage pool volumes are
always located onsite.

• Copy storage pools

• Copy storage pools contain active and inactive versions of data that is backed up from primary storage pools. Copy storage pools provide a means of recovering from
disasters or media failures.

• Active-data pools

• An active-data pool contains only active versions of client backup data. Active-data pools are useful for fast client restores, reducing the number of onsite or offsite
O ADVANCE TOPICS IN STORAGE VIRTUALIZATION – STORAGE
PROVISIONING

• Storage Provisioning
• What Does Storage Provisioning Mean?
• Storage provisioning is the process of assigning storage capacity to servers, computers, virtual machines or
any other computing device. A broad term, storage provisioning incorporates manual and automated
mechanisms used to allocate server storage space within a networked computing environment.
• Storage provisioning is implemented in a computing environment where the core storage resides in a storage
area network (SAN) server. Storage may be provisioned to all connected devices – either manually, by the
storage administrator, or automatically on runtime and on demand through a SAN software appliance.
• The storage provisioning process requires careful planning on the front end. This helps the administrator or
SAN software predict future storage requirements and optimize the underlying network for faster and more
reliable data storage and retrieval.
• Storage provisioning may be classified as fat provisioning or thin provisioning.
 CLOUD STORAGE

• S3
• Glacier
• storage
tiering
 CLOUD STORAGE – S3,
• What is Amazon S3?
• Amazon Simple Storage Service (Amazon S3) is a scalable, high-speed, web-based cloud storage service. The service is
designed for online backup and archiving of data and applications on Amazon Web Services (AWS). Amazon S3 was designed
with a minimal feature set and created to make web-scale computing easier for developers.
• Amazon S3 features
• S3 provides 99.999999999% durability for objects stored in the service and supports multiple security and compliance
certifications. An administrator can also link S3 to other AWS security and monitoring services, including CloudTrail, CloudWatch
and Macie. There's also an extensive partner network of vendors that link their services directly to S3.
• Data can be transferred to S3 over the public internet via access to S3 application programming interfaces (APIs). There's also
Amazon S3 Transfer Acceleration for faster movement over long distances, as well as AWS Direct Connect for a private,
consistent connection between S3 and an enterprise's own data center. An administrator can also use AWS Snowball, a physical
transfer device, to ship large amounts of data from an enterprise data center directly to AWS, which will then upload it to S3.
• In addition, users can integrate other AWS services with S3. For example, an analyst can query data directly on S3 either with
Amazon Athena for ad hoc queries or with Amazon Redshift Spectrum for more complex analyses.
• Amazon S3 use cases
• Amazon S3 can be used by organizations ranging in size from small businesses to large enterprises.
S3's scalability, availability, security and performance capabilities make it suitable for a variety of data
storage use cases. Common use cases for S3 include the following:
• data storage;
• data archiving;
• application hosting for deployment, installation and management of web apps;
• software delivery;
• data backup;
• disaster recovery (DR);
• running big data analytics tools on stored data;
• data lakes;
• mobile applications;
• internet of things (IoT) devices;
• media hosting for images, videos and music files; and
• website hosting -- particularly well suited to work with Amazon CloudFront for content delivery.
• How Amazon S3 works
• Amazon S3 is an object storage service, which differs from other types of cloud
computing storage types, such as block and file storage. Each object is stored as a
file with its metadata included. The object is also given an ID number. Applications
use this ID number to access objects. This is unlike file and block cloud storage,
where a developer can access an object via a representational state transfer
(REST) API.
• The S3 object storage cloud service gives a subscriber access to the same systems
that Amazon uses to run its own websites. S3 enables customers to upload, store
and download practically any file or object that is up to 5 terabytes (TB) in size --
with the largest single upload capped at 5 gigabytes (GB).
 GLACER What is AMAZON Glacier?

• Amazon Glacier, also known as Amazon Simple Storage Service (S3) Glacier, is a low-cost cloud storage service for data with longer retrieval times offered by Amazon Web
Services (AWS).

• Amazon Glacier provides storage for data archiving and backup of cold data. Cold data refers to files that are infrequently accessed but are kept in case they are needed at a
later date. A developer will use a cold data service such as Amazon Glacier to move data that is not needed often to archival storage to save money on storage costs. Developers
can also move database backups from tape storage media to the cloud for long-term Glacier storage. Storing cold data this way offloads the burden on IT teams of managing
archival data and backups.

• Amazon advertises Amazon Glacier as an extremely low-cost storage service. To keep costs low, the service is optimized for infrequently accessed data -- where data retrieval
times can reach from three to five hours.

• AWS also manages the operational heavy lifting required for data retention. Customers can store small or large amounts of data with Amazon Glacier. After setting the service up
with the Amazon Web Services Management Console, users upload whatever data they choose.

• When an organization wants to retrieve data, it has three options:

• Bulk retrievals, which recover large amounts of data in a five- to 12-hour window; this is the lowest cost option.
• Standard retrievals, which take about three to five hours to recover stored data.
• Expedited retrievals, which can return data within five minutes. This option, which is the most costly, is designed for organizations that may need their stored data at any time.
• Additionally, Amazon considers Amazon Glacier an official part of Amazon Simple Storage Service, which is why it is also called S3 Glacier.
 What are the uses for Amazon S3 GLACER?
• Amazon designed S3 Glacier as a cloud-based alternative to on-premises magnetic tape backup drives. Although tape libraries can lower storage costs,
they still require upfront investments and maintenance. Amazon's Glacier service enables an organization to bypass the management and cost
investments involved in tape backup drives by instead using a low-cost cloud service.
• S3 Glacier is for long-term storage, so users can retain data for years. For example, S3 Glacier is a good option for organizations that need to reference
a set of data only once or twice a year, or those that want to use the data for backup.
• S3 Glacier is used in other scenarios and fields, such as:
• Regulatory or compliance archiving. Glacier enables financial services and other industries to keep data for regulatory and compliance archives over
extended periods of time.
• Healthcare archival data. Glacier can help hospitals meet regulatory requirements by archiving patient record data securely and cost-effectively.
• Digital preservation. Government agencies or libraries can use Glacier for digital preservation efforts.
• Media asset storage. Media files such as video tend to take up a lot of space when stored over time. Glacier provides a way to store media files
affordably.
• Where Amazon Glacier stores data: Archives and vaults
• Amazon Glacier stores data in archives and vaults. An archive is a block of data that may consist of a single file or aggregated data
• -- commonly in the form of a Tape Archive or ZIP file. Glacier archives range in size from 1 byte to 40 terabytes (TB). However, there are no limits to
how much data and how many archives an AWS user can store in Glacier. Amazon offers a multipart upload feature for higher throughput and
reliability for archives with object sizes of over 100 MB.
 What is the need of Amazon Glacier

• Amazon Glacier offers the following benefits:

• Lower cost. Glacier is designed as Amazon's lowest-cost storage class. This enables an organization to store large amounts of data at a reduced
cost compared to other Amazon storage services.
• Maintains archival database. An organization does not have to maintain its own archival database. AWS handles administrative tasks such as
capacity planning and hardware
• Durability. Glacier is distributed across at least three physical AWS Availability Zones at a time, increasing the ability to restore data if it is lost in
one zone.
• Scalability. Organizations can scale the stored data up or down as needed.
• Multiple data retrieval options. Organizations can choose from expedited, standard and bulk retrievals.
• Security. S3 Glacier supports security and compliance standards such as General Data Protection Regulation, Payment Card Industry Data Security
Standard, Health Insurance Portability and Accountability Act and Federal Information Security Management Act. It also supports encryption and
monitors storage application programming interface call activities.
• Integration. Glacier integrates with other AWS offerings such as AWS Snowball and AWS Direct Connect.
 TIERED STORAGE
WHAT IS TIERED STORAGE?

• Tiered storage is a method for assigning different categories of data to various types of storage media to reduce overall storage costs and improve the
performance and availability of mission-critical applications. A tiered storage architecture categorizes data hierarchically based on its business value,
with data ranked by how often it's accessed by users and applications. The data is then assigned to specific storage tiers that are defined by their
performance, availability and media costs.

• Generally, the most important data is served from the fastest storage media, which is typically the most expensive. In a basic configuration, mission-
critical data might be assigned to a high-performing tier that consists of flash solid-state drives (SSDs) and Intel Optane memory modules, while less
critical data is written to a second tier made up only of hard disk drives (HDDs). A third tier might then be used for archiving data that needs to be kept
indefinitely, with the data stored on tape drives or cloud storage platforms.

• Tiering is one link in a chain of activities governed by information lifecycle management (ILM).
• What is multi-tiered storage?
• A tiered approach to data management utilizes different types of storage media to create multiple tiers for accommodating different types of data. The
exact approach that organizations take to tiering depends on their specific storage, data and application requirements. Today's IT teams might support
anywhere between two and five tiers, sometimes even more.

• To a large degree, the number of tiers will depend on how an organization classifies its data. For example, business data is often grouped into one of
four categories: mission critical, hot data, warm data and cold data. Based on these categories, an organization might implement four storage tiers -- Tier
0, Tier 1, Tier 2, Tier 3 -- with Tier 0 supporting mission-critical workloads and Tier 3 storing cold data.

• Organizations aren't locked into this structure -- they might deploy more tiers or fewer tiers -- but this approach represents the basic principles that go
into a multi-tiered storage architecture.
 HIGH AVAILABILITY

Introduction to high availability

• High availability (HA) is the ability of a system to operate continuously without failing for a designated period of time. HA works to
ensure a system meets an agreed-upon operational performance level. In information technology (IT), a widely held but difficult-to-
achieve standard of availability is known as five-nines availability, which means the system or product is available 99.999% of the
time.
• HA systems are used in situations and industries where it is critical the system remains operational. Real-world high-availability
systems include military control, autonomous vehicle, industrial and healthcare systems. People's lives depend on these systems
being available and functioning at all times. For example, if the system operating an autonomous vehicle fails to function when the
vehicle is in operation, it could cause an accident, endangering its passengers, other drivers and vehicles, pedestrians and
property.
• Highly available systems must be well-designed and thoroughly tested before they are used. Planning for one of these systems
requires all components meet the desired availability standard. Data backup and failover capabilities play important roles in
ensuring HA systems meet their availability goals. System designers must also pay close attention to the data storage and access
technology they use.
• How does high availability work?
• It is impossible for systems to be available 100% of the time, so true high-availability systems generally strive for five nines
as the standard of operational performance.
• The following three principles are used when designing HA systems to ensure high availability:
• Single points of failure. A single point of failure is a component that would cause the whole system to fail if it fails. If a
business has one server running an application, that server is a single point of failure. Should that server fail, the application
will be unavailable.
• Reliable crossover. Building redundancy into these systems is also important. Redundancy enables a backup component to
take over for a failed one. When this happens, it's necessary to ensure reliable crossover or failover, which is the act of
switching from component X to component Y without losing data or affecting performance.
• Failure detectability. Failures must be visible and, ideally, systems have built-in automation to handle the failure on their
own. There should also be built-in mechanisms for avoiding common cause failures, where two or more systems or
components fail simultaneously, likely from the same cause.
• To ensure high availability when many users access a system, load balancing becomes necessary. Load balancing
automatically distributes workloads to system resources, such as sending different requests for data to different services
hosted in a hybrid cloud architecture. The load balancer decides which system resource is most capable of efficiently
handling which workload. The use of multiple load balancers to do this ensures no one resource is overwhelmed.
• The servers in an HA system are in clusters and organized in a tiered architecture to respond to requests from load
balancers. If one server in the cluster fails, a replicated server in another cluster can handle the workload designated for the
failed server. This sort of redundancy enables failover where a secondary component takes over a primary component's job
when the first component fails, with minimal performance impact.
• The more complex a system is, the more difficult it is to ensure high availability because there are simply more points of
failure in a complex system.
 CLUSTER FAIL OVER
WHAT IS FAILOVER ?

• In computing, a failover cluster refers to a group of independent servers that work together to maintain high availability of applications and
services. If one of the servers fails, another node in the cluster can take over its workload with little or no downtime. This process is known as
failover.
• In a failover cluster, the clustered servers can collectively improve the availability and scalability of applications and services. The servers work
together to provide either continuous availability or at least high availability (HA) through the failover process. Failover clusters can comprise
either physical servers or virtual machines (VMs).
• Each server is called a node and is connected to other servers in the cluster by physical cables and software. A failover cluster consists of at least
two nodes to transfer data, as well as software to process the data through the cables. In addition, a cluster uses one or more technologies for
load balancing, storage, parallel processing and other functions.
• The applications and services in a failover cluster are sometimes known as clustered roles. The cluster keeps these roles operational if one server
fails. At the same time, each role is proactively monitored to ensure that it is working properly. If it's not, it might be restarted or the role moved
to another node.
• The need for failover clusters
• Server failure can lead to application downtime, which can result in operational disruptions for
users. By providing continuous availability or high availability, failover clusters enable users to keep
using the applications and services they need without experiencing outages, even if a server fails.
There might be a brief service interruption with high-availability clusters during the failover process.
However, the system usually recovers quickly with little or no data loss and minimum downtime.
• Failover clusters play a vital role to ensure the ongoing availability of mission-critical applications
and systems such as online transaction processing systems that demand very high -- near 100% --
availability. Database replication and disaster recovery also require failover clusters. These
clusters provide geographic replication so if a server in one location goes down, data will still be
available on failover servers at other sites.
• How failover clusters work
• In a high-availability failover cluster, groups of independent servers share data and resources, including storage. At any time, a
least one node is active and at least one is passive. These clusters include a monitoring connection that allows each server to
check the health of the other servers. In a two-node cluster (the simplest possible configuration), if a node fails the other node will
recognize the failure via the monitoring connection and configure itself as the active node. Larger configurations usually use
dedicated servers that determine if any nodes are failing and then direct another node to assume the load and participate in the
failover process.
• In high-availability failover clusters using VMs, the VMs are pooled into a cluster along with the physical servers they reside on. If
there is a failure, the VMs on the failed host will be restarted on alternate hosts.
• In a continuous-availability failover cluster -- also known as a fault-tolerant cluster -- multiple systems share one copy of a
computer's operating system. Thus, the commands issued by one system are simultaneously executed on the other systems as
well. The cluster requires a continuously available and near-exact copy of a machine running the service (physical or virtual). This
redundancy model is known as 2N. It can automatically detect failures of hard drives, power supplies, network components and
CPUs. When the cluster identifies a failure point, a backup component -- or procedure -- takes its place instantaneously without
service interruption.
 Disaster recovery
• Disaster recovery (DR) is an organization's ability to respond to and recover from an event that negatively affects
business operations.
• The goal of DR methods is to enable the organization to regain use of critical systems and IT infrastructure as
soon as possible after a disaster occurs.
• To prepare for this, organizations often perform an in-depth analysis of their systems and create a formal
document to follow in times of crisis. This document is known as a disaster recovery plan.

What is a disaster?
• The practice of DR revolves around events that are serious in nature. These events are often thought of in terms
of natural disasters, but they can also be caused by systems or technical failure or by humans carrying out an
intentional attack. They are significant enough to disrupt or completely stop critical business operations for a
period of time. Types of disaster include:
• Cyber-attacks such as malware, DDoS and ransomware
attacks
• Sabotage
• Power outages
• Equipment failure
• Epidemics or pandemics, such as COVID-19
• Terrorist attacks or threats
• Industrial accidents
• Hurricanes
• Tornadoes
• Earthquakes
• Floods
• Fires
 WHY IS DISASTER RECOVERY IMPORTANT?
• Disasters can inflict many types of damage with varying levels of severity, depending on the scenario. A brief network outage
could result in frustrated customers and some loss of business to an e-commerce system. A hurricane or tornado could destroy an
entire manufacturing facility, data center or office.
• The monetary costs can be significant. The Uptime Institute's Annual Outage Analysis 2021 report estimated that 40% of outages
or service interruptions in businesses cost between $100,000 and $1 million, while about 17% cost more than $1 million. A data
breach can be more expensive; the average cost in 2020 was $3.86 million, according to the 2020 Cost of a Data Breach Report by
IBM and the Ponemon Institute.
• Additionally, many businesses are required to create and follow plans for disaster recovery, business continuity and data
protection in order to meet compliance regulations. This is particularly important for organizations operating in financial,
healthcare, manufacturing and government sectors. Failure to have DR procedures in place can result in legal or regulatory
penalties, so understanding how to comply with resiliency standards is important.
• Preparing for every potential disaster may seem extreme, but the COVID-19 crisis illustrated that even scenarios that seem
farfetched can come to
• pass. Businesses that had emergency measures in place to support remote work had a clear advantage when stay-at-home orders
were enacted.
• Thinking about disasters before they happen and creating a plan for how to respond can provide many benefits. It raises
awareness about potential disruptions and helps an organization to prioritize its mission-critical functions. It also provides a forum
for discussing these topics and making careful decisions about how to best respond in a low-pressure setting.