Storage Systems

Overview of Physical Storage Media

Flash memory is a non-volatile storage medium that retains data even without power. It
is used in devices such as USB drives, SSDs, and mobile devices. Flash memory is faster
and more reliable than magnetic disks, but its higher cost per byte makes it less ideal for
large-scale storage, though it is increasingly replacing magnetic disks in many systems.

Magnetic disks, commonly known as hard disk drives (HDDs), are the primary medium
for long-term data storage. They are non-volatile, offering high capacities at a lower
cost compared to flash memory. Although slower in access speed, they remain a popular
choice for storing large databases and files due to their affordability and durability.
Overview of Physical Storage Media
Optical storage, such as DVDs and Blu-rays, is primarily used for backups and data distribution. With
capacities ranging from 27 GB to 128 GB, optical disks are affordable but slow, making them unsuitable
for active databases. They are often used for archival storage, with write-once and rewritable options
available for flexibility.

Tape storage is the most cost-effective medium for long-term archival storage. It offers enormous
capacity and durability, ideal for backups and preserving data for decades. However, its sequential access
method makes data retrieval slow, limiting its use to applications where quick access is not required.

These storage types form a hierarchy based on speed and cost, with cache and main memory at the top as
primary storage, flash and magnetic disks as secondary storage, and optical disks and tapes as tertiary
storage.
 Overview of Physical Storage Media

Fig: Storage
device hierarchy.
Magnetic Disks
● They are the main storage devices in computers.
● Their capacity keeps growing, but not fast enough for the massive data needs of modern
applications.
● They are cheaper and good for storing huge amounts of data that don’t need to be accessed
frequently, like videos and old user files.

Magnetic disks vs SSDs (Solid-State Drives)

● SSDs are much faster than magnetic disks and becoming more popular.
● Their size is increasing, and prices are coming down, but they are still 6–8
times more expensive per byte than magnetic disks.
● SSD used mainly for enterprise data and performance-focused applications.
● Magnetic disks are cheaper and better for storing large, less-used data.
● SSDs are faster but cost more, so they’re used for important and frequently accessed
data.
Magnetic Disks
Magnetic Disks
Structure of Magnetic Disks:
● Disk Platters: The disk has flat, circular plates made of metal or glass. These are coated with a
magnetic material that stores the data.
● Tracks and Sectors: The surface of the disk is divided into circular tracks, and each track is divided
into small sections called sectors. A sector is the smallest unit of data that can be read or written.
● Read-Write Heads: The read-write head is a small part that moves over the disk to read or write data.
It floats just above the disk surface to avoid touching it.

How It Works:
● Spinning: The disk spins at a high speed (e.g., 5400 to 10,000 rotations per minute).
● Reading/Writing Data: The read-write head moves to different tracks to read or write data.
● Data Storage: Data is stored as tiny magnetic changes on the disk surface.

Disk Controller:
● The disk controller helps the computer talk to the disk. It handles tasks like moving the head to the
right track, reading or writing data, and checking if the data is correct.
Magnetic Disks
Performance Measures of Disks:
1. Capacity
● Refers to the total amount of data the disk can store.

1. Access Time
The time from issuing a read/write request to the beginning of data transfer. It includes:
● Seek Time:
○ The time taken for the disk arm to move to the correct track.
○ Typical range: 2 to 20 milliseconds, depending on the distance.
○ Smaller disks usually have lower seek times due to reduced travel distance.
○ Average Seek Time:
■ Average of seek times for random requests.
■ Approximately half of the maximum seek time, ranging from 4 to 10
milliseconds.
Magnetic Disks
Performance Measures of Disks:
● Rotational Latency:
○ The time taken for the desired sector to rotate under the head.
○ Average latency is half the time for a full rotation.
○ Rotational speed:
■ 5400 to 15,000 RPM (90 to 250 rotations per second).
■ Latency ranges from 4 to 11.1 milliseconds.
● Access Time = Seek Time + Rotational Latency.
○ Average access times: 5 to 20 milliseconds.

3. Data-Transfer Rate
● The speed at which data is read or written.
● Maximum rates: 50 to 200 MB/s, depending on the disk model.
● Inner tracks have lower sustained transfer rates (e.g., a disk with a 100 MB/s maximum rate
may sustain only 30 MB/s).
Magnetic Disks
Performance Measures of Disks:
4. Sequential vs. Random Access
● Sequential Access:
○ Data requests occur in sequence (same or adjacent tracks).
○ Minimal seek time, higher data transfer rates.
● Random Access:
○ Data requests are spread randomly across the disk.
○ Each request involves a seek, resulting in lower transfer rates.
● IOPS (Input/Output Operations Per Second):
○ The number of random block accesses per second.
○ For 4 KB block size, current disks support 50 to 200 IOPS.
5. Mean Time to Failure (MTTF)
● A measure of disk reliability.
● Average continuous operation time before failure:
○ Vendor claims: 500,000 to 1,200,000 hours (57 to 136 years).
○ Real-world life span: ~5 years.
● Failure rates increase significantly as disks age beyond a few years.
RAID (Redundant Array of Independent Disks)

RAID (Redundant Array of Independent Disks) is a technique used to manage multiple

disks in a way that improves both performance and reliability when dealing with large
amounts of data.

Reliability refers to the ability of a system can store, process, and retrieve data without
errors, downtime, or loss of information, even in the event of unexpected failures.

Performance refers to how efficiently and quickly a system can process, store, and
retrieve data nd capacity to handle multiple operations simultaneously.
 RAID (Redundant Array of Independent Disks)

Improvement of Reliability via Redundancy

one disk has a smaller chance of failure, but having many disks means the risk of
failure across the system is much higher.

To prevent this, redundancy is introduced, meaning extra copies of the data are stored so
that if one disk fails, the data can be recovered from another disk.

One way to achieve is through mirroring, where every piece of data is written to two
disks. If one disk fails, the data is still available on the other disk, and no data is lost. This
method increases reliability

The mean time to failure of a mirrored system is much longer, meaning it takes much
more time for data loss to occur.
 RAID (Redundant Array of Independent Disks)

Improvement of Reliability via Redundancy

Mean time to data loss depends on mean time to failure, and mean time to repair.
Mean Time to Failure (MTTF): This is the average time it takes for a single disk to fail.
For example, if the MTTF of a disk is 100,000 hours, it means on average, the disk will
fail after 100,000 hours of use.
Mean Time to Repair (MTTR): This is the average time it takes to replace or repair a
failed disk. For example, if the repair time is 10 hours, this means it takes 10 hours on
average to fix or replace a disk after it fails.
 RAID (Redundant Array of Independent Disks)
Improvement of Reliability via Redundancy

Example of a Mirrored Pair of Disks:

● MTTF of a single disk = 100,000 hours

● MTTR (repair time) = 10 hours

For a mirrored pair of disks (where data is stored on two disks), the MTDL (time before data loss
occurs) is calculated as:

it would take 57,000 years for data loss to occur, assuming the failures are independent (i.e.,
the two disks fail at different times) and repairs happen quickly. This calculation shows
that mirrored disks significantly increase the reliability of data storage.
RAID (Redundant Array of Independent Disks)

Improvement in Performance via Parallelism

Parallel access to multiple disks helps improve performance by allowing data to be read or written faster.

1. Disk Mirroring (RAID 1) for Performance:

● With mirroring, data is copied to two disks, so read requests can be handled by either disk. This
doubles the read speed because the system can read from both disks at the same time, as long as
both are functional.

2. Data Striping:

● Striping splits data into smaller parts and spreads it across multiple disks.
○ Bit-level striping breaks down each byte into smaller parts and stores them on different disks.
However, this is rarely used because it doesn't improve read speed significantly.
○ Block-level striping divides data into blocks and spreads these blocks across multiple disks.
When reading large files, the system can fetch multiple blocks from different disks at once,
increasing read speed.
RAID (Redundant Array of Independent Disks)

Improvement in Performance via Parallelism

Goals of Parallelism in Disk Systems:

1. Improving Throughput: It balances the workload among disks to handle many small
data requests efficiently.
2. Reducing Response Time: It speeds up large data reads by allowing multiple disks
to work together.

In essence, parallel access to multiple disks improves both the speed of small data
accesses and the efficiency of large data reads.
RAID (Redundant Array of Independent Disks)

RAID levels:

1. RAID 0 (Striping):
● Data is split across multiple disks to improve performance (faster reads and writes).
● No redundancy, so if one disk fails, data is lost.
● Used for applications that need speed but can tolerate data loss.
1. RAID 1 (Mirroring):
● Data is duplicated on two or more disks. Each disk has an exact copy of the data.
● Provides high data reliability because data can be recovered from the mirrored disk
if one fails.
● Slower write speed because data is written to multiple disks, but read speeds are
improved as data can be read from either disk.
RAID (Redundant Array of Independent Disks)

RAID levels:
3. RAID 5 (Striping with Parity):
● Data is striped across multiple disks (like RAID 0), and parity information (extra
data used for recovery) is distributed among the disks.
● Can tolerate one disk failure without data loss. If a disk fails, the data can be
recovered using the parity block.
● Provides a balance between performance, capacity, and data protection.
4. RAID 6 (Double Parity):
● Similar to RAID 5, but with an extra parity block for added protection.
● Can tolerate two disk failures without data loss.
● Provides higher data reliability than RAID 5 but has slightly slower performance due
to the additional parity block.
RAID (Redundant Array of Independent Disks)

RAID levels:
5. RAID 10 (1+0):

a. Combines RAID 1 (mirroring) and RAID 0 (striping). Data is mirrored across pairs
of disks and then striped across multiple pairs.
b. Provides high performance and reliability, but it requires at least four disks.

Summary:
RAID levels offer different trade-offs between performance and data protection.

● RAID 0 offers speed.

● RAID 1 provides reliability.
● RAID 5 offers a balance.
● RAID 6 offers higher reliability with the ability to tolerate multiple failures.
● RAID 10 Provides high performance and reliability
RAID (Redundant Array of Independent Disks)

RAID levels:
RAID (Redundant Array of Independent Disks)
Hardware Issues:

Software RAID and Hardware RAID are methods of implementing RAID (Redundant
Array of Independent Disks).

Software RAID: Implemented purely through software, without requiring hardware

changes. It can be more flexible but requires more effort to handle issues like incomplete
writes after power failure.

Hardware RAID: Uses special hardware to support RAID operations, such as

non-volatile RAM to store write operations before they're completed. This helps prevent
data loss in case of power failure. It can also perform recovery operations more
efficiently than software RAID.
RAID (Redundant Array of Independent Disks)
Hardware Issues:

Advantages of Hardware RAID:

1. Power Failure Protection:

○ If the power goes out, hardware RAID can still remember any data that wasn’t fully
written to the disk and finish the process once the system is back on. This helps prevent data
loss.
2. Latent Failures (Bit Rot):
○ Sometimes, a part of the disk may have data that’s been written but is now unreadable due to
damage (bit rot). Hardware RAID can detect and fix such problems by "scrubbing" the data
— reading all sectors to find and repair any unreadable data.
3. Hot Swapping:
○ If a disk fails, you can replace it without turning off the system, which reduces downtime
and speeds up the repair process.
4. Reliability:
○ Hardware RAID systems often have backup power supplies and extra disk connections to
ensure that if one part fails, the system keeps working without interruption.
RAID (Redundant Array of Independent Disks)
Key Factors in Choosing a RAID Level:
Cost of Extra Storage

● The Financial cost of additional disks required for redundancy.

Performance Requirements
● The number of I/O operations per second needed for the application.
Performance During Disk Failure
● How well the system performs while a disk is failed and being recovered.
Rebuild Performance
● The speed and resource usage during the rebuilding of data on a new disk.
Application Needs
● The specific workload, such as read-intensive, write-intensive, or critical data safety
requirements.
RAID (Redundant Array of Independent Disks)
Key Factors in Choosing a RAID Level:

● RAID 0: If speed is your main concern and you don't need data protection (e.g., for
gaming or temporary files).
● RAID 1: If you need high data protection and can afford to double your storage (e.g.,
for storing important documents).
● RAID 5: If you want a good balance of speed, storage, and protection (e.g., for
general data storage).
● RAID 6: If you need extra data safety and are okay with spending more on storage
(e.g., for critical applications or databases).
Optimization of Disk-Block Access
1. Buffering:
● Temporarily store blocks read from the disk in an in-memory
buffer to satisfy future requests without accessing the disk
repeatedly.
2. Read-Ahead:
● Preload consecutive blocks from the disk into memory when a
block is accessed, anticipating sequential access and minimizing
time spent on seeks and rotational latency.
Optimization of Disk-Block Access
3. Disk-Arm Scheduling:

● Optimize the order of disk access requests to minimize disk-arm movement. The
elevator algorithm is a common approach, processing requests in a manner that
reduces seek time.
Optimization of Disk-Block Access
4. File Organization:

● Store blocks of files in a sequence that matches expected access patterns (e.g.,
sequentially on adjacent cylinders) to minimize the number of seeks.

5. Defragmentation:

● Reorganize files to ensure their blocks are stored contiguously on the disk, reducing
fragmentation and the need for frequent seeks.

6. Non-Volatile Write Buffers (NVRAM):

● Use NVRAM to temporarily hold writes, allowing for faster acknowledgment of

write operations. The disk controller later writes the data to disk in an optimized
order.
What is Database Failure?
Database failure refers to a situation in which a database is unable to function properly due to a hardware,
software, or database environment issue.

This can cause the database to become inaccessible or to fetch incorrect or incomplete data.

Database failure can be the result of malfunctioning hardware or software.

Database failures can have serious consequences, depending on the importance of the database and the
amount of data that is lost. To minimize the risk of database failure, it is important to regularly back up data.
What is Database Backup?

A database backup is a copy of a database that is created at a specific point in time. It is

typically used to protect against data loss due to hardware failure, software bugs,
human error, or other unforeseen events.

By having a backup of the database, you can restore the database to a previous state if
something goes wrong.
What is Database Recovery?
Database recovery is the process of restoring a database to a previous state after a failure or data
loss event. This can be done by using a database backup that was created before the mishap.

Log files are the most important element of a database. They keep a record of which file was
modified and when.

Database recovery can be:

Roll forward: Redo all records from a particular point in time.

Roll back: Undo all records from a particular point in time.
Importance of Database Backup and Recovery Mechanisms
1. Data Integrity and Consistency
● Ensures the database can be restored to a consistent state after failures, minimizing
the risk of data corruption or loss.

Ex: If a system crash happens after deducting the ₹1,000 but before adding it to the
checking account, the accounts are inconsistent

Recovery Ensures Consistency by Rollback or Undo the deduction from savings,

restoring the original balance.
Importance of Database Backup and Recovery Mechanisms
2. Protection Against Catastrophic Failures

● Handles major disasters such as hardware failures, disk crashes, or natural calamities
(floods, earthquakes) by restoring the database using backups.

3. Minimized Downtime

● Reduces downtime by enabling quick recovery of critical systems, ensuring business

continuity.

Ex: A hospital loses its main data center in a fire.

● Protection Against Catastrophic Failures: The database is restored using regular

offsite backups.
● Minimized Downtime: Recovery is completed quickly, allowing doctors to access
patient records and resume critical services without delays.
Importance of Database Backup and Recovery Mechanisms
4. Preservation of Transaction History

● Backed-up logs preserve the history of committed transactions, allowing recovery of

recent changes even after catastrophic failures.

Ex: In online banking, transaction logs ensure all deposits, withdrawals, and
transfers can be recovered, even after a system crash, preserving recent updates to
customer accounts.

5. Cost-Efficiency

● Avoids the high cost of rebuilding lost data or compensating for operational
disruptions.

6. Protection Against Human Errors

● Recovers data accidentally deleted or modified due to user mistakes or malicious actions.
Catastrophic failure
Catastrophic failure is a type of database failure that is severe and often results in the loss
of a significant amount of data.
Key consequences include:

1. Complete data inaccessibility: The system may become entirely unresponsive, halting all
database operations.
2. Potential data loss: Without robust backup and recovery mechanisms, substantial portions
of the database may be lost.
3. Significant recovery efforts: Restoring the system requires extensive planning, resources,
and time, often involving backups, logs, and sometimes the reconstruction of missing data.
Catastrophic failure
Hardware Failure: Sudden physical damage to critical components like disks or servers
can render the database unusable without immediate recovery measures.
Software Bugs or Corruption: Logical errors or corruptions can compromise the integrity
of the database, leading to significant operational disruptions.
Natural Disasters: Physical destruction of infrastructure causes a total or partial loss of
access to database systems, making recovery essential for continuity.
Human Errors: Mistakes like accidental deletions or misconfigurations can result in
irreparable data loss or operational issues, necessitating recovery plans.
Cyberattacks: These can lead to complete data compromise or ransom demands, affecting
data availability, integrity, and confidentiality.
How to Recover from Catastrophic Failure?
Catastrophic failure often results in the loss of a significant amount of data.

In these cases, it may not be possible to recover the database using traditional methods, such as restoring
from a backup or repairing the database.

If you are faced with a catastrophic failure, there are several steps you can take to try to recover the database:

1. Assess the damage: The first step in recovering from a catastrophic failure is to assess the damage and
determine the extent of the data loss. This will help you to understand what needs to be done to rebuild
the database.
2. Shut down the archiver: The archiver software might still be trying to add increments. Close it down
since the backend is not working.
3. Identify the point of failure: Ascertain which components have failed. It can be hardware, software,
and database environments. Repair and replace components as required.
How to Recover from Catastrophic Failure?
4. Gather data from other sources: If you do not have a recent, valid backup of the
database, you may need to gather data from other sources, such as spreadsheets or text
files, to rebuild the database.

5. Recreate the database structure: You will need to recreate the database structure,
including tables, indexes, and other objects, in order to rebuild the database.

6. Import the data: Once the database structure has been recreated, you can import the
data into the new database. This may involve cleaning and formatting the data to ensure
that it is compatible with the new database structure.