100% found this document useful (1 vote)
93 views45 pages

Overview of Object Storage Systems

The document discusses the evolution from block and file-based data storage to object storage. It covers the basics of block-based storage, file-based storage, and object-based storage. Object storage uses a flat structure rather than a hierarchical directory structure, and stores metadata with each object. Key benefits of object storage include scalability, durability, lower costs at large scale, and simplicity over traditional storage systems.

Uploaded by

SaravanaRaajaa
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
100% found this document useful (1 vote)
93 views45 pages

Overview of Object Storage Systems

The document discusses the evolution from block and file-based data storage to object storage. It covers the basics of block-based storage, file-based storage, and object-based storage. Object storage uses a flat structure rather than a hierarchical directory structure, and stores metadata with each object. Key benefits of object storage include scalability, durability, lower costs at large scale, and simplicity over traditional storage systems.

Uploaded by

SaravanaRaajaa
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
  • Object Storage Introduction
  • Storage Evolution
  • Data Access Methods
  • Block Paradigm
  • File Systems
  • Object Paradigm
  • Comparison and Benefits
  • Object Types and Characteristics
  • Challenges and Applications
  • Advanced Storage Solutions
  • Conclusion

Object Storage

Manas Minglani
Wednesday, 2015

The Storage Evolution: From Blocks, Files and


Objects to Object Storage Systems
Structure Data vs Unstructured

Topics
Block-Based Data Access
File-Based Data Access
Object-Based Data Access
Object-Based Storage Devices (OSD)
Object Storage Systems
Object Storage Server (OSS)
Content Addressable Storage (CAS)
Content Aware Storage (CAS)

Data Access
Data Access

Block

SCSI
SAS
iSCSI
SATA

Object

OSD

File

Local FS
Distributed FS
Global, Distributed & Parallel FS

The Block Paradigm

Block Paradigm (contd.)


It allows for convenient manipulation of data at a byte-level, useful for Random
I/O and where only small portions of data are required.
Serious overhead when block-based storage system expanded beyond hundred
terabytes or beyond multiple petabytes. Durability issues will also present itself.
Solving the provisioning management issues presented by the expansion of
storage at this scale is where object storage shines.
Object storage is resilient while mitigating costs. Objects remain protected by
storing multiple copies of data over a distributed system; if one or more nodes
fail, the data can still be made available, in most cases, without the application or
the end user ever being impacted.

Block Paradigm (contd.)


However, the degree to which low-level interaction can be performed is very
limited compared to Block/File storage.
Interaction can occur via a single API endpoint. This eliminates complex LUN
mapping, storage network topologies, etc. from the application infrastructure
design.

Local File Systems (One more level of Indirection)

Distributed File Systems


e.g NAS with NFS

Inode Metadata

File Storage (contd.)

File systems allow concurrent access to smaller groups & they enable read and
write operations. Overhead to manage permissions and operations such as file
locking.
Much of the unstructured data that is generated today, does not require
concurrent access; so the file systems overhead is unnecessary and simply adds
cost and complexity.
Hierarchical file structure consisting of directories and a hierarchy of nested
folders, subfolders and files; the content and data contained within a file is not
anywhere as important as is the location in the directories; as a consequence
each file will have only basic metadata attached to it, such as file name, date
created, last modified, file type, and person who created the file.

The Old Block Paradigm

The New Object Paradigm Flat Address Space

The New Object Paradigm (contd.)


WRITE 26,763 Bytes
QoS = High
Description = X-Ray
Retention = 50 years
Access Key = *&^%#

Object Storage Responsibilities:


Space Management
Access Control (Identity Mgmt)
QoS Management
Cache, Backup
Policy Migration, Retention

In a Flat Namespace

Benefits over File Storage


No directory Tree
Object Storage uses a flat structure, storing objects in containers, rather than a nested tree
structure
Eliminating the overhead of keeping track of large quantities of directory metadata, one
major performance bottleneck that is typically seen once tens of millions of files are
present on a filesystem is eliminated.

Metadata lives with the object

Objects

Inodes vs Objects

Virtual View/Virtual File Systems

Benefits over Virtualization


Object Storage is one (or potentially few in the case of multi-region
deployments) giant volume. Virtually all storage management
overhead of Block and File Storage is eliminated as a result (i.e. no
more resizing or remapping volumes).

Comparison
We dont recommend you use object storage for transactional data,
especially because of the eventual consistency model outlined
previously. In addition, its very important to recognize that object
storage was not created as a replacement for NAS file access and
sharing; it does not support the locking and sharing mechanisms
needed to maintain a single accurately updated version of a file.
Good examples for block storage use cases are structured database
storage, random read/write loads, and virtual machine file system
(VMFS) volumes.

Object Types
The ANSI T10 SCSI OSD standard defines four different objects:
The root object -- The OSD itself
User objects -- Created by SCSI commands from the application or client
Collection objects -- A group of user objects, such as all .mp3 objects or all objects belonging to a project
Partition objects -- Containers for user objects and collections that share common security and space management
characteristics, such as quotas and keys

Characteristics for Object Storage


Scalability: Many of the features of Object Storage seem inconvenient at very small scales, but as data
scale reaches hundreds of TB and moves into the PB range and beyond, these features become
invaluable, and allow continued horizontal scalability for virtually any quantity of data.

Durability: Due to the design of most Object Storage systems (3 file replicas is the most common
paradigm), durability levels at scale are extremely high compared to conventional storage solutions
(think 99.99999% to 99.999999999%, 7 to 11 nines). Object Storage systems have internal mechanisms
to verify file consistency, and handle failed drives, bit-rot, server and cabinet failures, etc. These
features allow the system to automatically replicate data as needed to retain the desired number of
replicas, which results in extremely high durability and availability of data.
Cost: Because many Object Storage platforms are designed to run on commodity hardware, even with
3x overhead, the price point is attractive when compared to block or file storage. At scale, costs of
pennies per gig per month are typical. Think the comparable or better durability than tape, nearly the
same cost as tape, and the convenience and performance of hot storage, plus all the benefits of a
cloudy storage platform.

Other Characteristics for Object Storage


Redundancy: redundancy and high availability by storing copies of the same object on multiple nodes. When an object
is created, it's created on one node and subsequently copied to one or more additional nodes, depending on the
policies in place The lack of in-place update support enables multicopy object redundancy with very little complexity.
For traditional storage systems, keeping copied (replicated) files and blocks in-sync across multiple instances is a
tremendous challenge; it's very complex and can only be done with a set of very strict restrictions, such as within
defined latency constraints
Protocol Support: Traditional block- and file-based protocols work well within the data center where performance is
good and latency isn't an issue. But they're not appropriate for geographically distributed access and the cloud where
latency is unpredictable. Furthermore, traditional file systemprotocols (CIFS and NFS) communicate on TCP ports that
are available on internal networks, but rarely exposed to the Internet. Conversely, object storage is usually accessed
through a REST API over HTTP (OpenStack, Amazon S3). Simple commands put, get, delete, list.
Application support and integration: Since the industry is still struggling to agree on standards, widespread object
storage integration is still sparse. Custom application integration, some commercial applications, especially for backup
and archiving, have added object storage integration support, primarily linking to Amazon S3 cloud storage.
Cloud features: Multi-tenancy and the ability to securely segregate different users' data are must-haves for an object
storage product to be used beyond the enterprise. Security is more than encryption, and should include provisions to
control access to tenants, namespaces and objects.

The ANSI T10 object-based Storage Standard


In 2004, the ANSI (American National Standards Institute) T10 Standards body ratified an Object-based Storage
Device (OSD) command set for SCSI (Small Computer System Interface) storage devices that implements the OSD
interface.

New Paradigm
Evolving the storage interface. The figure indicates how the
OSD migrates (lower arrow) a portion of the file system into
the storage device while leaving the file system high-level
policy functions (e.g., user authentication) to the server
(upper arrow). A cryptographic mechanism (symbolized by
the keys in the figure and generated using private keys
stored in the OSD) is used to authorize client access.

current storage devices already


perform extensive processing, virtualizing block storage
addresses to remap unusable sectors and automatically
recover from I/O errors. However, block-based storage
devices have no information about the relationship
between blocks, making it impossible for the device to
intelligently allocate its internal resources

Rationale
An object can grow or shrink dynamically and is completely contained
and managed within a single OSD.
Objects are grouped into partitions that enable security, space
management, and quota management. Each partition represents a
security domain with its own set of keys.
The OSD object model is flexible, allowing higher-level software to
map each of its entities (e.g., file and table) to a single object,
multiple objects, or a partial object.

Basic Command Set

OSD Optimizations

One of the common usage patterns of an OSD involves an operation (e.g., WRITE)
on the data segment of an object and then accessing some attributes (e.g., a file
system maintained last-access-time attribute). To support this common access
pattern, the OSD standard allows all commands to include a SET_ATTR and/or
GET_ATTR operation. We call this piggybacking an attribute access on a
command. Piggybacking attribute updates can significantly reduce the number of
messages and hence the message processing on each OSD. Piggybacking can also
improve client latency by removing the additional network roundtrip that a
separate GET_ATTR or SET_ATTR command would create.

Security

Object Storage Device Security Methods

Seagate Object Drive

Seagate designed and made a protype of an object disk that could be a


continuation of its existing line of highend hard disks.
The OSD code ran on one of the processors alongside other portions of the drive
firmware
OSD extends earlier interfaces by providing two crucial pieces
First, since space management is delegated to drives, drives now have basic information
such as which blocks are free and which are in use. This allows drives to optimize reliability
and service quality.
The second piece of information gained from the OSD relates to information about
application requirements. Through shared attributes, applications can relay preferences for
QoS and reliability.

With traditional sector-based drives, disk access is based on fixed-length blocks


accessed by block addresses. As a result, any change in the sector size requires
widespread software modifications. With an OSD, the drive manages block size
and space management

Seagate Object Drive (contd.)


The addition of space management is a natural extension of the functionality
provided by the drive. Current drives already contain relocation maps for defect
management, zoning, and other performance and reliability tracking. Allocation
maps extend this functionality by providing richer semantics to help minimize
seeks and to optimize read/write caching.
Exception handling and recovery are critical for the proper functioning of the
drive. As with a host-based file system, each drive must protect metadata. This
can be done by internally mirroring the metadata or by protecting it with more
powerful error-correction codes.
OSD drives can unobtrusively perform repairs or fence objects if they are not
repairable.
Achieving high data-transfer rates to a hard disk is difficult because of its limited
computational power. Specialized hardware was used wherever possible. The
OSD protocol has more complex commands than those used in standard SCSIs

Challenges for Object Stores

Eventual Consistency
Scale out by adding more x86 servers, nodes to the Object Store
Due to distributed nature of Object Storage, it is subject to Brewers CAP theorem.
It states that it is impossible for a distributed system to simultaneously
Provide consistency, availability, and partition tolerance
Object stores can offer two but not three

Eventual Consistency - Eventual consistency means that in case


data objects are stored and receive no new updates, that eventually
all nodes with access to these data objects will return the last
updated value. Eventual consistency has been proposed so Object
Stores can offer an acceptable performance.

Latency and Performance

Virtual Machines especially IOPS devouring applications require storage latency


and high performance storage SAS disks, fiber channel, infiniband and All-Flash
Arrays were introduced to offer the necessary bandwidth and an acceptable
latency.
Object Storage is developed and optimized to contain a massive amount of data.
To maximize the amount of storage capacity per node in the Object Storage
Cluster, large SATA disks are selected as these provide the best price per GB. By
selecting these large disks, you cant achieve the IOPS and storage performance
needed by Virtual Machines. SANs have been fitted with fast, but small SAS
drives. Try to maximize the amount the storage per node and select smaller, more
expensive SSD disks, but the price per GB stored data skyrockets.

Different Management Paradigms


Object Stores understand objects, while hypervisors understand Virtual
Machines. What is needed is a software layer that plugs into the hypervisor such
that the system administrator doesn't need to understand LUNs, RAID groups, etc
but can just manage Virtual Machines. This software layer has to translate a VM
paradigm into an Object Store paradigm.

Turning Object Storage into Virtual Machine Storage

Open vStorage takes this different approach and is designed from the ground up
with Virtual Machines and their performance requirements in mind. It uses a
well-considered architecture which allows Object Storage to be turned into block
storage for Virtual Machines and avoids pitfalls such as seen with distributed file
systems linked to Object Storage.
Open vStorage creates a unified namespace for the Virtual Machines across
multiple Hosts.

OpenStack Swift
Data is distributed by making distributed copies or replicas to different nodes and
drives across the Swift cluster
The number of replicas is user-configurable
The access is provided via RESTfil API and am Amazon S3 compatible RESTful API
Swift is based on ring architecture with three categories: account, container, and
objects

Ceph

Stores data on a single distributed computer cluster and provides interfaces for
object-, block-, and file-level storage
Does not have a single point of failure
Scalable to Exabyte level
Replicates data and make it fault tolerant

Ceph employs four distinct kinds of daemons:[4]


Cluster monitors
Metadata
Object Storage Devices
Representational state transfer

Characteristics
API-level access vs. filesystem-level
Flat structure vs. hierarchical structure
Scalable metadata
Scalable platform
Durable data storage
Low-cost data storage

Advantages
Scalable capacity (many PB easily)
Scalable performance (environment-level performance scales in a
linear manner)
Durable
Low cost
Simplified management
Single Access Point
No volumes to manage/resize/etc.

Disadvantages
No random access to files
POSIX utilities do not work directly with object-storage (it is not a
filesystem)
Integration may require modification of application and workflow
logic (This could easily be seen when we are trying to use FIO for the
Kinetic Drives). Due to API in there we have to alter the FIO to make it
run benchmarks on the Kinetic Drive.
Typically, lower performance on a per-object basis than block storage

Use Cases
Currently the datasets best-suited for Object Storage are the following:
Unstructured data

Media (images, music, video)


Web Content
Documents
Backups/Archives

Archival and storage of structured and semi-structured data


Databases
Sensor data
Log files

Thanks

Common questions

Powered by AI

Object storage simplifies data management by using a flat namespace and eliminating directory hierarchies, thus reducing the complexity associated with managing large quantities of directory metadata typically encountered in file systems . This approach also alleviates the need for administrators to manage complexities like volume resizing, LUN mapping, and detailed RAID configurations typical of block storage systems . The system’s use of RESTful APIs allows for straightforward data access and management operations such as 'put', 'get', and 'delete' without intricate network and protocol interplay . For data administrators, this translates into reduced operational overhead, simplified scaling processes, and more focus on data policy and QoS management rather than on storage architecture intricacies, enabling a shift from micromanaging storage components to managing data at a higher level of abstraction .

Redundancy and availability features of object storage are pivotal to their suitability for cloud infrastructure, primarily due to their ability to store multiple copies of objects across different nodes, ensuring high data availability even in the event of node failures . The system's design allows for automatic data replication, which promptly generates additional copies when a failure is detected, ensuring that data remains accessible and durable without impact on the application or end-user. This design aligns well with the cloud paradigm, where scalability, multi-tenancy, and fault tolerance are essential. The redundancy mechanism contributes to effective load balancing, allowing seamless data retrieval based on availability, enhancing overall resilience in cloud environments . These attributes make object storage particularly advantageous for distributed cloud applications where high data availability and low risk of data loss are critical .

Object storage systems face integration challenges primarily due to the lack of standardization and the reliance on RESTful APIs over HTTP, which can be less compatible with existing software designed for traditional file and block storage protocols like CIFS and NFS . While block and file storage have established protocols that fit well within data center environments with predictable performance and latency , object storage requires custom integration by applications, particularly for cases such as backup and archiving linking to services like Amazon S3 . This often necessitates changes in application workflows and logic to accommodate the new storage paradigm, unlike block and file systems which are commonly supported and utilized in plug-and-play configurations .

Eventual consistency in object storage impacts data availability and latency by ensuring that while all updates will eventually propagate across the system, there may be a delay before all parts of the system reflect the same data. This can be a crucial limitation for applications requiring real-time consistency and low-latency access . Geographically distributed systems further complicate this, as data needs to synchronize across nodes in different locations, potentially increasing access time and causing temporary inconsistencies. Protocol support using REST APIs over HTTP simplifies access in cloud environments but can introduce additional latency compared to internal data center protocols optimized for lower latency environments . These factors collectively can limit the use of object storage in applications where immediate consistency and minimal latency are critical. However, for use cases tolerant of eventual consistency, such as media distribution and archival storage, these drawbacks are outweighed by object storage's scalability and redundancy features .

The cost benefits of using object storage compared to traditional storage solutions largely stem from its ability to scale using commodity hardware, which significantly reduces the cost per gigabyte even with the additional overhead of data replication. Object storage’s design is inherently cloud-oriented, optimizing costs since it takes advantage of low-cost, scalable architectures that do not require the high-performance hardware often demanded by block storage to meet its latency requirements. Unlike traditional storage, object storage eliminates the necessity for expensive file system overhead and complex management of directories, reducing operational costs . The flat structure and high durability levels allow organizations to use object storage economically for unstructured data and archival purposes, retaining performance comparable to more costly solutions like tape storage .

Object storage systems store metadata along with the object itself in a flat structure, eliminating the hierarchical tree structure used in traditional file systems. This approach allows for scalability by removing the performance bottleneck associated with maintaining extensive directory metadata. This flat structure enables object storage to efficiently manage tens of millions of files without the overhead experienced with file systems when scaling to such numbers . It also supports enhanced data management tasks such as access control, policy creation, and Quality of Service management .

Integrating object storage with virtualization technologies helps address storage management overhead by abstracting complex storage tasks such as LUN management and RAID configurations into a more flexible storage paradigm. Open vStorage, for instance, converts object storage for virtual machines into block storage, offering a spacious and unified namespace across hosts. This synergy simplifies the virtualization layer, as it enables system administrators to manage virtual machine storage without focusing on the underlying complexities of LUNs or RAID groups . The performance impact can be positive, as the architecture is specifically designed to meet the demands of virtual machine performance requirements, aligning object storage's scalability with virtualization needs. However, this approach may necessitate architectural changes to leverage the strengths of both technologies fully .

To mitigate challenges in scaling object storage systems, strategies such as implementing more sophisticated data replication policies and using distributed consensus algorithms can help maintain data consistency across nodes. Employing a robust load balancing mechanism can distribute data efficiently to minimize latency. Choosing appropriate consistency models, like linearizability for critical data sets, can help balance performance and consistency needs. Additionally, architectural designs that include failover strategies and automatic data reallocation can support performance maintenance and system reliability. Leveraging cloud-native technologies and microservices architectures can further enhance scalability by enabling seamless integration of new nodes without disrupting service availability . Advanced caching techniques and intelligent data partitioning can be utilized to optimize data access patterns, maintaining high system performance across distributed nodes .

The object-based storage model using the ANSI T10 SCSI OSD standard provides scalability by enabling horizontal scaling to handle very large quantities of data, typically hundreds of terabytes to petabytes and beyond . Object storage allows for high durability due to its design, which often involves three or more file replicas, ensuring a high level of data protection and availability (e.g., 99.99999% to 99.999999999% durability). This replication is automatic and flexible, allowing the system to self-manage failures such as failed drives or bit-rot, effectively enhancing both scalability and durability compared to traditional storage which often struggles to seamlessly replicate data and manage substantial data scales .

Object storage systems are not ideal for transactional data because they generally use an eventual consistency model which can lead to latencies in data synchronization across distributed nodes. This makes it unsuitable for transactional systems that require immediate consistency and real-time data integrity . In contrast, block storage is well-suited for transactional workloads due to its random read/write capabilities and stricter consistency models, upheld by the nature of low-latency operations and structured data environments . Furthermore, object storage lacks file locking and sharing mechanisms essential for maintaining a single updated version of a file in transactional applications .

Object Storage
Manas Minglani
Wednesday, 2015
The Storage Evolution: From Blocks, Files and 
Objects to Object Storage Systems
• Structure Data vs Unstructured
Topics
• Block-Based Data Access
• File-Based Data Access
• Object-Based Data Access
– Object-Based Storage Devices (OSD)
– O
Data Access
Data Access
File
Object
Block
SCSI
SAS
iSCSI
SATA
OSD
Local FS
Distributed FS
Global, Distributed & Parallel FS
The Block Paradigm
Block Paradigm (contd.)
• It allows for convenient manipulation of data at a byte-level, useful for Random 
I/O and where onl
Block Paradigm (contd.)
• However, the degree to which low-level interaction can be performed is very 
limited compared to Bl
Local File Systems (One more level of Indirection)
Distributed File Systems
e.g NAS with NFS
Inode Metadata

You might also like