Drivers for Object-based Storage

• Amount of data created annually is growing exponentially and

more than 90% of data generated is unstructured
– Rapid adoption of third platform technologies leads to significant
growth of data
– Longer data retention due to regulatory compliance also leads to
data growth
• Data must be instantly accessible through a variety of devices
from anywhere in the world
• Traditional storage solutions are inefficient in managing this
data and in handling the growth

Module 7: Object-based and Unified Storage 1

© Copyright 2015 EMC Corporation. All rights reserved.
Object-based Storage Device
Object-based Storage Device

Stores data in the form of objects on flat address space based on its content
and attributes rather than the name and location.

• An object is the fundamental unit of object-based

storage Object IDs

• Object contains user data, related metadata, and

user-defined attributes Object Object
Object
– Each object stored in the object-based storage Object ID Object Object

system is identified by a unique identifier called the Data Metadata

object ID. The object ID allows easy access to Attributes Object Object

objects without the need to specify the storage

location.

• OSD provides APIs to integrate with software-

defined data center and cloud Module 7: Object-based and Unified Storage 2
© Copyright 2015 EMC Corporation. All rights reserved.
Hierarchical File System Vs. Flat Address
Space
• Hierarchical file system organizes data in the form of files/directories
– Limits the number of files that can be stored

• OSD uses flat address space that enables storing large number of objects
– Enables the OSD to meet the scale-out storage requirement of third platform
Object
File names/inodes IDs

Object Object Object

Object
ID
Object Object
Data Metadata

Attributes
Object Object

Hierarchical File system Flat Address Space

Module 7: Object-based and Unified Storage 3

© Copyright 2015 EMC Corporation. All rights reserved.
 Components of Object-based Storage Device
• OSD system typically comprises three key components:
– OSD nodes (controllers) A node is a server that runs the OSD operating
environment and provides services to store, retrieve, and manage data in the
system
– Internal network The internal network provides node-to-node connectivity
and node-to-storage connectivity
Metadata
Metadata
Service
Metadata
Service
Service
Storage
IP Internal
Storage
Service
Network Storage
Service 0
Network
Service
Application
Server
OSD Nodes/Controllers
Storage

OSD System

• Storage uses low-cost and high-density disk drives to store the objects
Module 7: Object-based and Unified Storage 4
© Copyright 2015 EMC Corporation. All rights reserved.
Key Features of OSD
Feature Description
Provides linear scalability where nodes are
Scale-out
independently added to the cluster to scale
architecture massively
Enables multiple applications/clients to be served
Multi-tenancy from the same infrastructure

Metadata-driven Intelligently drive data placement, protection, and

policy data services based on the service requirements

Abstracts storage from the application and provides

Global namespace a common view – independent of location and
making scaling seamless

Module 7: Object-based and Unified Storage 5

© Copyright 2015 EMC Corporation. All rights reserved.
Key Features of OSD (Cont’d)

Features Description
Supports REST/SOAP APIs for web/mobile access,
Flexible data access
and file sharing protocols (CIFS and NFS) for file
method service access

Automated system Provides auto-configuring, auto-healing capabilities

management to reduce administrative complexity and downtime

Object is protected using either replication or

Data protection —
erasure coding technique and the copies are
Geo distribution distributed across different locations

Module 7: Object-based and Unified Storage 6

© Copyright 2015 EMC Corporation. All rights reserved.
Object Protection: Erasure Coding
• Provides space-optimal data redundancy to protect data loss against multiple drive failures
– A set of n disks is divided into m disks to hold data and k disks to hold coding
information
– Coding information is calculated from data
– In general, erasure coding technique breaks the data into fragments, encoded with
redundant data and stored across a set of different locations.
– In a typical erasure coded storage system, a set of n disks is divided into m disks to
hold data and k disks to hold coding information, where n, m, and k are integers.

Module 7: Object-based and Unified Storage 7

© Copyright 2015 EMC Corporation. All rights reserved.
Object-based Storage Implementations
Software-based Hardware-based
Object Storage Object Storage
• Object storage software is • Both software and purpose-built
installed on any compatible hardware are provided by
hardware vendor
• Provides the flexibility to reuse • Typically pre-configured and
the existing infrastructure pre-tested
(compute and storage) and to • Provides better performance
use commodity hardware
• Object-based software can also
be installed on virtual machines

Module 7: Object-based and Unified Storage 8

© Copyright 2015 EMC Corporation. All rights reserved.
Process of Storing Object in OSD System
2. OSD node divides the data into two parts,
user data and metadata

Metadata
Metadata
Service
1. Application server sends a data to OSD
Metadata
Service
Service
Storage 3. OSD node generates
Storage
Service object ID from the user data
Storage
Service
6. Acknowledgment sent to the Service
application server
Application
Server OSD Nodes/
Controllers
5. OSD stores user data (object) 4. OSD stores metadata and
using the storage service object ID using the metadata service

Storage
Module 7: Object-based and Unified Storage 9
© Copyright 2015 EMC Corporation. All rights reserved.
Process of Retrieving Object from OSD System
1. Application server requests data from OSD
2. Metadata service locates the
Metadata object ID for the requested data
Metadata
Service
3. Metadata service sends the object ID to Metadata
Service
the application server Service
Storage
Storage
Service
4. Application server sends the object ID to the OSD Storage
Service
storage service for object retrieval Service
Application
Server OSD Nodes/
6. Storage service sends the data to the Controllers
application server
5. OSD storage service retrieves the
object from the storage device

Storage
Module 7: Object-based and Unified Storage 10
© Copyright 2015 EMC Corporation. All rights reserved.
Unified Storage Architecture
SAN NAS Object
(iSCSI/FC/FCoE) (CIFS/NFS) (REST/SOAP)

Block File Object

Request Request Request

Block Storage NAS OSD

Functionality Functionality Functionality
Unified Controller

Block
Request

Unified Storage
Module 7: Object-based and Unified Storage 11
© Copyright 2015 EMC Corporation. All rights reserved.