Niktips's Blog

NetApp and VMware stuff

Archive for the ‘Isilon’ Category

Isilon Performance Considerations

February 20, 2014

([Link]
my previous post “EMC Isilon Overview ([Link]
overview/)” I talked about general Isilon configuration. In this post I want to describe some of the
performance tuning options.

SSD Acceleration

You can choose from three types of Isilon nodes: S-series, X-series and NL-series. And within the
node series you can select amount of memory, number/size/type of disk drives and the number of
1GB/s and 10GB/s network ports.

S- and X- series nodes can have SSD drives, which you can use for metadata and/or data,
depending on how much flash storage you have. Isilon have four SSD strategies: “Metadata Read
Acceleration”, “Metadata Read/Write Acceleration”, “Data on SSDs” and “Avoid SSDs”. All
strategies are pretty much self-explanatory. When using first strategy Isilon creates a copy of
metadata on SSD disks for read acceleration. With second strategy it mirrors metadata reads and
writes to SSDs (requires four to six times more space). Third strategy allows you to have data on
SSDs. And forth disables SSD acceleration all together. You can configure SSD strategy on a file
level policy level. And if you want to have “Data on SSDs” you can redirect only particular files
(say with the most recent access timestamps) to SSDs.
Isilon allows you to have SSD metadata acceleration even on node pools that don’t have SSDs in
them. It’s called Global Namespace Acceleration (GNA) and requires at least 20% of nodes to have
1 or more SSD disks and requires 1.5% or more (2% is recommended) of total storage to be SSD-
based.

Data Access Patterns

Isilon can be used for different type of workloads. You can have VMware datastores on it
connected via iSCSI or have CIFS file shares or maybe use it for streaming. Depending on your
data access patterns you can tweak Isilon for Random, Concurrency or Streaming content, which
affects how Isilon writes data on disks and how it uses its cache.

“Random” read/write type of workloads are typical for VMware environments. With this setting
Isilon disables prefetching of data into read cache. And this setting is default for iSCSI LUNs.

With “Concurrency” Isilon optimizes data layout for simultaneous access of many files from the
storage array. It uses moderate level of prefetching in this case. And for every 32MB of a file it
tries to hit the same disk within the node. This it is default for all data except iSCSI LUNs.

And “Streaming” is for the fast access to big files like media content. It has the most aggressive
level of prefetching and tries to use as many disk drives as possible when writing data on a
cluster.

SmartCache

This setting affects write caching. With SmartCache turned on, data that comes in to a cluster will
be cached in node’s memory. This is not NVRAM and if node fails uncommitted data is lost. This
might not be that critical for NFS and CIFS data, but loosing iSCSI blocks can result in file system
being unreadable. So be careful with SmartCache. You can specifically disable write cache on
iSCSI LUNs if you want to.

Accelerator Nodes

Isilon provides two types of accelerator nodes: Backup Accelerator and Performance Accelerators.
Both of them don’t contribute their disks to storage pool and provide additional capabilities to a
cluster instead. Backup accelerator has four 4GB/s FC ports for connections to tape libraries and
allows to offload backups from the storage nodes. And performance accelerators add additional
CPU and memory capacity to a cluster without adding any storage.

Data Protection Overhead

The default data protection level on Isilon is +2:1. It protects the storage system from two disk
failures or one node failure. It gives you sufficient level of protection and doesn’t have too much
overhead. If you need a higher level of protection, you need to realize that it can introduce much
overhead. Below is the table which shows amount of overhead depending on protection level and
number of nodes.
([Link]

As you can see, for six nodes +1 and +3:1 protection levels have the same overhead, but +3:1 gives
better protection. So you need to understand the impact of changing protection level and set it
according to your needs.

Tags:acceleration, cache, CIFS, concurrency, data protection, datastore, EMC, Global Namespace
Acceleration, GNA, iSCSI, Isilon, LUN, metadata, NAS, overhead, pattern, performance,
prefetching, random, Scale-Out, share, SmartCache, SSD, vmware, workload
Posted in Isilon | 2 Comments »

EMC Isilon Overview

February 20, 2014

([Link] Overview

EMC Isilon OneFS is a storage OS which was built from the ground up as a clustered system.

NetApp’s Clustered ONTAP for example has evolved from being an OS for HA-pair of storage
controllers to a clustered system as a result of integration with Spinnaker intellectual property. It’s
not necessarily bad, because cDOT shows better performance
([Link]
world-record-specsfs2008-nfs-benchmark-beats-emc-isilon) on SPECsfs2008 than Isilon, but these
systems still have two core architectural differences:

1. Isilon doesn’t have RAIDs and complexities associated with them. You don’t choose RAID
protection level. You don’t need to think about RAID groups and even load distribution between
them. You don’t even have spare drives per se.

2. All data on Isilon system is kept on one volume, which is a one big distributed file system.
cDOT use concept of infinite volumes, but bear in mind that each NetApp filer has it’s own file
system beneath. If you have 24 NetApp nodes in a cluster, then you have 24 underlying file
systems, even though they are viewed as a whole from the client standpoint.

This makes Isilon very easy to configure and operate. But its simplicity comes at a price of
flexibility. Isilon web interface has few options to configure and not very feature rich.

Isilon Nodes and Networking

In a nutshell Isilon is a collection of a certain number of nodes connected via 20Gb/s DDR
InfiniBand back-end network and either 1GB/s or 10GB/s front-end network for client
connections. There are three types of Isilon nodes S-Series (SAS + SSD drives) for transactional
random access I/O, X-Series (SATA + SSD drives) for high throughput applications and NL-series
(SATA drives) for archival or not frequently used data.

If you choose to have two IB switches at the back-end, then you’ll have three subnets configured
for internal network: int-a, int-b and failover. You can think of a failover network as a virtual
network in front of int-a and int-b. So when the packet comes to failover network IP address, the
actual IB interface that receives the packet is chosen dynamically. That helps to load-balance the
traffic between two IB switches and makes this set up an active/active network.

([Link]

On the front-end you can have as many subnets as you like. Subnets are split between pools of IP
On the front-end you can have as many subnets as you like. Subnets are split between pools of IP
addresses. And you can add particular node interfaces to the pool. Each pool can have its own
SmartConnect zone configured. SmartConnect is a way to load-balance connections between the
nodes. Basically SmartConnect is a DNS server which runs on the Isilon side. You can have one
SmartConnect service on a subnet level. And one SmartConnect zone (which is simply a domain)
on each of the subnet pools. To set up SmartConnect you’ll need to assign an IP address to the
SmartConnect service and set a SmartConnect zone name on a pool level. Then you create an “A”
record on DNS for the SmartConnect service IP address and delegate SmartConnect DNS zone to
this IP. That way each time you refer to the SmartConnect zone to get access to a file share you’ll
be redirected to dynamically picked up node from the pool.

SmartPools

Each type of node is automatically assigned to what is called a “Node Pool”. Nodes are grouped
to the same pool if they are of the same series, have the same amount of memory and disks of the
same type and size. Node Pool level is one of the spots where you can configure protection level.
We’ll talk about that later. Node Pools are grouped within Tiers. So you can group NL node pool
with 1TB drives and NL node pool with 3TB drives into an archive tier if you wish. And then you
have File Pool Policies which you can use to manage placement of files within the cluster. For
example, you can redirect files with specific extension or file size or last access time to be saved on
a specific node pool or tier. File pool policies also allow you to configure data protection and
override the default node pool protection setting.

SmartPools is a concept that Isilon use to name Tier/Node Pool/File Pool Policy approach. File
placement is not applied automatically, otherwise it would cause high I/O overhead. It’s
implemented as a job on the cluster instead which runs at 22:00 every day by default.

Data Layout and Protection

Instead of using RAIDs, Isilon uses FEC (Forward Error Correction) and more specifically a Reed-
Solomon algorithm to protect data on a cluster. It’s similar to RAID5 in how it generates a
protection block (or blocks) for each stripe. But it happens on a software level, instead of
hardware as in storage arrays. So when a file comes in to a node, Isilon splits the file in stripe units
of 128KB each, generates one FEC protection unit and distributes all of them between the nodes
using back-end network. This is what is called “+1″ protection level, where Isilon can sustain one
disk or one node failure. Then you have “+2″, “+3″ and “+4″. In “+4″ you have four FECs per stripe
and can sustain four disk or node failures. Note however that there is a rule that the number of
data stripe units in a stripe has to be greater than number of FEC units. So the minimum
requirement for “+4″ protection level is 9 nodes in a cluster.
([Link]

The second option is to use mirroring. You can have from 2x to 8x mirrors of your data. And the
third option is “+2:1″ and “+3:1″ protection levels. These protection levels let you balance between
the data protection and amount of the FEC overhead. For example “+2:1″ setting compared to “+2″
can sustain two drive failures or one node failure, instead of two node failure protection that “+2″
offers. And it makes sense, since simultaneous two node failure is unlikely to happen. There is
also a difference in how the data is laid out. In “+2″ for each stripe Isilon uses one disk on each
node and in “+2:1″ it uses two disks on each node. And first FEC in this case goes to first subset of
disks and second goes to second.

One benefit of not having RAID is that you can set protection level with folder or even file
granularity. Which is impossible with conventional RAIDs. And what’s quite handy, you can
change protection levels without recreation of storage volumes, as you might have to do while
transitioning between some of the RAID levels. When you change protection level for any of the
targets, Isilon creates a low priority job which redistributes data within the cluster.

Tags:cDOT, cluster, Data ONTAP, DOT, EMC, FEC, File Pool Policy, Forward Error Correction,
IB, InfiniBand, Isilon, NetApp, node, Node Pool, OneFS, RAID, Reed-Solomon, SmartConnect,
SmartPools, storage, stripe, tier
Posted in Isilon | Leave a Comment »

The Kubrick Theme. Create a free website or blog at [Link].

Follow

Follow “Niktips's Blog”