NTS-2001-I:

Nimble Introduction and Administration

Course Revision 5
Nimble Storage Partner Learning Paths
 NTSP Learning Path (Nimble Partners Only)

NTSP Accreditation
NS-1001-E NTS-1001-E NTS-2001-E NTSP
Conclusion

Nimble Technical Sales Professional (NTSP) Accreditation

– NS-1001-E: Introduction to Nimble Storage
– NTS-1001-E: Introduction to Nimble Storage Technology
– NTS-2001-I: Nimble Storage Introduction and Administration

Above courses may be taken in any order.

– To complete the learning path you must view the NTSP Accreditation Conclusion.
Introductions

– Name
– Employer
– Job function
– Data storage experience
– Hands on experience with Nimble products?
– What do you hope to get out of this course?
 Course Topics
– Product Introduction – Backup and Disaster Recovery
– Module 1: AF-Series Introduction – Module 8: Introduction to Nimble Storage
– Module 2: CS-Series Introduction Snapshots
– Module 3: NimbleOS WebUI Introduction – Module 9: Introduction to Nimble Storage
Replication
– Module 4: Introduction to Customer Support
and InfoSight
– Exam completion – required for course
completion
– Product Utilization
– Module 5: Pre-Installation, Racking, and
Cabling
– Module 6: Initial Configuration of the Unified
Flash Fabric Overview
– Module 7: Working with Nimble Storage
Volumes
Documentation
Documentation location

1. Log into InfoSight – Categories

2. Click Resources and then Documentation – Release Notes
– Product User Guides
– Solution and Integration Guides
– Support and Knowledge Base Articles

– Refine Searches By:

– Document type
– Nimble Software and Solutions
– NimbleOS Version
– Integration
– Platform
Units of Measure Defined
Raw versus Usable versus Effective Capacity
NOTE: Numbers displayed are only for training
purposes and do not represent actual array
numbers. AF-Series arrays using
deduplication will see
additional savings.
Subtract
capacity
for Add storage
Raw RAID Triple capacity due
to inline Effective
Capacity Parity,
Usable compression Capacity
& system
Capacity (typical 30%
reserves
to 75%)

Effective: 49.5TB
Raw: 48TB Usable: 33TB
(assuming 50% compression)
TiB Vs. TB

collateral utilizes
TB and TiB
Nimble OS uses TiB

Tebibyte (TiB) and terabyte (TB) are related

– TB is base 10 (i.e., 1 TB = 1,000,000,000,000 bytes)

– Ti, or tebi, represents multiplication by 10244. So the math using Ti is more

accurate:
– 1 tebibyte = 240 bytes or 1,099,511,627,776 bytes or more commonly stated as 1024
gigabytes
NTS-2001-I:
Nimble Introduction and Administration

Module 1:
AF-Series Introduction

Course Revision 5
 Module 01 Objectives
At the end of this module, you will be able to:
– Describe the AF-Series hardware
– Understand the AF-Series Write Operations
– Deduplication
– Compression
– Sequential stripe writes
– Triple+ Parity with Integrated Spare
– Integrated spare rebuild
– Quick RAID rebuild
– SmartSecure Encryption
– Understand AF-Series Read Operations
– Changed block operations
– Scale-to-Fit with the AF-Series
Nimble Product Line Comparison

Performance Capacity
IOPS Effective GBs

AF-Series CS-Series SF-Series

All-Flash Arrays Adaptive Flash Arrays Secondary Flash Arrays

Storage
Tier Primary storage Secondary storage

Usage/ High performance Other primary Backup, DR, Test/Dev,

Workloads primary applications applications other secondary apps
All Flash Array Family: AF-Series
AF9000

Up to 1.86PB
AF7000 Effective Capacity
300,000 IOPS

Up to 1.19PB
AF5000 Effective Capacity
230,000 IOPS

Up to 680TB
AF3000 Effective Capacity
120,000 IOPS

AF1000 Up to 335TB
Effective Capacity
50,000 IOPS IOPS based on a 70% Read
Up to 165TB and 30% write workload
Effective Capacity
35,000 IOPS
AF-Series Hardware Tour
What’s in an AF-Series Array?

Controller Head Shelf

Expansion Shelves (optional)
Back: Dual Power Supplies
(AC and DC available)

Front: SSD Drives Back: Dual Controllers (CPU, Network)

AF-Series Chassis Front View

– 4U Chassis
– 48 SSD Drive capacity
– 2 Banks of 24 x 3.5” SSD drives
– Bank A – SSD Drives 1 -24
– Bank B – SSD Drives 25 – 48

Slots 21-24

Slots 17-20

Slots 13-16

Slots 9-12

Slots 5-8

Slots 1-4
 AF-Series Chassis Front View / Dual-Flash Carrier (DFC)

Power On LED
Heartbeat LED
NIC 1/2

Power Fault Over

Temperature

 Hot swappable base

carrier hosting two banks.
 Hot swappable SSD
carrier provides tool-less
drive installation.
AF-Series Dual-Flash Carrier (DFC)
Bank B Bank B Ejector latch

DFC latch

Bank A Bank A Ejector latch

Presence & Drive Fault

Slot Fault
Activity LED

Base Carrier
Release

SSD Carrier
Release
 AF-Series Hardware Tour - Back
1

2
Power Supply
Options 8

6
3

4 5 7
1. Power 3. Fans 5. KVM/Serial Port 7. SAS Ports
Supply 4. Management Network 6. Data Networks 8. Controller
2. Power Inlet
AF-Series Connectivity Options

– By default there are two onboard 10Gbase-T ports

– An array can be ordered with only these ports if desired
– Ethernet
– 1Gb and 10Gb ports available for order
– Optical available
– Support for up to 12 ports
– Quad port cards
Configurable ports
– Fiber Channel
– 16GbFC
– Support for up to 12 ports
– Quad port cards

– Mix of port types available

Default onboard ports

Controller Detail View (AF-Series and Newer CS-Series)
 AF-Series Spares / FRUs (Controller Head Shelf)

Or
Spares Kit
Controller
(NICs/HBAs not included) Power Supply

Additional Spares
AF-Series SSD
NIC / HBA

Cables
AF-Series Write Operations
 Nimble OS Architecture
8K 4K 4K
ack

Mirrored
1) Writes are sent by a variety of
applications in variable block sizes
NVDIMM (Active CTRL) NVDIMM (Standby CTRL)
2) Nimble OS places incoming writes
8K 4K 4K 8K 4K 4K
into the active controllers NVDIMM
which is backed up via an Ultracap
3) Nimble OS mirrors the active DRAM
controllers NVDIMM to the standby
controllers NVDIMM
4) Nimble OS acknowledges the write

AF-Series

Index
 Nimble OS Architecture
8K 4K 4K
ack

Mirrored
5) Blocks are copied into DRAM
a) What happens next depends of
NVDIMM (Active CTRL) NVDIMM (Standby CTRL)
type of array
8K 4K 4K 8K 4K 4K
b) All-Flash Array
1. Variable block deduplication
is applied DRAM
2. Variable block compression 8K 4K 4K
Variable Block Variable Block
is applied Deduplication Compression
Sequential Stripe
3. Blocks are formed into a
sequential stripe write
4. Blocks written to SSD and Index
are indexed in DRAM
AF-Series

Index
 Nimble OS Architecture
– What happens should the array lose power?
– The contents of the NVDIMM are protected long enough by an Ultracap to complete a data flush operation
– The data in NVDIMMs is flushed to flash on the NVDIMM with a safety retention that exceeds 50 years with no
battery requirement.

– Once power is restored:

– The array comes up normally
– Data in NVDIMM flash reloads into the NVDIMM DRAM.
– The array resumes where it left off at full line speed with no required integrity checks or human intervention

– Health of the Ultracap:

– Should the capacitance of the ultracap be discharged and insufficient to protect against such a sudden power loss, a
CRITICAL severity level alert will be generated
AF-Series: Deduplication
Mirrored
NVDIMM (Active CTRL) NVDIMM (Standby CTRL)

DRAM

Variable Block
Deduplication

AF-Series

Index
Deduplication Primer

Inline Deduplication

Vs.
Source / Server Post Process
Deduplication

Target / Storage Array

Inline Deduplication:
• Dedupe applied as the writes are processed

Post Process Deduplication:

• Writes the data as-is and uses a background operation to remove duplicates
Process Review
Incoming writes Batched in NVRAM/NVDIMM

Batch Flush

Variable Block Deduplication

• Secured by strong SHA2-256 encryption

Variable Block Compression

Full Stripe Writes

Deduplication Block Sizes
Variable Block Dedupe Compression

Fixed Block Dedupe

Target / Storage Array

Variable Block Level Dedupe:

• Unit of duplicate detection is volume block which depends on the performance policy.
• Requires less metadata by adapting the detection size to the application block size, thereby
improving efficiency and speed.
Application Aware:
DB VDI DB Exchange

VDI Exchange VDI VDI

Fingerprint Matching or Byte Comparison?

With most forms of deduplication,

performance is adversely affected Fingerprint Index
in some way.

Deduplication
Performance • Better than block-by-block
• Secured by strong SHA2-256 encryption
SHA256

– Why SHA256?
– Large fingerprint space.
– Fast implementation (low CPU overhead)
– Space overhead reasonable (0.8%)

– 2^256 = 115792089237316195423570985008687907853269984665640564039457584007913129639936

– No collisions have been found in generated data ever!

Dedupe Isn’t Free

– Fingerprinting content costs CPU

– Identifying duplicates costs:
– memory
– I/O (flash reads)
– CPU
– Tracking content costs storage
– Flash content/FDR

– Not every workload benefits!

– These costs are true of dedupe in general, not our implementation.
 Deduplication Global, Domain Level, or Off?
On – On –
Off
Globally Domain/Application
• Per Performance Policy
 Based on application Eliminate the
category deduplication
• Per Storage Pool “tax”
• Per Volume

Make informed decisions about whether to use

deduplication or not and what workloads or volumes
would be best served by deduplication.
InfoSight
Predictive Analytics
Deduplication Management – AFA only
Global Deduplication

Deduplication at Domain/Application Level

5
3
 Application Category Management – AFA only
– Application Categories are introduced
with Deduplication and are found as
part of the performance policy
– Application Category:
– A data structure in the WebUI that limits
the scope of duplication on volumes
attached to an Application Policy
– There are 8 fixed Application
Categories that can be assigned when
creating a custom Performance Policy
 Nimble Differentiators – More about App Category

App Category VDI Database

VDI-Policy-1 VDI-Policy-2 VDI-Policy-3 VDI-Policy-4 DB-Policy-1

Perf Policy BS=4K BS=8K BS=4K BS=8K BS=32K
Dedupe=ON Dedupe=ON Dedupe=ON Dedupe=OFF Dedupe=OFF

Volume V1 V2 V3 V4 V5 V6 V7 V8 V10 V11

V9
Like colored volumes
denote block sharing Dedupe = OFF
AF/CS-Series: Compression
Mirrored

NVDIMM (Active CTRL) NVDIMM (Standby CTRL)

DRAM

Variable Block Variable Block

Deduplication Compression

AF-Series

Index
What you need to know about Lempel-Ziv 4

–Compression performed in-line

–Applied after deduplication, if deduplication is enabled
–LZ4 is a fast lossless compression algorithm
–Provides compression speeds of 300 MB/s per CPU core
–Provides a fast decoder that provides speeds up to and beyond 1GB/s per
CPU core
–It can reach RAM speed limits on multi-core systems.

40
 Application Compression with Nimble

*March 2016
 Viewing Data Reduction Savings
Monitor >> Capacity

• Mouse-over Total
Savings
• Mouse-over
application total
savings
Sequential Stripe Write Mirrored

NVDIMM (Active CTRL) NVDIMM (Standby CTRL)

DRAM
Sequential
Variable Block Variable Block Stripe 10MB
Deduplication Compression

Index

AF-Series

Index
Common File Systems– Write in place file system (EMC, EQL)

Pros
•Simple to implement, long
history

•Good sequential read

performance without cache

Cons
•Poor random write performance

•Slow, high overhead compression

Common File Systems – Hole filling (WAFL, ZFS)
WAFL – Write Anywhere File Layout
ZFS – Copy on write transactional model Pros
•Good random write performance until
disk fills up

•More efficient redirect-on-write

snapshots
Cons
•Performance degrades over time

•Slow, high overhead compression

Nimble Storage File System – Always write full stripes

– Good AND consistent write performance

– Very efficient snapshots
– Fast inline compression
– Efficient flash utilization,
long flash life
– Ground up design
– Enables variable block size
– Uses a sweeping process to ensure full stripe write
space
Background Sweeping

• Data blocks are indexed as they are written

• Over time the deletion of snapshots and data leaves stale data blocks
• Sweeping removes stale blocks and forms new stripe writes with the
remaining active blocks
AF-Series: Triple+ Parity With
Integrated Spare Mirrored

NVDIMM (Active CTRL) NVDIMM (Standby CTRL)

DRAM
Sequential
Variable Block Variable Block Stripe 10MB
Deduplication Compression

Index

AF-Series

Triple+ Parity with

Integrated Spare Index
 Triple+ Parity With Integrated Spare

Triple+ Parity RAID Stripe

D1 D2 D3 P Q R
–Triple Parity: 3 drives worth
of parity information is
written
 Intra-Drive Parity: Data Layout

Triple+ Parity RAID Stripe

– Every chunk in a stripe consists of Data,
D1 D2 D3 P Q R
Metadata (checksums, sequence number etc…),
and Parity (XOR of Data)
– This parity can be used to recover 1 sector
corruption or IO error, when there is no
redundancy in the stripe (3 drive failure)
– This recovery can be done independent of the
condition of other drives: can recover from 1-
Data
sector corruption on every drive read in the
stripe. Chunk
(1 per drive)
– Note: Intra-Drive Parity is also available in CS-
Series starting with Nimble OS 3.X. Chances of Metadata
data recovery in HDD is lower than SSD due to Parity
errors/corruptions in HDD are often larger.
 AF-Series Tipple+ Parity RAID Groups
– Bank A – SSD drives are used to construct a RAID group
– Bank B – SSD drives are used to construct another RAID group

Bank B
Bank A
 Integrated Spare: Data, Parity, and Spare layout
AFA Triple+ Parity
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
D D D D D D D D D D D D D D D D D D D D P Q R S
D D D D D D D D D D D D D D D D D D D P Q R S D
D D D D D D D D D D D D D D D D D D P Q R S D D Tolerates Simultaneous Failure of Any 3 SSDs
D D D D D D D D D D D D D D D D D P Q R S D D D
D D D D D D D D D D D D D D D D P Q R S D D D D
D D D D D D D D D D D D D D D P Q R S D D D D D
D D D D D D D D D D D D D D P Q R S D D D D D D
D D D D D D D D D D D D D P Q R S D D D D D D D
D D D D D D D D D D D D P Q R S D D D D D D D D
D D D D D D D D D D D P Q R S D D D D D D D D D
D D D D D D D D D D P Q R S D D D D D D D D D D Built in virtual spare allows 4th failure
D D D D D D D D D P Q R S D D D D D D D D D D D
D D D D D D D D P Q R S D D D D D D D D D D D D
D D D D D D D P Q R S D D D D D D D D D D D D D
D D D D D D P Q R S D D D D D D D D D D D D D D
D D D D D P Q R S D D D D D D D D D D D D D D D
D D D D P Q R S D D D D D D D D D D D D D D D D
D D D P Q R S D D D D D D D D D D D D D D D D D
D
D
D
P
P
Q
Q
R
R
S
S
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
Intra-drive parity fixes sector loss in single read
P Q R S D D D D D D D D D D D D D D D D D D D D
Q R S D D D D D D D D D D D D D D D D D D D D P
R S D D D D D D D D D D D D D D D D D D D D P Q
S D D D D D D D D D D D D D D D D D D D D P Q R

Data Quick RAID rebuild

Metadata
Parity
Integrated Spare: Rebuild
Mirrored
Operation NVDIMM (Active CTRL) NVDIMM (Standby CTRL)

DRAM
Sequential
Variable Block Variable Block Stripe 10MB
Deduplication Compression

Index

AF-Series

Index
Integrated Spare RAID : Single Drive Failure Scenario
Failed
Disk
Disk 0 Disk 1 Disk 2 Disk 3 Disk 4 Disk 5 Disk 6 Disk 7 Disk 8 Disk 9 Disk 10
D D D D D D D P Q R D
D D D D D D P Q R D D
D D D D D P Q R D D D
Spare Chunk
D D D D P Q R P D D D
D D D P Q R Q D D D D
D D P Q R R D D D D D

D P Q R S D D D D D D
P Q R D D D D D D D D
Q R D D D D D D D D P
R D D D D D D D D P Q
D D D D D D D D P Q R

–Generate data/parity belonging to failed drive

–Write data to spare chunk
 Integrated Spare RAID : Replacement Drive Rebuild Process

Rebuild to Replacement (Physical) Drives

–Walk through each stripe with valid data
–Copy data/parity from spare chunks back to replacement drive if available, OR
–Rebuild data/parity from the rest of the drives and write to replacement drive
–After rebuild process is complete, every spare chunk is erased and returned to
the drive as “overprovision” capacity

 Why is this IMPORTANT ?

 Size of SSD Dives / associated length of time to rebuild
 Wear-levelling increases the chances of multiple drive loss
 Failure Scenarios – Complete vs Segment
Quick RAID Rebuild Mirrored

NVDIMM (Active CTRL) NVDIMM (Standby CTRL)

DRAM
Sequential
Variable Block Variable Block Stripe 10MB
Deduplication Compression

Index

AF-Series

Index
 Quick Raid Rebuild: Motivation & Goal

Long rebuilds create frustration when:

– User pulls a drive out by mistake
– Drive is not completely failed, but needs to be reseated for various reasons
– S/W stack needs to power cycle the drive for recovery purposes
– S/W stack needs to reset interposer for recovery purposes that may lead HBA link timeout

Goal:
– Enable the NOS to take action to help reset or recover an unhealthy drive.
– See whether there is any way the RAID system can bypass the painful long rebuild or reduce its
impact to minimum
Quick Raid Rebuild: How does it work?

– The RAID superblock contains the rebuild history of all the drives
– If the drive is out, the rebuild history for that drive will be updated with the last segment
written to the disk
– If the drive is later put back (same serial #), the rebuild history will return the last segment
number written to that disk
Quick RAID Rebuild: How does it work?
The RAID layer checks the segment layer for the physical slot and corresponding segment numbers
that are less than the segment already written. If any such segments exist, the data is verified using the
checksum.

Process repeated for 100 random slots.

Verify 100 slots,

testing a
maximum of 500 Spot check fails
Spot check slots. or in case of
passes
any i/o error

Begin rebuild from the last

segment number stored in Begin rebuild from scratch.
rebuild history.
Quick Raid Rebuild: Impact to end-user

– When a drive is mistakenly taken out, it can be reinserted immediately without lengthy rebuild time
– No timeout period for quick rebuild eligible (!)
– Rebuild is essentially incremental in some scenarios
– The feature is ready on both AFA and CS in the NimbleOS 3.X release

Notes:
– Not applicable when replacing a failed drive
– “Incremental” rebuild, subject to certain conditions

60
SmartSecure Encryption Mirrored

NVDIMM (Active CTRL) NVDIMM (Standby CTRL)

DRAM
Sequential
Variable Block Variable Block Stripe 10MB
Deduplication Compression

Index

AF-Series

Index
 SmartSecure Software Based Encryption
What is SmartSecure?
Encryption that:
– Ensures the secrecy of data “at rest”
– Uses AES-256-XTS cipher for cryptographic protection of data
– FIPS 140-2 level 1 certified
*****
– Protects against threat vectors
– Theft of entire storage array Hello Hello
– Theft of disks (HDDs or SSDs)
– Is transparent to host access to encrypted volumes
– Data is encrypted at write time when entering the array
– Data is decrypted at read time when exiting the array
 SmartSecure Software Based Encryption
Why Deploy SmartSecure?
– Theft or disposal of drives or an entire array does not expose data
– Data Center physical security breach
– Shipping or transporting an array to an affiliate site or new location
– RMA of a failed component
– Ability to irrecoverably shred (destroy) data at a volume level
– Volume keys marked for deletion when an encrypted volume is deleted
– Requirements for data to be encrypted on disk
– Government, compliance, customer, etc.
– Ensures data secrecy on replication streams over a WAN
 SmartSecure Software Based Encryption
Key Data Points
– No additional cost
– Familiar Nimble Storage licensing/pricing model
– Supported on all Nimble Storage arrays running NimbleOS 2.3 and higher
– Leverages the Intel AES-NI instruction set
– Best performance on later model CS series arrays
– CS235, CS300, CS500, CS700

– Supports encryption of data on both HDDs and SSDs

– Preserves compression feature
– Data is compressed prior to encryption
– Easy to administer & manage
Administration >> Security >> Encryption
 SmartSecure Software Based Encryption
Technical Data Points
– Volume Keys
– The encrypted state of a volume is defined at volume creation time
– Encrypted volumes cannot be “converted” to unencrypted volumes
– Unencrypted volumes cannot be “converted” to encrypted volumes
– New encrypted volumes each get a new encryption key
– 256 bits in length
– Encrypted by the master key using AES-256-KeyWrap
– Stored in the key table
– Cloned encrypted volumes are also encrypted
– Clones get their own new volume key
– But also have access to their ancestor’s key to read shared blocks
 SmartSecure Software Based Encryption
Technical Data Points
– Volume Key Retrieval

1) Volume keys stored Clear Text

1) encrypted with Key Table Volume Key
1) AES-256-KeyWrap ID Key
7 Encrypted 256 Bit Key
8 Encrypted 256 Bit Key
9 Encrypted 256 Bit Key

Clear Text
Master Key

Encrypted
Clear Text
Volume
Host
Clear Text
 SmartSecure Software Based Encryption
Technical Data Points
– Volume Key Retrieval

1) Volume keys stored Clear Text

1) encrypted with Key Table Volume Key
1) AES-256-KeyWrap ID Key
7 Encrypted 256 Bit Key
2) Volume key
8 Encrypted 256 Bit Key
1) unencrypted with
9 Encrypted 256 Bit Key
1) clear text master
Clear Text
key Master Key

Encrypted
Clear Text
Volume
Host
Clear Text
 SmartSecure Software Based Encryption
Technical Data Points
– Volume Key Retrieval

1) Volume keys stored Clear Text

1) encrypted with Key Table Volume Key
1) AES-256-KeyWrap ID Key
7 Encrypted 256 Bit Key
2) Volume key
8 Encrypted 256 Bit Key
1) unencrypted with
9 Encrypted 256 Bit Key
1) clear text master
Clear Text
key
3) Clear text volume Master Key
key
Encrypted
1) used to encrypt Clear Text
Volume
writes Host
1) and to decrypt reads
Clear Text
AF-Series Read Operations
 Nimble OS Architecture

Mirrored
1) Read from NVDIMM
2) If not found, check DRAM
NVDIMM (Active CTRL) NVDIMM (Standby CTRL)
3) If not found, read from SSD using the
index for a quick lookup 1 8K 4K 4K 8K 4K 4K

• If found, validate checksum,

decompress, dedupe, and return 2 DRAM
data
Variable Block Variable Block
Deduplication Compression

Index
3
AF-Series

Index
Changed block operations review
 Compression Performance Comparison During Changed Block
Operation
Fixed block Architecture
Nimble OS
Variable Blocks
8 blocks Individual blocks
grouped & 1 2 3 4 5 6 7 8
compressed and
compressed coalesced into stripe
Group placed into
N fixed size slots 1 12 2 33 4 5 56 67 87 8
Updated data block
Entire group read compressed and
& uncompressed coalesced into new stripe
Block updated
with new data
New group compressed
& re-written

Other Array Manufacturers Nimble Storage

 Compression Performance Comparison For A Changed Block
Fixed block Architecture
Nimble OS
Variable Blocks
8 blocks Individual blocks
grouped & compressed and
compressed Cost of fixed block architecture relative to Nimble OS: coalesced into stripe
Group placed into
N fixed size slots 1. Additional blocks read from disk
2. Additional CPU cycles for decompression & Updated data block
Entire group read recompression of all N blocks compressed and
& uncompressed 3. Additional M-1 blocks written to disk coalesced into new stripe
Block updated
with new data
New group compressed
& re-written

Other Array Manufacturers Nimble Storage

Scale-to-Fit with the AF-Series
Nimble Storage AF-Series Scale-to-Fit: Scale Out

4 Node AF9000
Cluster
AF9000
300K IOPS Up to 1.4M IOPS
8180TB Capacity

AF7000
230K IOPS
IOPS
based on a
70% Read AF5000
and 30% 120K IOPS
write
workload
SCALE UP

AF3000
50K IOPS

SCALE DEEP

AF3000 and 5000 + 1 AFS2 AF7000 + 2 AFS2 AF9000 + 2 AFS2

 AF-Series Scale Out

SCALE DEEP

AF3000 and 5000 + 1 AFS2 AF7000 + 2 AFS2 AF9000 + 2 AFS2

Scale Deep with the AFS2 Expansion Shelf

The AF-Series AFS2 Expansion Shelf

AFS2 Front View
AFS2 Bezel Panel

Power Button

Power On LED Heartbeat LED

On=PowerOK Always On
Off=No Power
AFS2 shelves have the same
front panel LEDs, except the
SAS-out SAS-in
SAS port LEDs replace the
On=Link On=Link
NIC LEDs Off=No Link Off=No Link

Power Fault Over Temperature

Off=No Fault Off=OK
or No PSU On=System OverTemp
On=PSU Fault, Blink=At least 2 fans failed
No AC, or Internal Fault
 AFS2 Rear View

Path A Path B

Expander FW Status SAS IN port

Green-OK
Red-FW Failed to Load
SAS Link Status
Green-12Gbps Red-Fault
Orange-6Gbps Off-No link
 AF-Series AFS2

– To ensure the most accurate information regarding number of AFS2 shelves supported by the AF-Series
arrays, download the Nimble All Flash Array Configuration Support Matrix from InfoSight.
AFS2 Connections – AF3000/AF5000

 One AFS2 is supported.

 Connect P1 SAS-out from the
head to SAS-in of AFS2
– P2 would work just as well but P1 SAS expansion port (p1) SAS expansion port (p1)
is preferred for consistency

SAS IN port SAS IN port

AFS2 Connections – AF7000/AF9000

 Support up to 2 AFS2.
 First AFS2 - Use P1 SAS
 Second AFS2 – Use P2 expansion
port (p2)
– Do not daisy chain AFS2

AFS2 shelves ship with 1 meter cables. If

you do not order a 3 meter cable you will
need to install one AFS2 shelf above the
controller shelf and one below.
 AFS2 Web GUI

– When an AFS2 shelf is added, it

will need to be activated using
the Activate button.

– Once activated the shelf will be

added to the storage pool.

Once an expansion shelf is

activated, it cannot be
removed from the solution.
Expanding Existing System

1 Fill Expansion Shelf 2 Then balance capacity between them

until capacity utilization
matches the control
head

50% Capacity 50% Capacity

Expansion Shelf Controller Shelf

Power On/Off Order
 On  Off
– Power expansion shelves first, then the – Power off the controller shelf and then
controller shelves the expansion shelves

Controller Shelf

Expansion Shelf
 AF9000
300K IOPS

AF7000
IOPS 230K IOPS AF-Series Scale Up
based on a
70% Read
and 30% AF5000
120K IOPS
write
workload
SCALE UP

AF3000
50K IOPS
Scale Up with Model Upgrades

Model Upgrades
 AF-Series Controller Upgrades
To ensure the most accurate information regarding
Upgrade
controller upgrades for the AF-Series array download the controllers
Nimble All Flash Array Configuration Support Matrix from to AF9000
InfoSight.
Upgrade
controllers
to AF7000

Upgrade
controllers
to AF5000

Upgrade
controllers
to AF3000
Upgrade to the model
Start with an that best meets your
AF1000
new needs
Upgrading Controllers
1 Determine which controller is
currently in Standby mode.
• In WebUI –Select Hardware
• In CLI – use command:
ctrl --list
Upgrading Controllers
2 Label and disconnect all cables running to the Standby controller.

Management A Management B
ETH2 ETH2
Upgrading Controllers
3 Remove the Standby controller.
• Loosen the controller thumb screw.
• Pull the controller release lever out and down.
• Pull the controller straight out.
Upgrading Controllers
4 Remove the onboard USB stick from the old controller and install it into the new controller.

During controller upgrade, you must remove the USB stick from the existing controller and
install it into the new controller. Failure to perform this step will prevent the new controller
from coming online.
Upgrading Controllers
5 Install new controller and reconnect all cables.

6 Verify the controller powers up and is in standby mode.

Management Management
A B
ETH2 ETH2
Upgrading Controllers
7 Perform a failover to the new controller and ensure that the new
controller is in active mode.
• In WebUI – Select Hardware >> click “Make Active” button
• In CLI – use command failover
Upgrading Controllers
8 Confirm the non-upgraded controller is in standby mode and repeat steps 2 – 7 for the opposite
controller:
2. Label and disconnect all cables running to the Standby controller.
3. Remove the Standby controller
4. Remove the onboard USB stick from the old controller and install it into the new controller.
5. Install new controller and reconnect all cables.
6. Verify the controller powers up and is in standby mode.
7. Perform a failover to the new controller and ensure that the new controller is in active mode.
Upgrading Controllers
9 Verify that the model number in the WebUI or CLI reflects the new model number.
 4 Node AF9000
Cluster
Up to 1.4M IOPS
8180TB Capacity

AF-Series Scale Out

 Scale Out with Clustered Arrays
4 Node AF9000  Cluster up to 4 of any
Cluster Nimble Storage arrays
AF9000
125K IOPS Up to 1.4M IOPS
8180TB Capacity
 Simple configuration
 Automated host connection
AF7000 management
270K IOPS
 Dynamic load-balancing and
automated capacity rebalancing
AF5000
140K IOPS
 Linear performance
scaling
SCALE UP

AF3000,
60K IOPS
 Automatic data
SCALE DEEP
migrations
AF3000 and 5000 + 1 AFS2 AF7000 + 2 AFS2 AF9000 + 2 AFS2
Keep Applications Running – No Disruptions!

 Add or remove arrays

non-disruptively
- Automatically rebalances data

 Non-disruptively perform a technology refresh

- Migrates data to upgraded hardware Repurpose older
array for Disaster
Recovery

Exchange + SQL Server VDI VDI VDI

Scale-Out Management - Groups

–Group: Manage arrays from a single

console
– 1-4 arrays per group
– Add and remove arrays on the fly
– Uses a single management IP
–Get status and view performance and
capacity reports
–Create and manage storage pools and
volumes
Scale-Out Pools
Nimble Scale-Out Group
 Pools
Array A1* Array A2 Array A3 Array A4
 Simplify management
L
 Easy to grow or shrink
Pool Pool
- Array is used in only 1 pool at a time
 Can migrate HW (live)
Nimble Scale-Out : Finance-1
 How To Use Array A1* Array A2 Array A3 Array A4
Pooling by L
 Use to scale out apps Application
SharePoint VDI
 Use to simplify allocation to
departments
Nimble Scale-Out: Remote Office-1
Array A1* Array A2 Array A3 Array A4
Pooling by L
Organization
Sales IT/Back Office Apps
Scale Out – Managing Hosts & Paths
Nimble Connection Mgr (NCM)
• Automatic MPIO configuration and path management
• Eliminates manual connection setup to individual arrays
Nimble Connection
Manager or NCS/PSP
plug-in for VMware

NIC 1 NIC 2

Switch 1 Switch 2

NCM Nimble Scale-Out Group