System-Level Diagnostics Guide
System-Level Diagnostics Guide
Contents
Introduction to system‑level diagnostics ..................................................... 4
Requirements for running system-level diagnostics ................................................... 4
How to use online command-line help ........................................................................ 5
Running system installation diagnostics ..................................................... 7
Running system panic diagnostics ............................................................ 11
Running slow system response diagnostics .............................................. 14
Running hardware installation diagnostics .............................................. 17
Running device failure diagnostics ........................................................... 20
Copyright information ............................................................................... 24
Trademark information ............................................................................. 25
How to send comments about documentation and receive update
notifications ............................................................................................ 26
Index ............................................................................................................. 27
4
To run system-level diagnostics, you must already be running Data ONTAP because you need to
reach the Maintenance mode boot option in Data ONTAP. There are several approaches to get to
this option, but this is the recommended approach taken in the procedures documented in this guide.
Some hardware components in your system may require a specific approach, and this would be
documented in the applicable field replaceable unit (FRU) flyer. This guide does not provide detailed
definitions of specific commands, subcommands, tests, or conditions.
Once the command is entered, the tests run in the background and the passed or failed outcome of the
tests is logged in the internal memory-based log, which has a fixed size. Some tests are utilities and
will simply state completed rather than passed or failed. After you run the appropriate tests, the
procedures documented in this guide help you generate status report. Once the test results show a
successful completion of system-level diagnostics, it is a recommended best practice to clear the log.
In the event of test failures, the status reports will help technical support make appropriate
recommendations. The failure could be resolved by reinstalling the FRU, by ensuring cables are
connected, or by enabling specific tests recommended by technical support and then re-running those
tests. If the failure cannot be resolved, then there is a hardware failure and the affected hardware must
be replaced.
There are no error messages that require further definitions or explanations.
General requirements
• Each system being tested must be on a separate network.
The network interface test assigns unique static IP addresses, beginning with [Link], to all
available network interfaces on a storage system. This results in network interface ports on
different storage controllers being assigned the same IP address. If all the systems being tested are
Introduction to system‑level diagnostics | 5
on the same network, then duplicate ip address warning messages appear on the connected
consoles. These warning messages do not affect the test results.
NIC requirements
• All adjacent network interface ports on the system must be connected for best performance using
a standard Ethernet cable.
Examples of adjacent ports are e0a and e0b or e2c and e2d.
Attention: e0M and e0P ports cannot be connected together due to an internal switch
connection. In systems with e0M and e0P ports, the most efficient pairings are e0M with e0a
and e0P with e0b.
• If there are a number of network interface ports on the system, you may need to run the NIC
system-level diagnostic test several times, limiting each run to no more than two pairs each time.
SAS requirements
• When running the SAS system-level diagnostic tests, adjacent SAS ports must be connected for
best performance; storage shelves must be disconnected from the ports.
Note: Connections between adjacent SAS ports is no longer a requirement for systems running
Data ONTAP 8.2; however, only the internal loopback test will be run for systems with
unconnected SAS ports.
FC-AL requirements
• When running the FC-AL system-level diagnostic tests, you must have loopback hoods on FC-AL
interfaces on the motherboard or expansion adapters for best performance; all other cables for
storage or Fibre Channel networks must be disconnected from the ports.
Note: While the use of loopback hoods on FC-AL interfaces are no longer requirements for
systems running Data ONTAP 8.2, the scope of the test coverage on the interface is also
reduced.
CNA requirements
• The use of loopback hoods is not a requirement for running CNA system-level diagnostics tests.
Interconnect requirements
• Both platform controller modules in a dual controller system must be in Maintenance mode for
the interconnect system-level diagnostic test to run.
Attention: You will receive a warning message if you attempt to run the interconnect system-
level diagnostic test with other system-level diagnostic tests.
keyword
Specifies the name of a command or an option that must be entered as shown.
< > (less than, greater than symbols)
Specify that you must replace the variable identified inside the symbols with a value.
| (pipe)
Indicates that you must choose one of the elements on either side of the pipe.
[ ] (brackets)
Indicate that the element inside the brackets is optional.
{ } (braces)
Indicate that the element inside the braces is required.
You can also type the question mark at the command line for a list of all the commands that are
available at the current level of administration (administrative or advanced).
The following example shows the result of entering the environment help command at the
storage system command line. The command output displays the syntax help for the environment
commands.
Steps
Note: You must run this command from the LOADER prompt for system-level diagnostics to
function properly. The boot_diags command starts special drivers designed specifically for
system-level diagnostics.
3. View the version of system-level diagnostics present on your storage system by entering the
following command:
sldiag version show
4. Identify the device types in your new system installation so that you know which components to
verify by entering the following command:
sldiag device types
• fcache is the Flash Cache adapter, also known as the Performance Acceleration Module 2.
• fcal is a Fibre Channel-Arbitrated Loop device not connected to a storage device or Fibre
Channel network.
• fcvi is the Fiber Channel Virtual Interface not connected to a Fibre Channel network.
• storage is an ATA, FC-AL, or SAS interface that has an attached disk shelf.
5. Run all the default selected diagnostic tests on your storage system by entering the following
command:
sldiag device run
Your storage system provides the following output while the tests are still running:
After all the tests are complete, the following response appears by default:
*> <SLDIAG:_ALL_TESTS_COMPLETED>
7. Verify that there are no hardware problems on your new storage system by entering the following
command:
sldiag device status -long -state failed
Example
The following example shows how the full status of the failures is displayed in a test run without
the appropriate hardware:
STATUS: Completed
ib3a: could not set loopback mode, test failed
END DATE: Sat Jan 3 23:11:04 GMT 2009
LOOP: 1/1
TEST END --------------------------------------------
STATUS: Completed
Starting test on Fcal Adapter: 0b
Started gathering adapter info.
Adapter get adapter info OK
Adapter fc_data_link_rate: 1Gib
Adapter name: QLogic 2532
Adapter firmware rev: 4.5.2
Adapter hardware rev: 2
Running system installation diagnostics | 9
ioctl_status.class_type = 0x1
ioctl_status.subclass = 0x3
ioctl_status.info = 0x0
Started INTERNAL LOOPBACK:
INTERNAL LOOPBACK OK
Error Count: 2 Run Time: 70 secs
>>>>> ERROR, please ensure the port has a shelf or plug.
END DATE: Sat Jan 3 23:12:07 GMT 2009
LOOP: 1/1
TEST END --------------------------------------------
c. Verify that you have observed all the considerations identified for
running system-level diagnostics, that cables are securely connected,
and that hardware components are properly installed in the storage
system.
Steps
Note: You must run this command from the LOADER prompt for system-level diagnostics to
function properly. The boot_diags command starts special drivers designed specifically for
system-level diagnostics.
Your storage system provides the following output while the tests are still running:
After all the tests are complete, you receive the following default response:
*> <SLDIAG:_ALL_TESTS_COMPLETED>
5. Identify the cause of the system panic by entering the following command:
sldiag device status -long -state failed
Example
The following example shows how the full status of the failures is displayed in a test run without
the appropriate hardware:
STATUS: Completed
ib3a: could not set loopback mode, test failed
END DATE: Sat Jan 3 23:11:04 GMT 2009
LOOP: 1/1
TEST END --------------------------------------------
STATUS: Completed
Starting test on Fcal Adapter: 0b
Started gathering adapter info.
Adapter get adapter info OK
Adapter fc_data_link_rate: 1Gib
Adapter name: QLogic 2532
Adapter firmware rev: 4.5.2
Adapter hardware rev: 2
ioctl_status.class_type = 0x1
ioctl_status.subclass = 0x3
ioctl_status.info = 0x0
Started INTERNAL LOOPBACK:
INTERNAL LOOPBACK OK
Error Count: 2 Run Time: 70 secs
>>>>> ERROR, please ensure the port has a shelf or plug.
END DATE: Sat Jan 3 23:12:07 GMT 2009
LOOP: 1/1
TEST END --------------------------------------------
Running system panic diagnostics | 13
c. Verify that you have observed all the considerations identified for
running system-level diagnostics, that cables are securely connected,
and that hardware components are properly installed in the storage
system.
Steps
Note: You must run this command from the LOADER prompt for system-level diagnostics to
function properly. The boot_diags command starts special drivers designed specifically for
system-level diagnostics.
Your storage system provides the following output while the tests are still running:
After all the tests are complete, the following response appears by default:
*> <SLDIAG:_ALL_TESTS_COMPLETED>
5. Identify the cause of the system sluggishness by entering the following command:
sldiag device status -long -state failed
Example
The following example shows how the full status of the failures is displayed in a test run without
the appropriate hardware:
STATUS: Completed
ib3a: could not set loopback mode, test failed
END DATE: Sat Jan 3 23:11:04 GMT 2009
LOOP: 1/1
TEST END --------------------------------------------
STATUS: Completed
Starting test on Fcal Adapter: 0b
Started gathering adapter info.
Adapter get adapter info OK
Adapter fc_data_link_rate: 1Gib
Adapter name: QLogic 2532
Adapter firmware rev: 4.5.2
Adapter hardware rev: 2
ioctl_status.class_type = 0x1
ioctl_status.subclass = 0x3
ioctl_status.info = 0x0
Started INTERNAL LOOPBACK:
INTERNAL LOOPBACK OK
Error Count: 2 Run Time: 70 secs
>>>>> ERROR, please ensure the port has a shelf or plug.
END DATE: Sat Jan 3 23:12:07 GMT 2009
LOOP: 1/1
TEST END --------------------------------------------
c. Verify that you observed all the requirements for running system-
level diagnostics, that cables are securely connected, and that
hardware components are properly installed in the storage system.
Resulted in the same test Technical support might recommend modifying the default settings on
failures some of the tests to help identify the problem.
d. After you identify and resolve the problem, reset the tests to their
default states by repeating substeps 1 and 2.
Steps
Note: You must run this command from the LOADER prompt for system-level diagnostics to
function properly. The boot_diags command starts special drivers designed specifically for
system-level diagnostics.
3. Run the default tests on the particular device you added or replaced by entering the following
command:
sldiag device run [-dev devtype|mb|slotslotnum] [-name device]
◦ fcache is the Flash Cache adapter, also known as the Performance Acceleration Module
2.
◦ fcal is a Fibre Channel-Arbitrated Loop device not connected to a storage device or Fibre
Channel network.
◦ fcvi is the Fiber Channel Virtual Interface not connected to a Fibre Channel network.
◦ storage is an ATA, FC-AL, or SAS interface that has an attached disk shelf.
Your storage system provides the following output while the tests are still running:
After all the tests are complete, the following response appears by default:
*> <SLDIAG:_ALL_TESTS_COMPLETED>
5. Verify that no hardware problems resulted from the addition or replacement of hardware
components on your storage system by entering the following command:
sldiag device status [-dev devtype|mb|slotslotnum] [-name device] -long
-state failed
Example
The following example pulls up the full status of failures resulting from testing a newly installed
FC-AL adapter:
STATUS: Completed
Starting test on Fcal Adapter: 0b
Started gathering adapter info.
Adapter get adapter info OK
Adapter fc_data_link_rate: 1Gib
Adapter name: QLogic 2532
Adapter firmware rev: 4.5.2
Adapter hardware rev: 2
ioctl_status.class_type = 0x1
ioctl_status.subclass = 0x3
ioctl_status.info = 0x0
Running hardware installation diagnostics | 19
LOOP: 1/1
TEST END --------------------------------------------
c. Verify that you have observed all the considerations identified for
running system-level diagnostics, that cables are securely connected,
and that hardware components are properly installed in the storage
system.
Steps
Note: You must run this command from the LOADER prompt for system-level diagnostics to
function properly. The boot_diags command starts special drivers designed specifically for
system-level diagnostics.
3. Run diagnostics on the device causing problems by entering the following command: sldiag
device run [-dev devtype|mb|slotslotnum] [-name device]
◦ fcache is the Flash Cache adapter, also known as the Performance Acceleration Module
2.
◦ fcal is a Fibre Channel-Arbitrated Loop device not connected to a storage device or Fibre
Channel network.
◦ fcvi is the Fiber Channel Virtual Interface not connected to a Fibre Channel network.
◦ storage is an ATA, FC-AL, or SAS interface that has an attached disk shelf.
4. View the status of the test by entering the following command: sldiag device status
Your storage system provides the following output while the tests are still running:
After all the tests are complete, the following response appears by default:
*> <SLDIAG:_ALL_TESTS_COMPLETED>
5. Identify any hardware problems by entering the following command: sldiag device status
[-dev devtype|mb|slotslotnum] [-name device] -long -state failed
Example
The following example shows how the full status of failures resulting from testing the FC-AL
adapter are displayed:
STATUS: Completed
Starting test on Fcal Adapter: 0b
Started gathering adapter info.
Adapter get adapter info OK
Adapter fc_data_link_rate: 1Gib
Adapter name: QLogic 2532
Adapter firmware rev: 4.5.2
Adapter hardware rev: 2
ioctl_status.class_type = 0x1
ioctl_status.subclass = 0x3
ioctl_status.info = 0x0
Started INTERNAL LOOPBACK:
INTERNAL LOOPBACK OK
Error Count: 2 Run Time: 70 secs
>>>>> ERROR, please ensure the port has a shelf or plug.
22 | System-Level Diagnostics Guide
LOOP: 1/1
TEST END --------------------------------------------
c. Verify that you have observed all the considerations identified for
running system-level diagnostics, that cables are securely connected,
and that hardware components are properly installed in the storage
system.
Resulted in the same test Technical support might recommend modifying the default settings on
failures some of the tests to help identify the problem.
d. After you identify and resolve the problem, reset the tests to their
default states by repeating substeps 1 and 2.
Copyright information
Copyright © 1994–2017 NetApp, Inc. All rights reserved. Printed in the U.S.
No part of this document covered by copyright may be reproduced in any form or by any means—
graphic, electronic, or mechanical, including photocopying, recording, taping, or storage in an
electronic retrieval system—without prior written permission of the copyright owner.
Software derived from copyrighted NetApp material is subject to the following license and
disclaimer:
THIS SOFTWARE IS PROVIDED BY NETAPP "AS IS" AND WITHOUT ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE,
WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL NETAPP BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
NetApp reserves the right to change any products described herein at any time, and without notice.
NetApp assumes no responsibility or liability arising from the use of products described herein,
except as expressly agreed to in writing by NetApp. The use or purchase of this product does not
convey a license under any patent rights, trademark rights, or any other intellectual property rights of
NetApp.
The product described in this manual may be protected by one or more U.S. patents, foreign patents,
or pending applications.
RESTRICTED RIGHTS LEGEND: Use, duplication, or disclosure by the government is subject to
restrictions as set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer
Software clause at DFARS 252.277-7103 (October 1988) and FAR 52-227-19 (June 1987).
25
Trademark information
Active IQ, AltaVault, Arch Design, ASUP, AutoSupport, Campaign Express, Clustered Data ONTAP,
Customer Fitness, Data ONTAP, DataMotion, Element, Fitness, Flash Accel, Flash Cache, Flash
Pool, FlexArray, FlexCache, FlexClone, FlexPod, FlexScale, FlexShare, FlexVol, FPolicy, Fueled by
SolidFire, GetSuccessful, Helix Design, LockVault, Manage ONTAP, MetroCluster, MultiStore,
NetApp, NetApp Insight, OnCommand, ONTAP, ONTAPI, RAID DP, RAID-TEC, SANscreen,
SANshare, SANtricity, SecureShare, Simplicity, Simulate ONTAP, Snap Creator, SnapCenter,
SnapCopy, SnapDrive, SnapIntegrator, SnapLock, SnapManager, SnapMirror, SnapMover,
SnapProtect, SnapRestore, Snapshot, SnapValidator, SnapVault, SolidFire, SolidFire Helix,
StorageGRID, SyncMirror, Tech OnTap, Unbound Cloud, and WAFL and other names are
trademarks or registered trademarks of NetApp, Inc., in the United States, and/or other countries. All
other brands or products are trademarks or registered trademarks of their respective holders and
should be treated as such. A current list of NetApp trademarks is available on the web.
[Link]
26
Index
C R
comments requirements
how to send feedback about documentation 26 for running system-level diagnostics 4
running
system-level diagnostics 4
D running diagnostics
device failures after device failures 20
running diagnostics after 20 after hardware installations 17
diagnostics after slow system responses 14
running after device failure 20 after system installations 7
running after hardware installation 17 after system panics 11
running after slow system response 14 system-level 4
running after system installation 7 system-level requirements for 4
running after system panic 11
documentation S
how to receive automatic notification of changes to
26 slow system response
how to send feedback about 26 running diagnostics for 14
suggestions
how to send feedback about documentation 26
F system-level diagnostics
failures requirements for running 4
running diagnostics after device 20 systems
feedback running diagnostics after installation failures 7
how to send comments about documentation 26 running diagnostics after panics 11
running diagnostics for slow response 14
H
T
hardware installations
running diagnostics after 17 troubleshooting
device failures 20
hardware installations 17
I requirements for running system-level diagnostics 4
running system-level diagnostics 4
information
slow system response 14
how to send feedback about improving
system installation 7
documentation 26
system panics 11
Twitter
O how to receive automatic notification of
documentation changes 26
online command-line help 5