High Availability Systems

Reference Manual Original Instructions

High Availability Systems Reference Manual

Important User Information

Read this document and the documents listed in the additional resources section about installation, configuration, and
operation of this equipment before you install, configure, operate, or maintain this product. Users are required to familiarize
themselves with installation and wiring instructions in addition to requirements of all applicable codes, laws, and standards.

Activities including installation, adjustments, putting into service, use, assembly, disassembly, and maintenance are required to
be carried out by suitably trained personnel in accordance with applicable code of practice.

If this equipment is used in a manner not specified by the manufacturer, the protection provided by the equipment may be
impaired.

In no event will Rockwell Automation, Inc. be responsible or liable for indirect or consequential damages resulting from the use
or application of this equipment.

The examples and diagrams in this manual are included solely for illustrative purposes. Because of the many variables and
requirements associated with any particular installation, Rockwell Automation, Inc. cannot assume responsibility or liability for
actual use based on the examples and diagrams.

No patent liability is assumed by Rockwell Automation, Inc. with respect to use of information, circuits, equipment, or software
described in this manual.

Reproduction of the contents of this manual, in whole or in part, without written permission of Rockwell Automation, Inc., is
prohibited.

Throughout this manual, when necessary, we use notes to make you aware of safety considerations.

WARNING: Identifies information about practices or circumstances that can cause an explosion in a hazardous environment,
which may lead to personal injury or death, property damage, or economic loss.

ATTENTION: Identifies information about practices or circumstances that can lead to personal injury or death, property
damage, or economic loss. Attentions help you identify a hazard, avoid a hazard, and recognize the consequence.

IMPORTANT Identifies information that is critical for successful application and understanding of the product.

Labels may also be on or inside the equipment to provide specific precautions.

SHOCK HAZARD: Labels may be on or inside the equipment, for example, a drive or motor, to alert people that dangerous
voltage may be present.

BURN HAZARD: Labels may be on or inside the equipment, for example, a drive or motor, to alert people that surfaces may
reach dangerous temperatures.

ARC FLASH HAZARD: Labels may be on or inside the equipment, for example, a motor control center, to alert people to potential
Arc Flash. Arc Flash will cause severe injury or death. Wear proper Personal Protective Equipment (PPE). Follow ALL Regulatory
requirements for safe work practices and for Personal Protective Equipment (PPE).

2 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Table of Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
About This Publication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Download Firmware, AOP, EDS, and Other Files . . . . . . . . . . . . . . . . . . . . 5
Additional Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Chapter 1
High Availability Systems What is High Availability?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Methods to Improve Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Increase Availability with ControlLogix Redundancy . . . . . . . . . . . . . . . . 8

Chapter 2
High Availability Redundant Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
System Components ControlLogix 5580 Redundant Controllers . . . . . . . . . . . . . . . . . . . . . 10
ControlLogix Chassis Power Supplies. . . . . . . . . . . . . . . . . . . . . . . . . . 11
Redundancy Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Communication Modules in a Redundant Chassis . . . . . . . . . . . . . . 12
I/O Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Network Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Network Nodes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Legacy Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
I/O Products. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1756 ControlLogix I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1715 Redundant I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
FLEX 5000 I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1794 FLEX I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1734 POINT I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Field Device Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
FOUNDATION Fieldbus Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
PROFIBUS PA Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Chapter 3
High Availability Networks General Network Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
IT/OT Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Dedicated Upstream Communication for Non-converged Topologies. 23
Upstream Redundant Star Versus Ring Topologies . . . . . . . . . . . . . 24
Network Cable Routing Considerations. . . . . . . . . . . . . . . . . . . . . . . . 25
Recommended Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
DLR I/O Network Topologies and Design Guidelines . . . . . . . . . . . . . . . 28
Direct DLR, Non-converged Topology . . . . . . . . . . . . . . . . . . . . . . . . . 29
Indirect DLR, Non-converged Topology. . . . . . . . . . . . . . . . . . . . . . . . 30
Fiber Segments within DLR Networks . . . . . . . . . . . . . . . . . . . . . . . . . 32
Indirect Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Direct DLR, Converged Topology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 3

 PRP I/O Network Topologies and Design Guidelines . . . . . . . . . . . . . . . 35
PRP, Non-converged Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
PRP, Converged Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Redundant Adapters with ControlLogix Chassis in PRP Topologies . 38

Chapter 4
Additional High Availability Virtualization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Considerations Computer Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Application Level Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Microsoft Domain Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
FactoryTalk Directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
FactoryTalk Server Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
FactoryTalk View Site Edition (SE) HMI Servers . . . . . . . . . . . . . . . . 42
FactoryTalk Alarms and Events Servers . . . . . . . . . . . . . . . . . . . . . . . . 42
FactoryTalk Linx or OPC Data Servers . . . . . . . . . . . . . . . . . . . . . . . . . 42
FactoryTalk ThinManager Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
FactoryTalk Historian Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Redundant ControlLogix Controller Shortcut Paths . . . . . . . . . . . . . . . . 45

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Preface

About This Publication This publication describes guidelines for high availability systems, including
redundant system components, networks, and other hardware and software
considerations.

Download Firmware, AOP, Download firmware, associated files (such as AOP, EDS, and DTM), and access
EDS, and Other Files product release notes from the Product Compatibility and Download Center at
[Link]/pcdc.

Additional Resources These documents contain additional information concerning related products
from Rockwell Automation.
Table 1 - Additional Resources
Resource Description
Controllers Reduce Unplanned Downtime with Logix Redundant Control Solutions, Provides an overview and high-level features of Logix redundant
publication 1756-PP014 controller solutions.
ControlLogix 5580 Redundant Controller User Manual, Describes how to install, configure, program, operate, and troubleshoot
publication 1756-UM015 a ControlLogix® 5580 redundancy system.
Configuration and Implementation User Manual, Provides guidelines and procedures for the implementation and
publication PROCES-UM100 configuration of a PlantPAx® distributed control system.
ControlLogix Redundant Power Supply Installation Instructions, Describes how to install, remove, and troubleshoot a redundant power
publication 1756-IN620 supply system.
ControlLogix 5580 and GuardLogix 5580 Controllers User Manual, Provides information about designing a system, operating a
publication 1756-UM543 ControlLogix or GuardLogix®-based controllers system, and developing
applications.
I/O Redundant I/O System User Manual, publication 1715-UM001 Describes how to install, configure, program, operate, and troubleshoot
a Redundant I/O system.
FLEX 5000 Standard and Safety I/O Modules User Manual, Describes how to use FLEX 5000™ standard and safety I/O modules in
publication 5094-UM001 Logix 5000™ control systems.
FLEX I/O, FLEX I/O-XT, and FLEX Ex Selection Guide, Provides guidance about how to select FLEX™ I/O, FLEX I/O-XT™, and
publication 1794-SG0002 FLEX Ex™ products.
Field devices EtherNet/IP and ControlNet to FOUNDATION Fieldbus Linking Device User Describes installation and operation of the 1788-EN2FFR and
Manual, publication 1788-UM057 1788-CN2FFR linking devices.
FOUNDATION Fieldbus Design, see the FOUNDATION Fieldbus Design Provides design choices and practices for implementing a FOUNDATION
Considerations Reference Manual, publication PROCES-RM005 Fieldbus network with the 1788-EN2FFR and 1788-CN2FFR linking
devices.
EtherNet/IP and ControlNet to PROFIBUS-PA Linking Device User Manual, Describes the installation and operation of the 1788-EN2PAR and 1788-
publication 1788-UM05 CN2PAR linking devices.
Networks Converged Plantwide Ethernet (CPwE) Design and Implementation Guide, Represents a collaborative development effort from
publication ENET-TD001 Rockwell Automation and Cisco® Systems. The design guide is built on,
and adds to, design guidelines from the Cisco Ethernet-to-the-Factory
(EttF) solution and the Rockwell Automation® Integrated Architecture™.
The design guide focuses on the manufacturing industry.
Deploying Device Level Ring within a Converged Plantwide Ethernet Provides design recommendations for connecting device-level
Architecture, publication ENET-TD015 topologies to networks comprised of Layer 2 switches. It also covers the
implementation of embedded switch technology within the Converged
Plantwide Ethernet (CPwE) Cell/Area zone.
Deploying the Resilient Ethernet Protocol (REP) in a Converged Plantwide Describes the implementation of REP for a switch ring topology in the
Ethernet System (CPwE) Design Guide, publication ENET-TD005 CPwE system.
EtherNet/IP Device Level Ring Application Technique, Describes Device Level Ring (DLR) topologies, configuration
publication ENET-AT007 considerations, and diagnostic methods.
EtherNet/IP Parallel Redundancy Protocol Application Technique, Describes Parallel Redundancy Protocol (PRP) topologies, configuration
publication ENET-AT006 considerations, and diagnostic methods.

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 5

Preface

Table 1 - Additional Resources (Continued)

Resource Description
Ethernet Reference Manual, publication ENET-RM002 Describes basic Ethernet concepts, infrastructure components, and
infrastructure features.
ControlLogix EtherNet/IP Network Devices User Manual, Describes how to use ControlLogix EtherNet/IP™ communication
publication 1756-UM004 modules with a Logix 5000™ controller and communicate with various
devices on the Ethernet/IP network.
Stratix Ethernet Device Specifications Technical Data, Provides specifications for Stratix® Ethernet switches and other
publication 1783-TD001 devices.
Virtualization Industrial Data Center: Your Manageable Gateway to Virtualization, Provides an overview of the Industrial Data Center from Rockwell
publication GSMN-PP001 Automation and highlights virtualization features and benefits.
FactoryTalk® products FactoryTalk View Site Edition User Guide, publication VIEWSE-UM006 Describes how to set up and use FactoryTalk View Site Edition features.
FactoryTalk Historian ME Module Configuration Guide, Describes how to configure, use, and troubleshoot hardware and
publication 1756-UM106 software of FactoryTalk Historian ME.
FactoryTalk Historian SE Installation and Configuration Guide, Describes how to install and configure FactoryTalk Historian ME.
publication HSE-IN025
FactoryTalk Alarms and Events System Configuration Guide, Describes how to to install, configure, and use FactoryTalk Alarms and
publication FTAE-RM001 Events services as part of a FactoryTalk-enabled automation system.
FactoryTalk Linx Getting Results Guide, publication LNXENT-GR001 Describes how to install, navigate, and use FactoryTalk Linx.
Safety and security Safety Guidelines for the Application, Installation, and Maintenance of Designed to harmonize with NEMA Standards Publication No. ICS 1.1-1987
Solid-state Control, publication SGI-1.1 and provides general guidelines for the application, installation, and
maintenance of solid-state control in the form of individual devices or
packaged assemblies incorporating solid-state components.
System Security Design Guidelines Reference Manual, Provides guidance about how to conduct security assessments,
publication SECURE-RM001 implement Rockwell Automation products in a secure system, harden
the control system, manage user access, and dispose of equipment.
Wiring Industrial Automation Wiring and Grounding Guidelines, Provides general guidelines for installing a Rockwell Automation
publication 1770-4.1 industrial system.
Product certifications Product Certifications website, [Link]/certifications. Provides declarations of conformity, certificates, and other certification
details.

You can view or download publications at [Link]/literature.

6 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 1

High Availability Systems

Topic Page
What is High Availability? 7
Methods to Improve Availability 8
Increase Availability with ControlLogix Redundancy 8

What is High Availability? Availability is the percentage of time that a system is functioning and able to
perform its mission. High availability is a characteristic of a system that aims
to achieve an agreed level of availability, for a higher than normal period.

Availability is often expressed as a percentage and referred to as the number of

nines. The following table shows how availability percentage, or the number of
nines, equates to possible downtime per year.

Availability % Possible Downtime Per Year

99% 3.65 days
99.9% 8.76 hours
99.99% 52.6 minutes
99.999% 5.26 minutes
99.9999% 30 seconds

Investing in a high availability system over simply an available one can make a
significant difference in lost time or costs due to unscheduled downtime.

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 7

Chapter 1 High Availability Systems

Methods to Improve When considering different methods to improve the availability of a system, it
Availability helps to understand some of the most common causes for unplanned
downtime. Operational errors, hardware, software, and other control system
infrastructure failures are common causes of unplanned downtime.

To improve the availability of a system, implement methods to increase system

reliability and maintainability. Some methods to improve the reliability and
maintainability of a system include the following:
• Redundancy—The most common method to increase availability is to
reduce single points of failure by implementing redundancy. Duplicate
hardware or software components within a system enable you to repair,
replace, maintain, or recover from a failure with little to no disruption to
operation.
• Network design—Design networks to provide a means to detect,
manage, and recover from single or multiple faults without interruption.
• Online updates—Specify control system components that enable you to
change the configuration without disruptions to operation.
• Diagnostics—Specify control system components that can quickly detect
and isolate failures.
• Virtualization—Consolidate computing resources in a way that multiple
operating systems and applications can share one physical server. This
approach reduces the reliance on a hardware-based infrastructure,
which can significantly improve system availability.

Depending on process and system requirements, methods to improve

availability across all areas of a control system are not always economical.
System designers must apply high availability methods that best match their
performance and economic requirements. Manufacturers must determine the
risk of different potential failures and make high availability investments
accordingly.

Increase Availability with This publication focuses on ControlLogix® 5580 redundancy with
ControlLogix Redundancy EtherNet/IP™ I/O networks. Rockwell Automation provides redundancy
solutions for many different components of a ControlLogix system. Product
selection for high availability is dependent on application requirements.

8 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 2

High Availability System Components

Topic Page
Redundant Controllers 9
I/O Networks 13
I/O Products 14
Field Device Interfaces 18

Redundant Controllers A ControlLogix® redundancy system provides greater availability by

establishing redundancy between a pair of controller chassis with identical
specific components. While running, the primary controller chassis detects
changes in data and synchronizes the data with the secondary controller. The
secondary controller is ready to take immediate control if events, such as a
controller fault, occur.

You can enable redundancy by checking the Redundancy Enabled checkbox in

the Studio 5000® Logix Designer application (Figure 1). No further
programming or configuration is required.

Figure 1 - Redundancy Enabled Checkbox

ControlLogix 5580 redundancy is available in the Logix Designer application,

version 33 and later. Benefits of ControlLogix 5580 include the following:
• Redundancy no longer requires separate firmware. You can use the
same firmware revision for standard and redundant systems.
• Redundancy no longer results in memory reduction. With
ControlLogix 5580 redundancy, available memory for redundant
controllers is doubled compared to similarly sized ControLogix 5570
controllers with earlier versions of ControlLogix redundancy.
• Redundancy capability requires no special hardware or separate
catalog numbers. System maintainability is improved by reducing the
variety of spares to manage.

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 9

Chapter 2 High Availability System Components

Certain ControlLogix 5580 controllers also support PlantPAx® 5.0, a modern

distributed control system (DCS). These controllers are designed for process
applications and provide an out-of-the box engineering environment for
process users. Features include the following:
• Redundancy support for systems that require high availability
• No continuous task and four predefined periodic tasks (Fast, Normal,
Slow, and System)
• View of instruction usage per task for optimal task balancing and
visibility, even while offline
• Embedded PlantPAx 5.0 process object library instructions

For more information about ControlLogix 5580 process controllers, see the
Configuration and Implementation User Manual, publication
PROCES-UM100.

The following sections provide high-level design guidance for ControlLogix

5580 redundancy. For more information about ControlLogix 5580 redundant
controllers, see the ControlLogix 5580 Redundant Controller User Manual,
publication 1756-UM015.

ControlLogix 5580 Redundant Controllers

All non-safety ControlLogix 5580 catalog numbers support redundancy. When

planning to use ControlLogix 5580 redundancy, follow these guidelines:
• Place only one ControlLogix 5580 controller in each redundant chassis.(1)
• Do not connect anything to the embedded Ethernet port on the front of
the controller. When redundancy is enabled, the port is disabled and
cannot respond to ICMP ping requests.
• To size a ControlLogix 5580 controller properly, use Integrated
Architecture® Builder. Integrated Architecture Builder is a free tool that
provides advanced selection assistance and a graphical interface for
designing a system. Download the tool from [Link]. When designing a
PlantPAx system, use the PlantPAx System Estimator (PSE). The PSE is a
free wizard that is built into Integrated Architecture Builder.

GuardLogix 5580 controllers do not support redundancy: 1756-L81ES,

1756-L81ESK, 1756-L82ES, 1756-L82ESK, 1756-L83ES, 1756-L83ESK, 1756-L84ES,
1756-L84ESK, 1756-L8SP, 1756-L8SPK.

(1) PlantPAx guidelines also recommend only one controller per ControlLogix redundancy
chassis. Non-PlantPAx ControlLogix 5570 redundancy applications support as many as two
controllers in each redundant chassis. If a non-PlantPAx application requires two controllers
in a redundant chassis, use a ControlLogix 5570 solution.
10 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020
 Chapter 2 High Availability System Components

ControlLogix Chassis Power Supplies

Each ControlLogix chassis supports single or redundant power supplies.

Redundant ControlLogix chassis power supplies reduce single points of
failure, increasing system availability.

Each power supply includes the option to add annunciator wiring to connect
the power supplies to remote input modules. Power supply annunciator wiring
enables quick isolation and detection of failures, which directly impacts
system maintainability. For more information, see the ControlLogix
Redundant Power Supply Installation Instructions, publication 1756-IN620.

Figure 2 - Redundant Power Supplies and Wiring

1756-PA75R or 1756-PB75R Power Supplies

Annunciator Wiring

Redundancy Modules

In each ControlLogix 5580 redundant chassis, there must be one 1756-RM2

redundancy module. Redundancy modules in the primary and secondary
chassis jointly supervise and manage the operating states and transitions of
the redundant controllers and communications modules. This bridge between
chassis facilitates the crossloading of control data and synchronization of
operations.

Redundancy modules support redundant fiber ports for crossloading. The

redundant ports can maximize system availability and eliminate a single point
of failure.

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 11

Chapter 2 High Availability System Components

Communication Modules in a Redundant Chassis

You can use any combination of as many as seven enhanced EtherNet/IP™

communication modules in each redundant chassis. For supported catalog
numbers and firmware revisions, access the product release notes from the
Product Compatibility and Download Center at [Link]/pcdc.

You cannot currently use 1756-EN4TR adapters in ControlLogix 5580

redundant chassis. However, you can use the adapters in a ControlLogix I/O
chassis. In an I/O chassis, you can use one adapter or a pair of adapters in a
redundant configuration. For more information about 1756-EN4TR redundant
adapters in a ControlLogix I/O chassis, see page 14.

12 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 2 High Availability System Components

I/O Networks ControlLogix 5580 redundancy connects with I/O devices through EtherNet/IP
networks only.

Network Communication

All connections that are consumed by a redundant ControlLogix controller

must be multicast. Unicast consumed tags are not supported in redundancy
systems. Redundancy systems do support unicast produced tags.

Network Nodes

ControlLogix 5580 controllers offer a simplified method for counting

controller resources. When designing a ControlLogix 5580 control system,
simply count the number of Ethernet nodes that you plan to include in the I/O
configuration of the controller project. The number of supported nodes for
ControlLogix 5580 controllers depends on the catalog number. For supported
nodes by catalog number, see the ControlLogix 5580 and GuardLogix® 5580
Controllers User Manual, publication 1756-UM543.

Legacy Networks

ControlLogix 5580 redundancy does not support the following:

• ControlNet® networks
• Remote I/O (RIO) networks
• DH+™ networks
• DeviceNet®(1) network
• Explicit messaging to legacy PLC-2®, PLC-5®, or SLC™ controllers

For ControlLogix redundancy applications that require support of these legacy

networks or peer-to-peer communication with these legacy controllers, you
must use a ControlLogix 5570 solution. For more information, see the
ControlLogix 5570 Redundant Controller User Manual, publication
1756-UM535.

(1) The Logix Designer application enables you to build a ControlLogix 5580 redundancy
application with DeviceNet scanners in a remote chassis. However, a warning appears when
verifying the application because DeviceNet outputs are not bumpless during a ControlLogix
5580 redundancy switchover. For that reason, we recommend that you do not use DeviceNet
with ControlLogix 5580 redundancy.
Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 13
Chapter 2 High Availability System Components

I/O Products ControlLogix 5580 redundancy supports most standard and redundant I/O
products. Legacy 1747, 1746, 1771, and 1769 I/O products are not supported. The
following section highlights the most common I/O networks that you can use
with ControlLogix 5580 redundancy.

1756 ControlLogix I/O

ControlLogix I/O modules mount in ControlLogix chassis that are separate

from the ControlLogix redundant controller chassis pair. ControlLogix I/O
modules provide these features to help increase system availability:
• Comprehensive I/O diagnostics for detection of both system and
field-side failures
• Removal and insertion under power (RIUP) lets you replace modules and
make cable connections while the system is in operation
• Electronic keying to help prevent replacement errors
• Conformal coating for durability

Redundant EtherNet/IP Adapters in an I/O Chassis

For added resiliency at the adapter level in a ControlLogix I/O chassis, use
redundant 1756-EN4TR adapters. In a qualified adapter pair, one adapter acts
as the primary and controls the I/O, while the other adapter acts as a secondary
and can take over as the primary if needed. Redundant adapter functionality is
available for 1756-EN4TR adapters with firmware revision 3.001 and later.

Figure 3 - Redundant Adapters

When planning to use redundant EtherNet/IP adapters, follow these

guidelines:
• Redundant adapters must reside in chassis slots 0 and 1 only.
• Each ControlLogix I/O chassis with redundant EtherNet/IP adapters
counts as two Device Level Ring (DLR) nodes towards the recommended
limit of 50 nodes per DLR network.
• A linear topology with redundant adapters is not supported.

For more information about redundant adapter functionality, see the

ControlLogix EtherNet/IP Network Devices User Manual,
publication 1756-UM004.

14 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 2 High Availability System Components

Unsupported Bridged I/O Topologies

When adding ControlLogix I/O to a system, only use non-bridged EtherNet/IP

connections. For more information about unsupported bridged
I/O connections, see Knowledgebase article Logix Platform: Unsupported Bridged
I/O Configurations.

Figure 4 - Bridged Connections

Primary Ethernet/IP Network Secondary EtherNet/IP Network

1715 Redundant I/O

1715 Redundant I/O enables a controller to communicate with a remote,

redundant I/O chassis over an EtherNet/IP network. Each 1715 Redundant I/O
system uses one IP address as the primary IP address for all communication.
The redundant adapter pair consists of two active modules, a primary adapter,
and its partner, a secondary module. For more information, see the Redundant
I/O System User Manual, publication 1715-UM001.

1715 Redundant I/O provides these features to help increase system availability:
• Redundant, partnered EtherNet/IP adapters monitor inputs, outputs,
and diagnostics for I/O modules in the remote chassis. If a fault occurs in
one of the redundant adapters, an IP address switchover occurs and I/O
monitoring and communication to the ControlLogix System continues
without interruption. This switchover is transparent. Status information
available to the ControlLogix application enables you to determine the
status of each 1715-AENTR adapter.
• Support for duplex I/O module pairs provides fault tolerance for module
failures. You can install or remove a partnered I/O module while the
system is operational without impact to the rest of the system.
• Hardware and software diagnostic features indicate when a module fault
occurs. Module status indicators and the Logix Designer application
graphically display faults. Fault messages describe the nature of the fault.
These features show how a module has been affected and what action to
take to resume normal operation.
• Removal and insertion under power (RIUP) lets you replace modules and
make cable connections while the system is in operation.
• Electronic keying helps prevent replacement errors.

IMPORTANT If you use 1715 Redundant I/O with ControlLogix 5580 redundancy, SIL 2
cannot be achieved. Systems that require SIL 2 with 1715 Redundant I/O
and ControlLogix redundancy must use a ControlLogix 5570 redundant
controller solution.

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 15

Chapter 2 High Availability System Components

Figure 5 - 1715 Redundant I/O

Each ControlLogix I/O chassis with redundant EtherNet/IP adapters counts as two
DLR nodes towards the recommended limit of 50 nodes per DLR network.

FLEX 5000 I/O

FLEX 5000™ adapters with firmware revision 5.011 or later offer a rugged,
distributed I/O solution for ControlLogix redundancy systems:
• 1 gigabit (Gb) backplane speed and 1 Gb EtherNet/IP connectivity
provides higher speed and increased bandwidth.
• Two copper or two fiber SFP ports provide flexibility in network
architecture.

FLEX 5000 I/O provides these features to help increase system availability:
• Support for DLR and PRP for fault tolerant network topologies
• Removal and insertion under power (RIUP) lets you replace modules and
make cable connections while the system is in operation
• Visibility to device tag diagnostics based on Device Description (DD) files
for ease of maintenance
• Support for HART 8-channel analog I/O modules with Premier
Integration in the Logix Designer application helps to reduce downtime
with process diagnostics
• Available with conformal coating for durability
• G3 rating for harsh environments

For more information, see the FLEX 5000 Standard and Safety I/O Modules
User Manual, publication 5094-UM001.

16 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 2 High Availability System Components

1794 FLEX I/O

1794 FLEX™ I/O modules offer flexibility with digital, analog, HART analog,
and specialty I/O:
• Removal and Insertion Under Power (RIUP) lets you replace modules
and make cable connections while the system is in operation.
• Adjustable keyswitch helps prevent incorrect module insertion into a
preconfigured terminal base.
• FLEX I/O provides additional savings if system problems develop.
Combining field-wiring terminations and the I/O interface into the same
location saves time and money by making the system easier to maintain
and troubleshoot.

For additional information, see the FLEX I/O, FLEX I/O-XT™, and FLEX Ex™
Selection Guide, publication 1794-SG0002.

1734 POINT I/O

1734 POINT I/O™ modules offer digital, analog, and specialty I/O. POINT I/O
modules provide these features to help increase system availability:
• Modules slide together to provide ease of installation and maintenance
• Removable wiring system saves time during installation and
troubleshooting
• Comprehensive diagnostics
• Removal and Insertion Under Power (RIUP) lets you replace modules
while the system is in operation
• Auto Device Replacement (ADR) reduces downtime
• Available with conformal coating for durability

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 17

Chapter 2 High Availability System Components

Field Device Interfaces Redundant process network interfaces are available for FOUNDATION
Fieldbus and PROFIBUS PA networks.

FOUNDATION Fieldbus Devices

The FOUNDATION Fieldbus network is a protocol that is designed for robust,

distributed process application control. Devices that are connected by a
FOUNDATION Fieldbus network can be used for process control with
seamless data distribution from the H1 device-level network.

Systems communicate with FOUNDATION Fieldbus devices through

EtherNet/IP linking devices. Redundant linking devices provide a fast and
integrated solution for adding FOUNDATION Fieldbus devices to any Logix
platform.

The FOUNDATION Fieldbus H1 linking device provides these features to help

increase system availability:
• Supports multiple levels of device and media redundancy
- Provides two physical H1 ports
- Intelligent Fieldbus Junction Box helps protect the network from both
short and open circuit faults, which improves the overall reliability of
the system
• Supports EtherNet/IP media redundancy
- Hosts two Ethernet ports through embedded Ethernet switch
technology
- Enables the device to be connected in a linear or DLR topology
• Supports master device redundancy
- Enables two identically configured redundant EtherNet/IP to
FOUNDATION Fieldbus linking devices to be connected on the same
H1 network
• Provides all basic diagnostic information in the input assembly
• Simplifies maintenance and troubleshooting through onboard advanced
diagnostics, including a node-specific H1 bus oscilloscope trace, which is
viewable within the Logix Designer application

For more information about the FOUNDATION Fieldbus H1 linking device, see
the EtherNet/IP and ControlNet to FOUNDATION Fieldbus Linking Device
User Manual, publication 1788-UM057.

For more information about FOUNDATION Fieldbus Design, see the

FOUNDATION Fieldbus Design Considerations Reference Manual,
publication PROCES-RM005.

18 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 2 High Availability System Components

PROFIBUS PA Devices

The PROFIBUS PA (process automation) network connects automation

systems and process control systems with field devices such as flow, level,
pressure, and temperature transmitters. The 1788-EN2PAR linking device
provides a fast and integrated solution for adding PROFIBUS PA field devices
to any Logix platform. The linking device provides a direct link between
PROFIBUS PA and EtherNet/IP networks with no intermediate PROFIBUS DP
(decentralized peripherals) layer required.

Features to help increase system availability include the following:

• Supports multiple levels of device and media redundancy
- Provides two physical PROFIBUS PA ports
- Intelligent Fieldbus Junction Box protects the network from both short
and open circuit faults, which improves the overall reliability of the
system
• Supports EtherNet/IP media redundancy
- Hosts two Ethernet ports through embedded Ethernet switch
technology
- Enables the device to be connected in a linear or DLR topology
• Supports Master Device redundancy
• Enables two identically configured Redundant EtherNet/IP to
PROFIBUS PA linking devices to be connected on the same PA network
• Provides all basic diagnostic information in the input assembly
• Simplifies commissioning through onboard advanced diagnostics,
including a node-specific PA bus oscilloscope trace, which is viewable
within the Logix Designer application

For additional information about the 1788-EN2PAR PROFIBUS PA linking

device, see the EtherNet/IP and ControlNet to PROFIBUS-PA Linking Device
User Manual, publication 1788-UM058.

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 19

Chapter 2 High Availability System Components

Notes:

20 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 3

High Availability Networks

Topic Page
General Network Guidelines 21
Recommended Topologies 27
DLR I/O Network Topologies and Design Guidelines 28
PRP I/O Network Topologies and Design Guidelines 35

Before designing a network that includes ControlLogix® redundancy, start

with a set of documented requirements. Application requirements, such as
availability and performance, help drive the choice of resiliency technology
and topology.

General Network Guidelines The following guidelines apply to all networks and provide important
considerations for system availability.

IT/OT Convergence

The convergence of information technology (IT) with operational technology

(OT) within industrial manufacturing has many benefits that directly impact
system availability. For detailed design guidance, see the Converged Plantwide
Ethernet (CPwE) Design and Implementation Guide,
publication ENET-TD001.

A Converged Plantwide Ethernet solution provides these operational benefits:

• Enables convergence of the Industrial Automation and Control Systems
(IACS) network with enterprise networks to improve the flow and
integration of manufacturing information into business systems.
• Enables remote access for engineers, partners, and IACS equipment
vendors for diagnostics and maintenance. Increases efficiency and
response time and enables IACS vendors to provide services to
manufacturers that can have limited subject matter expert resources.
• Helps reduce risk, increase plant uptime, and improve Overall
Equipment Effectiveness (OEE) through validated reference
architectures with a focus on network resiliency and application
availability.
• Helps reduce operating and capital costs by using open standards. Open
standards minimize the need to support multiple protocols in IACS
networks and provide manufacturing companies more options when
purchasing IACS equipment, which can lead to more uptime.

When designing a system with ControlLogix redundancy, consider whether

the application requires convergence down to the I/O networks of the system.

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 21

Chapter 3 High Availability Networks

Non-converged I/O Networks

To reach a non-converged I/O network from the manufacturing zone, network

traffic must communicate across the controller chassis backplane. Only
Common Industrial Protocol (CIP™) can bridge the chassis backplane.

Figure 6 - Non-converged I/O Network

Non-converged
I/O Network

Converged I/O Networks

With a converged I/O network, you can reach the I/O network from the
manufacturing zone without traversing the controller chassis backplane.
Converged I/O networks potentially improve availability by providing direct
access to devices in the I/O networks, which can simplify maintenance.

Figure 7 - Converged I/O Network

Converged
I/O Network

22 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 3 High Availability Networks

Dedicated Upstream Communication for Non-converged Topologies

A ControlLogix redundant chassis can support a maximum of seven Ethernet

modules. In a non-converged solution, follow these guidelines for Ethernet
modules in a redundant chassis:
• Reserve one Ethernet module for upstream communication.
• Use the remaining Ethernet modules to connect to other I/O networks.

For the upstream connection from each ControlLogix redundant chassis,

connect directly to separate switches. Do not connect the redundant chassis
pair to a single shared switch.

Although connecting a redundant chassis pair to a single shared switch is

supported, we recommend that you connect to separate switches for these
reasons:
• Eliminates a single point of failure.
• Reduces the risk of a switchover scenario where both the primary and
secondary chassis have a communication module that transitions to the
lonely state simultaneously. In a lonely state, a communication module
does not recognize any devices on the network. For more information
about the conditions that can cause a switchover, see the ControlLogix
5580 Redundant Controller User Manual, publication 1756-UM015.
Figure 8 - Separate Switches for Upstream Communication

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 23

Chapter 3 High Availability Networks

Upstream Redundant Star Versus Ring Topologies

Redundant star and ring topologies are common design choices for high
availability between controllers and distribution switches. For maximum
network performance and availability, Cisco® and Rockwell Automation
recommend that you plan, design, and implement network topologies that are
based on the redundant star configuration. However, requirements can
dictate the use of other topologies. The following table highlights the key
advantages and disadvantages of the redundant star versus ring topology
solutions.

Redundant Star Ring

• Resiliency from multiple connection failures • Resiliency from loss of one network connection
• Faster convergence to connection loss • When using Resilient Ethernet Protocol (REP), convergence times range from
• Consistent number of hops (typically two in a flat design) provides predictable 50…150 ms
and consistent performance and real-time characteristics • Variable number of hops makes designing predictable performance more
• Fewer bottlenecks in the design reduce the chances of segment over-subscription complex
• Less cabling complexity in certain plant floor layouts
• Multiple paths reduce the potential for over-subscription and bottlenecks

For more information, see the Converged Plantwide Ethernet (CPwE) Design
and Implementation Guide, publication ENET-TD001.

For more information about Resilient Ethernet Protocol (REP), see the
Deploying the Resilient Ethernet Protocol (REP) in a Converged Plantwide
Ethernet System (CPwE) Design Guide, publication ENET-TD005.

24 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 3 High Availability Networks

Network Cable Routing Considerations

When designing a high availability network, it is important to consider

network cable routing. A network design can be single fault tolerant, but cable
and routing designs within the network can potentially lead to a multi-fault
scenario.

In the following example, two networks route between buildings. Although

both single-fault-tolerant networks are identical, there is a risk of damage to
one of the conduits that run between the buildings. By using separate conduit,
as shown in the network on the left, you can achieve a more reliable network.

Figure 9 - Separate Conduit for High Availability

Building 1

Separate Conduit Shared Conduit

Building 2

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 25

Chapter 3 High Availability Networks

Another design consideration applies to multi-paired fiber cables routed

between buildings. For example, consider a logical fiber ring topology of
Stratix® switches that spans multiple buildings. This type of ring topology
commonly has ring connections in and out of a building that are shared
through one multi-fiber pair cable. While this design is valid, a single fault
scenario can become a multi-fault scenario if the ring breaks in two places.

Figure 10 - Separate Two Pair Fiber Cables for High Availability

Central Building

Two Pair Fiber Cables

Remote Building 1 Remote Building 2 Remote Building 3 Remote Building 4

As shown in these scenarios, it is important to consider how cable routing

impacts system availability. These design decisions can significantly affect
how a network handles certain fault scenarios.

26 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 3 High Availability Networks

Recommended Topologies This chapter provides recommended topologies and design guidance for high
availability Ethernet networks that include ControlLogix redundancy. The
recommended topology protocols include Device Level Ring (DLR) and Parallel
Redundancy Protocol (PRP). These protocols provide either single- or multiple-
fault tolerance. The following table summarizes the important differences
between the protocols.

Table 2 - Comparison of DLR and PRP

Attribute DLR PRP
Open DeviceNet® Vendors’ Association International Electrotechnical Commission
Standards organization (ODVA) (IEC)
Multiple-fault tolerance, depending on
Fault tolerance Single-fault tolerance topology or single-fault tolerance in the
worst case
Infrastructure Duplication of architecture not required Duplication of infrastructure required
Twice as many switches as a single
Switches No minimum requirement network
Topology Ring topology Any topology
Switchover time Fast recovery time Zero recovery time

DLR and PRP topologies can be converged or non-converged:

• In a converged topology, the I/O network is connected to the upstream
manufacturing zone.
• In a non-converged topology, the I/O network is isolated from the
upstream manufacturing zone.

DLR topologies can be direct or indirect:

• In a direct DLR topology, the redundant chassis pair is connected
directly to the DLR network.
• In an indirect DLR topology, the redundant chassis pair is not connected
directly to the DLR network.

While other topologies and configurations can be valid, follow these tested
topologies to decrease the risk of network downtime:
• Direct DLR, Non-converged Topology on page 29
• Indirect DLR, Non-converged Topology on page 30
• Direct DLR, Converged Topology on page 34
• PRP, Non-converged Topology on page 36
• PRP, Converged Topology on page 37

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 27

Chapter 3 High Availability Networks

DLR I/O Network Topologies Device Level Ring (DLR) is an EtherNet/IP™ protocol that is defined by the
and Design Guidelines Open DeviceNet Vendors’ Association (ODVA). DLR provides a means to
detect, manage, and recover from single faults in a ring-based network.

A DLR network includes the following types of ring nodes.

Node Description
A ring supervisor provides these functions:
• Manages traffic on the DLR network
Ring supervisor • Collects diagnostic information for the network
A DLR network requires at least one node to be configured as ring supervisor.
By default, the supervisor function is disabled on supervisor-capable devices.
Ring participants provide these functions:
• Process data that is transmitted over the network.
Ring participants • Pass on the data to the next node on the network.
• Report fault locations to the active ring supervisor.
When a fault occurs on the DLR network, ring participants reconfigure themselves and
relearn the network topology.
Redundant gateways are multiple switches that are connected to a single DLR network
Redundant gateways and also connected together through the rest of the network.
(optional) Redundant gateways provide DLR network resiliency to the rest of the network.

Depending on their firmware capabilities, both devices and switches can

operate as supervisors or ring nodes on a DLR network. Only switches can
operate as redundant gateways. To determine if your product supports DLR,
see the associated user manual.

For more information about DLR, see the EtherNet/IP Device Level Ring
Application Technique, publication ENET-AT007.

28 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 3 High Availability Networks

Direct DLR, Non-converged Topology

For a ControlLogix redundancy system, one of the most common designs is a

non-converged I/O network with redundant controller chassis that are directly
connected to the DLR network.

Figure 11 - Non-converged I/O Network with Direct Connection to DLR Network

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 29

Chapter 3 High Availability Networks

Indirect DLR, Non-converged Topology

An indirect DLR, non-converged I/O network topology is similar to a direct

DLR network topology except for one key difference: the redundant controller
chassis are not directly connected to the DLR network.

This topology is less common than the direct DLR, non-converged topology
described on page 29. However, it provides another topology option that can be
beneficial, depending on where the controllers and I/O are physically located.
For example, consider a scenario where the redundant controllers are in a
central location that is a considerable distance from the I/O. With this indirect
DLR topology, 1756 EtherNet/IP fiber modules in the redundant chassis pair
can connect to the remote Stratix switches in the field. This could potentially
reduce the amount of required fiber for the system, depending on how the I/O
is physically distributed throughout a facility.

Figure 12 - Non-converged I/O Network with Indirect Connection to DLR Network

DLR 1, VLAN 10
DLR 2, VLAN 20
DLR 3, VLAN 30

30 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 3 High Availability Networks

Media Converter Considerations

When designing a ControlLogix redundancy system with an indirect DLR,

non-converged I/O network, do not place any media converters between the
ControlLogix redundancy chassis and the Stratix switches in the DLR network.

This recommendation exists because of how redundancy works and the

different causes of a switchover. In this scenario, if the connection between the
media converter and the Stratix switch under the primary chassis fails, a
switchover event does not necessarily occur, even if the chassis cannot detect
downstream I/O. Because the primary chassis still has an active connection
with the media converter, it is not in a lonely state, which is required to trigger
a switchover.

Figure 13 - Indirect DLR, Non-converged Topology

DLR 1, VLAN 10
DLR 2, VLAN 20

DLR 3, VLAN 30

VLAN Considerations

When implementing an indirect DLR topology that includes rings of

DLR-capable Stratix switches mixed with other DLR nodes, the DLR networks
must be on separate VLANs.

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 31

Chapter 3 High Availability Networks

Fiber Segments within DLR Networks

When designing a DLR network, if the distance between two nodes

approaches or exceeds 100 meters in cable length, use fiber between the nodes:
• Ethernet taps can add multimode fiber segments to a DLR network. Note
that 1783-ETAP1F and 1783-ETAP2F modules do not support singlemode
fiber.
• Stratix switches that support DLR can add multimode or singlemode
fiber segments to a DLR network depending on the type of SFP module
installed.

For more information, see the Stratix Ethernet Device Specifications

Technical Data, publication 1783-TD001.

Figure 14 - Fiber Cables for Long Distance Between DLR Nodes

1783-ETAP1F and 1783-ETAP2F modules DLR-capable Stratix switches support

support only multimode fiber. both multimode and singlemode fiber.

IMPORTANT When using an Ethernet tap, know that the device counts as a DLR node,
even if no device is connected to the non-DLR port. Each Ethernet tap
counts against the recommended maximum of 50 nodes per DLR
network. For more information, see the EtherNet/IP Device Level Ring
Application Technique, publication ENET-AT007.

32 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 3 High Availability Networks

Indirect Nodes

When designing a DLR network, you can indirectly connect nodes to a DLR
network through a Stratix switch.

IMPORTANT Devices that connect to non-DLR configured ports of a Stratix switch

have a star topology, which results in the potential for multiple single
points of failure. Potential points of failure include components that are
introduced as part of the network design, such as the switch, cable, or
single port on the device itself.

Devices in Motor Control Centers (MCCs) can be integrated into a DLR

network by integrating MCC switches directly into a DLR network as shown in
Figure 15. If there are multiple Stratix switches within an MCC, we
recommend that you connect each of those switches to the DLR network. Avoid
daisy chaining switches from a single switch to the DLR network as
highlighted in the upper right of the figure.

Figure 15 - Indirect Nodes Connected via Stratix Switch

Stratix Switches in MCC

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 33

Chapter 3 High Availability Networks

Direct DLR, Converged Topology

A direct DLR, converged I/O topology uses DLR gateways to converge the I/O
network with the manufacturing zone. Converged I/O networks can
potentially improve availability by providing direct access to devices in the I/O
networks, which can simplify maintenance workflows.

Figure 16 - Converged I/O Network with Direct Connection to DLR Network

Traffic Through DLR Redundant Gateways

When a DLR redundant gateway switchover occurs, traffic traversing the DLR
gateways can be interrupted. If an application requires peer-to-peer, produce/
consume traffic through DLR redundant gateways, and the application cannot
tolerate the traffic disruption, we recommended that the design keeps
peer-to-peer communication on its own dedicated DLR network.

Figure 17 - Peer-to-Peer Traffic via DLR Redundant Gateways

34 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 3 High Availability Networks

PRP I/O Network Topologies Parallel Redundancy Protocol (PRP) is defined in international standard
and Design Guidelines IEC 62439-3 and provides high availability in Ethernet networks. PRP
technology creates seamless redundancy by sending duplicate frames to two
independent network infrastructures, which are known as LAN A and LAN B.

A PRP network includes the following components.

Component Description
LAN A and LAN B Redundant, active Ethernet networks that operate in parallel.
Double attached node (DAN) An end device with PRP technology that connects to both LAN A and LAN B.
An end device without PRP technology that connects to either LAN A or LAN B.
Single attached node (SAN) A SAN does not have PRP redundancy.
A switch with PRP technology that connects devices without PRP technology to
Redundancy box (RedBox) both LAN A and LAN B.
An end device without PRP technology that connects to both LAN A and LAN B
Virtual double attached node through a RedBox.
(VDAN) A VDAN has PRP redundancy and appears to other nodes in the network as a DAN.
A switch that connects to either LAN A or LAN B and is not configured as a
Infrastructure switch RedBox.

For for more information about PRP, see the EtherNet/IP Parallel Redundancy
Protocol Application Technique, publication ENET-AT006.

To maximize availability and take advantage of the multi-fault tolerant

capabilities of PRP, make sure that redundant LANs have these characteristics:
• Both LANs have resilient topologies, such as redundant star or ring and
not linear or star topologies.
• Both LANs are completely fault independent, such as having separate
power and cabling paths.

All PRP network design guidelines apply to all PRP networks, regardless of
whether there are redundant ControlLogix chassis in the design. For more
information about PRP, including more restrictions and requirements, see the
EtherNet/IP Parallel Redundancy Protocol Application Technique, publication
ENET-AT006.

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 35

Chapter 3 High Availability Networks

PRP, Non-converged Topology

ControlLogix redundancy can interface with PRP I/O networks by using

ControlLogix EtherNet/IP modules that support PRP. The non-converged PRP
solution shown in Figure 18 enables you to isolate the I/O network from the
upstream manufacturing zone.

Figure 18 - Non-converged I/O Network with PRP

LAN A LAN B

36 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 3 High Availability Networks

PRP, Converged Topology

Stratix 5400 RedBoxes can connect a PRP network to the supervisory network.
The following must be Layer 3 routed connections:
• Connections between RedBoxes and distribution switches in the
manufacturing zone
• Connections between RedBoxes

No additional Layer 2 connections can exist on these switches.

For Layer 3 redundancy in the PRP network, you can configure Hot Standby
Routing Protocol (HSRP) on redundant RedBoxes.

In Figure 19, all four Stratix 5400 switches are configured as RedBoxes.

Figure 19 - Converged I/O Network with PRP

LAN A LAN B

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 37

Chapter 3 High Availability Networks

Redundant Adapters with ControlLogix Chassis in PRP Topologies

When incorporating ControlLogix I/O chassis into a PRP topology, you can
increase availability further by using redundant EtherNet/IP adapters.
Redundant adapters eliminate a single point of failure in the ControlLogix I/O
chassis.

Follow these guidelines:

• Use EtherNet/IP modules that support the redundant adapter feature.
• Connect the redundant adapters to RedBoxes as shown in Figure 20.
Figure 20 - PRP Network with Redundant Adapters
1756-EN2TP Module 1756-EN2TP Module

LAN A LAN B

RedBoxes

1756-EN4TR Pair 1756-EN4TR Pair

38 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 4

Additional High Availability Considerations

Topic Page
Virtualization 39
Computer Hardware 40
Application Level Redundancy 40
FactoryTalk Server Redundancy 42
FactoryTalk ThinManager Redundancy 43
FactoryTalk Historian Redundancy 44
Redundant ControlLogix Controller Shortcut Paths 45

Virtualization Virtualization enables you to run multiple virtual computers, operating

systems, and applications on one physical server. These virtualization features
can significantly benefit system availability:
• Less physical hardware—Virtualization significantly reduces the number
of physical devices to maintain, which reduces the overall system
administration burden.
• Fast recovery from disasters—Virtualized computing resources are
easier to supply and deploy. Because they support cloning, a replica can
be ready for use in minutes rather than hours. Downtime due to disaster
recovery is more agile. Rather than replace, repair, or clean a damaged or
infected computer, you can build a new instance and recover its contents
from your backup solution.
• Failover modes—Virtualization platforms often have different failover
modes, which can reduce downtime and increase availability. For
example, VMware has a high availability feature that automatically
restarts a virtually machine if it stops running, which minimizes
downtime. The fault tolerance feature in VMware provides continuous
availability to virtual machines by continuously replicating them from
one physical server to another.
• Software upgrades—With the virtual environment, if something goes
wrong while applying a patch or update, you can revert a virtual machine
back to its previous state before you applied the patch.
• During a planned outage of a properly designed system, administrators
can shift their workloads so that a physical server can stop running with
no impact to the system. When the planned outage is complete, the
server can be placed back into service. Because virtualization is not
attached to a physical computer, migration between servers is possible
while the system remains running.

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 39

Chapter 4 Additional High Availability Considerations

Computer Hardware Most control systems rely on computer hardware. When multiple virtual
machines run on one physical host, a hardware failure can have a broader
impact than a 1:1 server-to-application model. By improving the reliability of
computer infrastructure, you can help to maintain a continuously available
processing environment. Some common ways to improve system reliability
with computer hardware are as follows:
• Uninterruptible power supplies (UPS)—Provide emergency power to a
load when the input power source or main power fails.
• Network Interface Card (NIC) Teaming—Provides fault tolerance or load
balancing.
• Redundant Array of Independent Disk (RAID)—Computer hard drive
virtualization technology that combines multiple physical disk drive
components into one or more logical units for data redundancy,
performance improvement, or both.

For more information about pre-engineered Industrial Data Center (IDC)

solutions, which include high availability and fault tolerance features, see
Industrial Data Center: Your Manageable Gateway to Virtualization,
publication GSMN-PP001.

Stratus, a Rockwell Automation Encompass™ Partner, offers hardware

solutions that help prevent instances of unplanned downtime in the data
center. For more information, see Stratus Technologies: Simplifying
Availability of Critical Applications, publication ENCOMP-BR014, or visit the
Stratus website.

Application Level Server applications that are not configured for redundancy introduce
Redundancy potential single points of failure. Both Microsoft® and Rockwell Automation
offer ways to increase system availability with application level redundancy.
System designers must determine the level of application level redundancy
that best match their application and economic requirements.

Microsoft Domain Controllers

FactoryTalk® Services in a distributed environment depend on these critical
functions to operate:
• Name resolution
• Time synchronization
• Authentication

If any of these three functions are unavailable or perform poorly, then the
entire distributed system can be adversely impacted. To provide these
functions in a centralized and secure manner, use a Windows® domain that is
implemented to optimize performance and resiliency. For systems with more
than 10 machines, a domain is required to minimize effort, centralize
administration, and provide additional security than a Windows workgroup
can provide.

For all high availability architecture designs, we strongly recommend that you
have redundant domain controllers. Place the domain controllers in a
replicating pair close to and on the same network segment as the
manufacturing zone.

40 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 4 Additional High Availability Considerations

FactoryTalk Directory

The FactoryTalk Directory role provides a central lookup service for all
FactoryTalk products in an application. The FactoryTalk Directory references
tags and other system elements, such as screens, from multiple data sources
and makes the information available to clients through a lookup service.

If the network directory becomes unavailable while client computers are

connected to an application, the clients use a local copy of the directory and
continue to run correctly. There is no need to restart previously connected
clients. Those clients can continue to resolve tag addresses, read and write tag
values, acknowledge alarms, and open graphic displays, even if the tags and
displays that the clients require were never used before. When the network
directory is available again, all dependent clients in the system resume using
the directory automatically.

Although you can locate the FactoryTalk Directory on the same host as another
FactoryTalk server component, such as a primary HMI server or Historian SE
server, this is not a best practice for high availability. To reduce risk and
maximize availability, we recommend that the FactoryTalk Directory role is
located on a separate host apart from any other FactoryTalk component for
these reasons:
• System startup—It is best if the FactoryTalk Directory is the first
component to start and the last to stop. Because most FactoryTalk
software products rely on various services that the FactoryTalk Directory
provides, the lowest risk scenario is to make directory available as these
products start up.
• Compatibility—While all CPR9 versions are generally compatible, the
FactoryTalk Directory is sometimes required to be at the highest version
of FactoryTalk Services Platform installed in the system. This
requirement can interfere with another FactoryTalk component if the
FactoryTalk Directory is co-located with another product.
• Patching/upgrading—Maintenance of the FactoryTalk Directory role
hosted on a dedicated computer translates to minimum system
downtime because it does not affect the operation of other FactoryTalk
components while restarting.
• Redundancy—In systems with other redundant FactoryTalk servers, it is
best if the FactoryTalk Directory remains available during any failover
scenarios. While the redundant server pair can function without the
FactoryTalk Directory, the lowest risk scenario is to keep it available.

For more information, see the following:

• Knowledgebase article FactoryTalk Services Platform Best Practices.
• FactoryTalk View Site Edition User’s Guide, publication
VIEWSE-UM006.

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 41

Chapter 4 Additional High Availability Considerations

FactoryTalk Server In a FactoryTalk View SE network distributed application, you can configure
Redundancy redundancy on paired computers for critical FactoryTalk components.
FactoryTalk server redundancy can be configured for the following products.

FactoryTalk View Site Edition (SE) HMI Servers

You can configure HMI redundancy in only network distributed applications.
One computer in the pair hosts the primary server, and the other hosts the
secondary (backup) server.

If the primary HMI server cannot provide service to application clients, the
system automatically switches to the secondary server, and FactoryTalk View
SE clients continue to function normally. There is no need to restart client or
active server computers. For more information, see the FactoryTalk View Site
Edition User Guide, publication VIEWSE-UM006.

FactoryTalk Alarms and Events Servers

To minimize disruptions to FactoryTalk Alarms and Events clients if a

FactoryTalk Alarms and Events server becomes unavailable or fails, you can
configure FactoryTalk Alarms and Events server redundancy. FactoryTalk
Alarms and Events server redundancy and data server redundancy are
supported only in network station applications and network distributed
applications. For more information, see the FactoryTalk Alarms and Events
System Configuration Guide, publication FTAE-RM001.

FactoryTalk Linx or OPC Data Servers

To minimize disruptions to data server clients if a data server becomes
unavailable or fails, you can configure data server redundancy. Similar to
FactoryTalk View SE HMI and FactoryTalk Alarms and Events server
redundancy, if a primary data server cannot provide service to application
clients, the system automatically switches to the secondary data server. For
more information, see the FactoryTalk Linx Getting Results Guide, publication
LNXENT-GR001.

42 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 4 Additional High Availability Considerations

FactoryTalk ThinManager ThinManager® is a content management system that provides secure

Redundancy configuration and delivery of content to any combination of device, user, and
location.
Centralized servers host applications.

Control Room
HMI HMI CMMS HTTTP ERP MES

Applications
& Content
ThinManager Deliver sessions to multiple Access Feeds from
monitors and virtual screens USB & IP Cameras
Application
on one thin client.
Delivery Provides secure configuration and content delivery
to any combination of device, user, and location.

PanelView Plus
Deliver content to the right person
Users at the right time and place.
Remote Desktop to HMI
VersaView 5200
Assigns content to users. Industrial Thin Clients
Applications follow a user Manage & deliver
Deliver applications based virtual desktops
from terminal to terminal.
on what is assigned to the while running PCs
terminal or user. Get mobile access to applications
as a thin client.
specific to user roles.
Offices

ThinManager can increase system reliability by minimizing or eliminating

unscheduled downtime in these ways:
• Supports fully redundant servers. If the system has two ThinManager
servers that are synchronized with the same configuration, the thin
clients can receive their configuration from either ThinManager server.
If the first ThinManager server fails, a thin client can connect to the
backup ThinManager server and receive the proper configuration and
connect to the proper terminal servers.

In a high availability system that uses Microsoft Remote Desktop

Services (RDS) as a content source, such as FactoryTalk View SE thin
clients, we highly recommend that at least two RDS servers are available
in a primary/standby configuration. If a primary server fails, a thin client
running ThinManager core software automatically detects the failure
and switches to an available secondary. ThinManager can also maintain a
secondary session. If a failure occurs, the secondary session appears in
the operator view, typically within one second.
• Provides a simplified method for providing physical Ethernet
redundancy for a ThinManager managed client as long as the thin client
has two network ports. If a network path from the thin client to the
server fails, the secondary path is used. For Ethernet redundancy, the
only network design requirement is to have separate cable routings with
different switches for each routing.

For more information about how ThinManager can increase system

availability, see LEARNING SERIES: ThinManager Benefits vs. a Standard
Microsoft Unmanaged Solution.

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 43

Chapter 4 Additional High Availability Considerations

FactoryTalk Historian Data loss prevention can be one of the most critical considerations when
Redundancy designing a system that requires high availability. FactoryTalk Historian SE
provides separate methods for data collection redundancy and data
availability redundancy.

For high availability to data, follow these practices:

• If your system requires minimal data loss, use redundant FactoryTalk
Historian SE interface nodes to provide redundant data collection
capabilities. By using redundant interfaces, you can update FactoryTalk
Historian SE software without any data loss. The interface node role is
commonly installed on the same computers as the FactoryTalk Linx data
server role.
• If your system requires high availability to data for historical trending or
reporting, use redundant Historian SE Servers, known as a Collective.

Clients
User workstation connects to
any available data source.

Collective Secondary Configuration Changes Primary

Historian Historian

Data Building Data Building

Services Services
Redundant Interfaces
Historian Historian
Interfaces Interfaces

• When designing a FactoryTalk Historian architecture, consider the

physical location of the data servers in reference to the data sources. To
reduce the risk of a network failure resulting in data loss, place data
collectors as close as possible to the data sources.

One way to minimize the path from a ControlLogix® data source and a
data collector is to use Historian ME modules. You can place Historian
ME modules in a non-redundant ControlLogix chassis that is located as
close as possible to the data source. Like interface nodes, Historian ME
modules can buffer historical data if the module temporarily loses
communication with the Historian SE server. The Historian ME module
forwards its buffered data to the Historian SE server once
communications is established.

For more information about the Historian ME module, see the Historian ME
Module Configuration Guide, publication 1756-UM106.

For more information about FactoryTalk Historian SE high availability

architectures, see the following:
• FactoryTalk Historian SE Installation and Configuration Guide,
publication HSE-IN025
• Knowledgebase article FactoryTalk Historian SE Collectives

44 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Chapter 4 Additional High Availability Considerations

Redundant ControlLogix When configuring the connection between a FactoryTalk Linx data server and
Controller Shortcut Paths a redundant ControlLogix controller, you can configure redundant shortcut
paths to the primary and secondary controllers. These shortcut paths help to
reduce data server communication recovery time during a redundancy
switchover. Data server communication recovery time applies to any software
that uses tag data, such as HMI displays, data loggers, alarm systems, or
historians.

Redundant controller shortcut paths are available starting with FactoryTalk

Linx version 6.00.00. To retain communication when a redundancy switchover
occurs, you can configure two shortcut paths to the primary and secondary
Logix 5000™ controllers in a ControlLogix redundancy system, revision 31.5x.
For details about how to implement this feature, see the FactoryTalk Linx
Getting Results Guide, publication LNXENT-GR001.

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 45

Chapter 4 Additional High Availability Considerations

Notes:

46 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 Index

Numerics DLR
about 28
1734 POINT I/O 17 direct, non-converged topologies 29
1756 ControlLogix I/O 14 fiber segments 32
1756 Redundant I/O 15 indirect nodes 33
1756-EN4TR adapters 12, 14 indirect, non-converged topologies 30
1756-RM2 redundancy modules 11 PRP comparison 27
redundant gateways 34
1794 FLEX I/O 17 VLAN considerations 31
domain controllers 40
A
adapters E
1756-ENT4TR 12, 14 enable redundancy 9
FLEX 5000 16
EtherNet/IP adapters,1756-ENTR 14
annunciator wiring 11
explicit messaging 13
availability
methods to improve 8
percentage 7 F
FactoryTalk Alarms and Events 42
B FactoryTalk Directory 41
bridged I/O topologies 15 FactoryTalk Historian 44
FactoryTalk Linx 42, 45
FactoryTalk Server 42
C FactoryTalk Services 40
cable routing 25 FactoryTalk ThinManager 43
chassis, redundant 12 FactoryTalk View SE 42
CIP 22 fault tolerance
communication modules 12 multiple 27
computer hardware 40 single 27
conformal coating 14, 16, 17 fiber ports, redundant 11
consumed tags 13 fiber segments 32
controller resources 13 field device interfaces 18
controllers FLEX 5000 I/O 16
ControlLogix 5570 10 FOUNDATION Fieldbus devices 18
ControlLogix 5580 10
GuardLogix 5580 10
ControlLogix 5570 redundancy 10 G
ControlLogix 5580 redundancy 9 - 12 gateways, redundant 34
ControlLogix chassis 11 GuardLogix 5580 controllers 10
ControlNet network 13
converged plantwide Ethernet 21
converged topologies 22, 27, 34, 37 H
hardware, computer 40
Historian SE servers 44
D HMI redundancy 42
data loss prevention 44
data servers 42, 44, 45
DeviceNet network 13 I
DH+ network 13 I/O products
direct DLR topologies 29 1715 Redundant I/O 15
distributed control system 10 1734 POIN I/O 17
1756 ControlLogix 14
1794 FLEX 17
FLEX 5000 16
indirect nodes 33
interfaces, field device 18

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 47

 L redundant system components
adapters 14
legacy networks 13
chassis 12
linking devices 18, 19 controllers 9
fiber ports 11
HMI 42
M I/O 15
MCCs 33 power supplies 11
process network interfaces 18
media converter considerations 31
ThinManager servers 43
messaging, explicit 13 remote I/O network 13
modules ring topologies 24
communication 12 routing, cable 25
I/O 14
redundancy 11
multicast 13 S
multiple-fault tolerance 27
server applications 40
servers, data 42, 44, 45
N servers, ThinManager 43
network communication 13 shorcut paths 45
network guidelines 21 single-fault tolerance 27
network nodes 13 Stratix switches 30, 31, 32, 33, 37
networks, legacy 13
nework cable routing 25 T
NIC teaming 40
tags, produced and consumed 13
nodes
teaming, NIC 40
indirect 33
network 13 ThinManager 43
non-converged topologies 22, 23, 27, 30, 36 topologies
number of nines 7 bridged I/O 15
converged 22, 27, 37
direct DLR, converged 34
P direct DLR, non-converged 29
DLR 27, 28
paths, shortcut 45 indirect DLR, non-converged 30
percentage, availability 7 non-converged 22, 23, 27, 36
PlantPAx 10 PRP 27
PlantPAx System Estimator 10 PRP, converged 37
PRP, non-converged 36
power supplies, redundant 11 redundant star 24
produced tags 13 ring 24
PROFIBUS PA devices 19
PRP
about 35 U
converged topologies 37 unicast 13
DLR comparison 27 upstream
non-converged topologies 36 communication 23
redundant adapters 38
redundant star topologies 24

R V
RedBoxes 37
virtualization 39
redundancy modules 11
VLAN considerations 31
redundancy, ControlLogix 5580 8, 9, 10
redundant gateways 34
redundant shortcut paths 45 W
redundant star 24 wiring, annunciator 11

48 Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020

 High Availability Systems Reference Manual

Rockwell Automation Publication HIGHAV-RM002A-EN-P - September 2020 49

Rockwell Automation Support
Use these resources to access support information.

Technical Support Center Find help with how-to videos, FAQs, chat, user forums, and product notification updates. [Link]/support
Knowledgebase Access Knowledgebase articles. [Link]/knowledgebase
Local Technical Support Phone Numbers Locate the telephone number for your country. [Link]/phonesupport
Literature Library Find installation instructions, manuals, brochures, and technical data publications. [Link]/literature
Product Compatibility and Download Center Download firmware, associated files (such as AOP, EDS, and DTM), and access product [Link]/pcdc
(PCDC) release notes.

Documentation Feedback
Your comments help us serve your documentation needs better. If you have any suggestions on how to improve our
content, complete the form at [Link]/docfeedback.

Waste Electrical and Electronic Equipment (WEEE)

At the end of life, this equipment should be collected separately from any unsorted municipal waste.

Rockwell Automation maintains current product environmental compliance information on its website at [Link]/pec.

Allen-Bradley, CIP, ControlLogix, DH+, Integrated Architecture, FactoryTalk, FLEX, FLEX 5000, FLEX Ex, FLEX I/O-XT, GuardLogix, Logix 5000, PlantPAx, PLC-2, PLC-5, POINT I/O,
Rockwell Automation, Rockwell Automation Encompass, Rockwell Software, SLC, Stratix, and Studio 5000 Logix Designer, and ThinManager, are trademarks of Rockwell Automation, Inc.
CIP, CIP Sync, ControlNet, DeviceNet, and EtherNet/IP are trademarks of ODVA, Inc.
Cisco and Cisco Systems are trademarks of Cisco Systems, Inc.
Microsoft and Windows are trademarks of Microsoft Corporation.
Trademarks not belonging to Rockwell Automation are property of their respective companies.

Rockwell Otomasyon Ticaret A.Ş. Kar Plaza İş Merkezi E Blok Kat:6 34752, İçerenköy, İstanbul, Tel: +90 (216) 5698400 EEE Yönetmeliğine Uygundur

Publication HIGHAV-RM002A-EN-P - September 2020

Copyright © 2020 Rockwell Automation, Inc. All rights reserved. Printed in the U.S.A.