Page 1 of 6 The Journal of Engineering

This article has been accepted for publication in a future issue of this journal, but has not been fully edited.
Content may change prior to final publication in an issue of the journal. To cite the paper please use the doi provided on the Digital Library page.

Assessment and Analysis of Different Process Bus Redundancy

Networks Performance for IEC61850 Based Digital Substation
L. Xu*, H. Li*, L. Chen*, C. Patterson†, P. Mohapatra†

*The University of Manchester, UK, [Link]@[Link]

†
Scottish Power Energy Networks (SPENs), UK, pmohapatra@[Link]

Keywords: IEC 61850, Process Bus, High-availability outage time. The high cost, high risk associated with outage
Seamless Redundancy (HSR), Parallel redundancy protocol time will reduce the availability and reliability of electricity
(PRP), OPNET. transmission and distribution networks. In addition, the data
communication between different vendor IEDs is often
Abstract vendor specific. Due to lack of interoperability between
different vendor IEDs, it has been difficulty to consider any
To achieve a highly reliable data communication network, new equipment from other vendors during the replacement or
Future Intelligent Transmission Network Substation maintenance of IEDs.
(FITNESS) project has considered three different substation
data communication redundancy networks. These include To address these issues, IEC 61850 standards were released in
process bus with two independent parallel star networks, 2003 [2] with the main aim to promote the interoperability
process bus using a ring network with High-availability among multiple vendors IEDs. To improve the flexibility and
Seamless Redundancy (HSR) protocol, a station bus using a reliability performance, UK National Grid completed
parallel star network with Parallel Redundancy Protocol (PRP) Architecture for Substation Secondary System (AS3) project
protocol. This paper presents the assessment and comparison to standardise digital interfaces between IEDs and primary
analysis of the data flow capability performance for these equipment [3]. However AS3 project proposed two
three different data communication redundancy networks. A independent parallel star networks at process bus level. The
typical 275kV substation with the consideration of these three performance of two independent process buses network are
redundancy networks is modelled and simulated using evaluated and analysed [4]. However, it focused on protection
OPNET a communication network simulation tool. The data control equipment redundancy and the communication
flow capability performance with respect of the network time network redundancy was not considered. Two reliable
delay for the typical protection and control equipment networks with redundancy protocols are published in IEC
requirements (e.g. IEDs and MUs) in one bay are simulated 62439-3, they are High-availability Seamless Redundancy
and analysed. Results show that two independent parallel star (HSR) and Parallel Redundancy Protocol (PRP) [4]. Huu-
networks have the highest data flow capacity. The capacity of Dung [5] have assessed and evaluated the HSR network
PRP network is reduced by a half, while the capacity of HSR performance on process level by OPNET simulation tool. Xi
network is reduced to a quarter. The results also reveal that and Peter [6] designed a test platform in the lab for a process
Ethernet Switch can consume a part of network bandwidth as bus with PRP protocol to access the IEDs interoperbility
well. The work would help power utilities to determine an performance from different suppliers. [7] and [8] present a
optimal number of devices to be installed and to ensure highly substation with the implementation of HSR and PRP
reliable data communication network for protection and protocols respectively. Recently, Scottish Power Energy
control functions in an IEC 61850 based digital substation. Networks (SPENs)’ has considered to deploy a real live full
digital substation for a 275kV substation in Scotland through
UK government Ofgem’ Network Innovation Competition
1 Introduction (NIC) funded Future Intelligent Transmission Network
Multi-function microprocessor based Intelligent Electronic Substation (FITNESS) project [9]. In order to test the
Devices (IEDs) are widely installed and used in substation reliability and data capability performance, FITNESS
automation systems (SAS) to provide measurements, architecture considers all three redundancy networks and will
monitoring and control functions for power systems. The test their performances in a live substation.
IEDs can hardly stay in service for more than 10 years before This paper presents the assessment for process bus network
becoming obsolescent compared with the conventional capability performance and the comparison between the
analogue relays due to fast technology change demand and mentioned three different process bus redundancy networks.
lack of availability of technical support. In contrast the Firstly, the FITNESS architecture with three redundancy
lifespan of primary plants can reach over 40-50 years [1]. networks are presented and explained. The protection scheme
Since the life mismatch between IEDs and primary for the three different process bus redundancy network
equipment, the replacements or maintenance for IEDs to topologies and protocols are modelled and implemented using
connecting and disconnecting primary equipment will require a data communication network simulation software tool,

IET Review Copy Only

 The Journal of Engineering Page 2 of 6
This article has been accepted for publication in a future issue of this journal, but has not been fully edited.
Content may change prior to final publication in an issue of the journal. To cite the paper please use the doi provided on the Digital Library page.

OPENT. The case studies for the network capability The two independent parallel star networks is the most
performance with respect of the system time delay for the straight forward configuration which consists of two
typical protection and control equipment requirements separated star process bus 1 and 2. As shown in Fiig.3, the
(including IEDs and MUs) in one bay are considered and equipment, e.g. IEDs and MUs are connected to their
analysed. In addition the network bandwidth utilization study corresponding process bus 1 or 2, respectively. Failure of any
has been carried out. devices in one of the networks will not affect the operation of
the other network. Therefore, process bus 2 can be regarded
2 Digital Substation Redundancy Networks as a backup solution for process bus 1 failure and vice versa.
Hence the reliability performance for the protection and
2.1 FITNESS Architecture control function can be improved effectively.
Station Bus
Fig.1 shows the single line diagram for a typical UK 275kV
substation, which contains a bus coupler bay, two feeder bays,
a bus section bay and a transformer bay. The protection and MP1 MP2

control scheme in each bay is independent with other bays.

Reserve 1 Reserve 2 ESW1 ESW2
Bus
275kV Section 275kV
Bay
MU1 CBC1 MU2 CBC2
Main 1 Main 2
Instrument Transformer
(CT/VT)

Bus Circuit Breaker

Coupler
Bay Transformer
Feeder Bay Feeder Bay Bay Fig. 3. Overview of Two Independent Parallel Star Networks
1 2 13kV

B. Process Bus using a ring network with HSR protocol

Fig. 1. Single line Diagram of a UK 275kV Substation
As shown in Fig. 4, HSR process bus network basically forms
FITNESS architecture has implemented two feeder bays for a ring network that allows two independent communication
the 275kV substation. The two bay configurations in paths (clockwise and anticlockwise) between the source and
FITNESS architecture is presented in Fig. 2. destination. In order to realize the function of HSR
configuration, devices have to be able to support Doubly
HMI
Attached Node with HSR (DANH)[10]. Devices that are
Station Level unable to support DANH are known as Single Attached Node
Station Bus 1 Station Bus 2 (SAN), SANs cannot connect to HSR network directly and
would require pairing with Redundancy Boxes (RedBoxes) in
order to be transmit message packets in both directions within
MP1 RedBox MP2 MP1 MP2
Bay Level
the ring network. Moreover, for the communication between
Process Bus 1 Process Bus 2 the HSR process bus and the Station Bus, a Redbox is
necessary as well.
MU1 CBC1 MU2 CBC2 MU1 CBC1 MU2 CBC2
Process Level
Station Bus
Vendor A Bay Vendor B Bay
MU: Merging Unit CBC: Circuit Breaker Controller MP: Main Protection

Fig. 2. Overview of FITNESS Architecture MP1 MP2

FITNESS architecture uses two vendors’ IEDs. Each vendor RedBox

takes the responsibility for its own designated bay, but the
bay needs to incorporate the other vendor’s IEDs. Vendor A MU1 CBC1 MU2 CBC2
bay is applying HSR protocol and vendor B considers two Instrument Transformer
(CT/VT)
independent parallel star networks at process bus level and
PRP at station level. Circuit Breaker

2.2 Process Bus Level Network Model Fig. 4. Overview of HSR Network

In structure of FITNESS architecture, each bay consists of C. Process Bus using a parallel star network with PRP
two main protection (MP1 and MP2), two merging units protocol
(MUs) and two circuit breaker controllers (CBCs) which are
connected to the process bus network or the station bus. High availability for the process bus can be provided by two
fully connected parallel star networks with PRP protocols. As
A. Process Bus with Two Independent parallel Star the message packets are duplicated and sent via two star
Networks networks, any one of networks failure will not affect the
protection and control function. Similarly PRP network also

IET Review Copy Only

Page 3 of 6 The Journal of Engineering
This article has been accepted for publication in a future issue of this journal, but has not been fully edited.
Content may change prior to final publication in an issue of the journal. To cite the paper please use the doi provided on the Digital Library page.

requires software functions to support Doubly Attached Node and a CBC via central Ethernet switch. These four devices are
with PRP (DANP) regards as DANP devices. IEDs without modeled using OPNET model library. Continuous SVs are
DANP function are regarded as SAN devices which will generated in MU while GOOSE messages are sent from MP
require a RedBox to be connected into the PRP network, in and CBC. Rapid Spanning Tree Protocol (RSTP) is utilized in
order to transmit two copies of message packets into two process bus with two independent parallel star networks.
separate paths. Fig. 5 shows the overview of the process bus Process bus 1 has been acted as main protection in the
with PRP and DANP devices. substation. Therefore, in case of any Ethernet link
Station Bus
disconnected in process bus 1, a short period of time is
needed for using backup protection (process bus 2). As a
result, the recovery time of RSTP may not meet the IEC
MP1 MP2 61850 SAS communication network performance
requirements.
ESW1 ESW2
destination

MU1 CBC1 MU2 CBC2

Instrument Transformer
(CT/VT)

Circuit Breaker

Fig. 5. Overview of PRP Network

3 Network Modelling
source

3.1 Substation Process Bus modelling

Fig. 6. Single Star Networks Modelling
The process buses with three described redundancy
communication networks described above are modelled and B. Process Bus using a Ring Network with HSR Protocol
simulated, respectively, using OPNET. Modelling
The time triggered message Sampled Value (SV) packets are The process bus using a ring network with HSR protocol
published in MUs which carries a fixed data set containing modelling is presented in Fig. 7, each device model is
voltage and current information from instruction programed and configured as DANH to realize the following
transformers. SV multicast capabilities permits same frame functions. The message frame will be generated and
transmit to multiple IEDs. The IEDs subscribe to the desired duplicated as HSR frame A and HSR frame B in the source
SV streams from the Process Bus. Event triggered message is IED, and these two frames will be circulated into separate
modelled as GOOSE. Since GOOSE trip messages are mainly directions into the HSR ring simultaneously. Intermediate
from IEDs to CBCs, the time requirement for the GOOSE IEDs will receive and forward the frame to the adjacent IEDs
messages is very short. This can ensure the status of circuit within 5 μs [11]. Thus, after propagating in the whole ring,
breaker to be updated immediately. Once a fault is detected the frame will return to the source IED, and then be
by an IED, a sequence of GOOSE messages will be generated discarded. For the destination IED, it will be reached by the
to the CBC via the Process Bus. The CBC will then trip the duplicated frame successively, in case of one frame loss. The
CB in a short time. The packet size and interval time of each first frame is considered to accept by end node and the other
message are shown in Table 1. And 100 Mbps fully duplex copy of frame should be discarded. Therefore, HSR is
Ethernet links are utilized to connect devices in the network. efficient to cope with single point failure on the Ethernet link
and zero recovery time is provided.

destination

Table 1 Message configuration in OPNET

A. Process Bus with Two Independent parallel Star
Networks Modelling
HSR frame B HSR frame A
In this case, two star networks forming process bus 1 and 2
are running separated without affecting the other operation. source

Thus, one of the star network, either process bus 1 or 2 can be

chosen for the process bus capability performance studies. As Fig. 7. HSR Network Modelling
shown in Fig.6, MP is connected with its corresponding a MU

IET Review Copy Only

 The Journal of Engineering Page 4 of 6
This article has been accepted for publication in a future issue of this journal, but has not been fully edited.
Content may change prior to final publication in an issue of the journal. To cite the paper please use the doi provided on the Digital Library page.

C. Process Bus using a Parallel Star Network with PRP Finally the end to end time delay is given 𝑇 = 𝑇𝑎 + 𝑇𝑏 + 𝑇𝑐
protocol Modelling
Time requirements for different messages have been
Fig. 8 presents the simulation scenario for process bus using a illustrated in Table 2. The maximum transfer time of each
parallel star network with PRP protocol. DANP has been message is specified. The time is depended on the message
configured in this network. And DANP has two individual applications within the network. It can be obtained that
communication ports in each device. Two independent ports substation communication network time delay should within
only transmit two copies of frames without receiving and 3ms to avoid message packets delay or even loss.
forwarding. However, all frames will be duplicated and Message Transfer time
Applications
transmit into two separate network as LAN A and LAN B Type (ms)
simultaneously. It should be noted the topologies of LAN A GOOSE 3 Fast Messages (Trip)
and LAN B are not required to be identical. The frame GOOSE 20 Fast Messages (Others)
arriving first at the destination node is received whilst the MMS 100 Medium Speed Messages
later one will be discarded. All traffic will never return to MMS 500 Low Speed Messages
source. In addition, normal Ethernet switches can be utilized SV 4 Raw Data Messages
in this network.
MMS 1000 File Transfer Functions
destination Table 2 Time Requirements for Different Messages [12]

4 Results and Analysis

The simulations have been conducted for several times with
different number of MUs to access and evaluate the
LAN A LAN B
performance of three kinds of process bus network. During
PRP frame A
the simulation, each scenario has been run for 30 seconds.
Results of network time delay and communication network
bandwidth utilization under different situation are presented.
PRP frame B
source
4.1 System Time Delay study
Fig. 8. PRP Network Modelling
Fig.10 shows the system time delay for three redundancy
3.2 IEC 61850 SAS Network Performance Requirement networks with basic protection scheme as 2 MUs connected.
It can be inferred that the time delay for HSR network is
The system time delay is an important indicator to reflect the 0.095ms which is longer than PRP (0.075ms) and two
performance of substation communication network, which is independent parallel star networks (0.065ms). This is because
defined in IEC 61850-5[13]. The message time delay consists message packets need long distance to arrive destination in
of the total time it generated in source IED and collect by ring topology, while message packets are propagated through
destination IED completely through communication network. central Ethernet switch to be received by IEDs in star
In this paper, three layers in the IED transmission stack are topology. And PRP is more complex than star network. In this
considered. As Fig. 9 shows, they are Ethernet layer, TCP/IP scenario, all three networks are performed well.
layer and application respectively.
Ta Tb Tc

Applicaiton Tsa Applicaiton Tda

TCP/IP Tst Communication TCP/IP Tdt

Ethernet Tse Network Ethernet Tde

Source IED Destination IED

Fig. 9. Definition of Message Transfer Time

During the message transmission process, the time delay may

raise in three different stages which are shown in below:
Stage 1: Time delay in the source IED 𝑇𝑎 = 𝑇𝑠𝑎 + 𝑇𝑠𝑡 + 𝑇𝑠𝑒
Fig. 10. Time Delay Performance of three Redundancy
Stage 2: Time delay in the network transmission (Ethernet
Networks with 2 MUs Connected
link, Ethernet switch) 𝑇𝑏
Stage 3: Time delay in destination IED 𝑇𝑐 = 𝑇𝑑𝑎 + 𝑇𝑑𝑡 + 𝑇𝑑𝑒

IET Review Copy Only

Page 5 of 6 The Journal of Engineering
This article has been accepted for publication in a future issue of this journal, but has not been fully edited.
Content may change prior to final publication in an issue of the journal. To cite the paper please use the doi provided on the Digital Library page.

Fig.11 presents the simulation results of system time delay for network bandwidth utilization of HSR network is more than
three networks under increasing number of equipment being doubled compared with other two networks, since the
connected. As can be seen that the system time delay will be message packets will be duplicated and propagated into
increased with more devices (MUs) connected to the network. separate paths for whole ring in the HSR network, the packets
While the relation between system time delay and number of from the source have to return to the source and discard while
MUs is almost linear. According to the network performance the packet from the source will determinate at the receiver. In
requirement, the maximum number of MUs are able to PRP network and single star network, the message packets
connect to each network can be easily found. The capacity of will never return to source IEDs.
HSR network is 8 MUs, and the capacity for the PRP network
Network Bandwidth Utilization (%)
and single star network has doubled as 17 MUs. Therefore, Number of MUs
the capacity of two independent parallel networks can be HSR Network PRP Network Single Star Network
reached up to 34 MUs based on the assumption. In case of 2 21.7 9.3 7.6
one more MU linked to the network, the system time delay
3 30.5 13.6 11.9
will be over 3ms. It is necessary to emphasize the duplicated
5 47.5 22.2 20.1
frames have a strong occupation in the Ethernet link in HSR
network. Therefore, the capacity of HSR network will be 8 73.4 35.4 32.4
limited by these extra data. Although all frames are duplicated 9 82.2 38.9 35.7
in the source IEDs for PRP network, they are transmitted into 12 - 51.7 47.5
two star networks (LAN A and LAN B). Thus, the capacity 15 - 62.4 58.6
could be doubled. Compare with the two independent parallel 17 - 72.8 69.3
star networks, the capacity of PRP network has been reduced 18 - 77.1 75.9
by half. It is because two copies of frames are generated in a Table 3 Network Bandwidth Utilization in different cases
DANP node and propagated into two separate central
Ethernet switches. However, in the two independent parallel 5 Conclusion
star networks, each time a MU will only transmit one frame to
the Ethernet central switch. This paper presents the FITNESS architecture with the three
different redundancy networks including process bus with
two independent parallel star networks with no redundancy
protocol, process bus with a ring network with HSR protocol,
process bus using a parallel star network with PRP protocol.
The methodology for the data flow capability performance
assessment based on a typical the protection and control
equipment requirements is described and discussed. Three
redundancy network models based on process level are
developed in the OPNET simulation tool. The data flow
capability performance for the maximum number of devices
(MUs) to be integrated in a full digital substation are also
evaluated and analysed. Results show the maximum number
of equipment for process bus with two independent parallel
star networks has doubled as 34 MUs in comparing them with
PRP. The process bus with a ring network and HSR protocol
Fig. 11. Time Delay Performance of three Networks with has lowest network capacity as only 8MUs. For the network
number of MUs connected bandwidth utilization study, it can be concluded that DANH
and Ethernet switches will consume a part of network
4.2 Network Bandwidth Utilization Study bandwidth for message packet transferring. The results show
when considers network redundancy protocols to improve the
As the network utilization varies with different number of network reliability, the data transmission capability of the
equipment connected. Table 3 shows the network bandwidth network will be reduced. The results also show that PRP has
utilization in each redundancy network. For HSR network, better performance than HSR. The work will helps future
when 9 MUs have been connected, 82.2% communication digital substation design with considering a trade-off between
channel is occupied but system delay time is far more than the the network reliability, availability, flexibility and the
performance requirements. This is because DANH can protection and control data flow capability performance.
occupy a part of network bandwidth for message receiving
and forwarding. Under this circumstance, message packets Acknowledgements
are queueing to pass through DANH. In the same way, it can
be found in the table that the central Ethernet switches in PRP The author would like to thank to Scottish Power Energy
network and single star network are taking over a part of the Networks (SPENs), United Kingdom to support this work
network bandwidth as well. And the simulation results have through FITNESS project.
revealed that with same number of MUs connected, the

IET Review Copy Only

 The Journal of Engineering Page 6 of 6
This article has been accepted for publication in a future issue of this journal, but has not been fully edited.
Content may change prior to final publication in an issue of the journal. To cite the paper please use the doi provided on the Digital Library page.

References
[1] V. Soni, "A Redundant Power Supply Architecture for
Self-healing Substation using Ultracapacitors,"
International Journal of Research in Engineering And
Technology, July 2014.
[2] IEC 61850, "Communication networks and systems in
substations," 1st ed. 2003.
[3] U. B. Anombem, "Evaluation of IEC 61850 Process Bus
Architecture and Reliability" PhD Thesis, The University
Of Manchester, 2012.
[4] S. Kumar, N. Das, J. Muigai and S. Islam, "Performance
evaluation of data transmission in a single and double bus
network within the utility substation based on IEC
61850," 2014 IEEE PES General Meeting | Conference &
Exposition, National Harbor, MD, 2014, pp. 1-5.
[5] IEC. “Industrial communication networks — High
availability automation networks: Part 3: Parallel
Redundancy Protocol (PRP) and High availability
Seamless Redundancy (HSR)”, IEC 62439-3: 2012, July
2012.
[6] H. D. Ngo, H. S. Yang, D. W. Ham, "An improved High-
availability Seamless Redundancy (HSR) for dependable
Substation Automation System," 16th International
Conference on Advanced Communication Technology,
Pyeongchang, 2014, pp. 921-927.
[7] X. Chen, H. Guo and P. Crossley, "Interoperability
Performance Assessment of Multivendor IEC61850
Process Bus," in IEEE Transactions on Power Delivery,
vol. 31, no. 4, pp. 1934-1944, Aug. 2016.
[8] J. T. Yu, "A practical and effective approach to
implementing High Availability Seamless Redundancy
(HSR)," 2017 IEEE Conference on Dependable and
Secure Computing, Taipei, 2017, pp. 392-399.
[9] J. Chang, B. Vincent and M. Reynen, "Protection and
control system upgrade based on IEC- 61850 and PRP,"
2014 67th Annual Conference for Protective Relay
Engineers, College Station, TX, 2014, pp. 496-517.
[10] P. Mohapatra, "Future Intelligent Transmission Network
Substation (FITNESS)," Scottish Power Energy
Networks (SPENs), 2015.
[11] H. Heine and D. Bindrich, "Designing reliable high-
performance IEC61850 substation communication
networks based on PRP and HSR topologies," 22nd
International Conference and Exhibition on Electricity
Distribution (CIRED 2013), Stockholm, 2013, pp. 1-4.
[12] J. Á. Araujo, J. Lázaro, A. Astarloa, A. Zuloaga and J. I.
Gárate, "PRP and HSR for High Availability Networks in
Power Utility Automation: A Method for Redundant
Frames Discarding," in IEEE Transactions on Smart
Grid, vol. 6, no. 5, pp. 2325-2332, Sept. 2015.
[13] IEC 61850, "Communication networks and systems in
substations, Part 5: Communication requirements for
functions and device models," 1st ed. 2003.

IET Review Copy Only