Communication Requirements for a Hybrid VSC Based HVDC/AC Transmission Networks State Estimation

Abstract

The communication infrastructure of the modern Supervisory, Control and Data Acquisition (SCADA) system continues to enlarge, as hybrid High Voltage Direct Current (HVDC)/Alternating Current (AC) networks emerge. A centralized SCADA faces challenges to meet the time requirements of the two different power networks topologies, such as employing the SCADA toolboxes for both grids. This paper presents the modern communication infrastructure and the time requirements of a centralized SCADA for hybrid HVDC/AC network. In addition, a case study of a complete cycle for a unified Weighted Least Squares (WLS) state estimation is tested on a hybrid HVDC/AC transmission network, based on Voltage Source Converter (VSC). The cycle estimates the elapsed times from the sensors up to the SCADA side, including the data acquisition and the WLS processing times. The case study is carried out on the Cigre B4 DC test case network with 43 virtual Remote Terminal Unit (RTU)s installed and 10 data concentrators, all connected through a fiber-based communication network. It is concluded that the time requirements can be fulfilled for a hybrid HVDC/AC network.

1. Introduction

The need for a supervisory and control system known as SCADA is still required as the power networks are evolving. The SCADA system is a complete structure of hardware components, communication equipments and software toolboxes [1,2,3,4]. The software components are available in the SCADA operator room, which are connected to substations and end-side devices through a Wide Area Network (WAN) communication.

The research on enhanced SCADA systems has become essential, mainly due to the integration of HVDC and low-carbon technologies such as wind and solar generation. The hybrid HVDC/AC networks require faster, robust and complex SCADA systems [5]. The recent researches have discussed the SCADA hardware and software upgrades and modifications. New communication infrastructures are proposed to cover AC and DC requirements [5,6,7] and new DC side RTUs are currently being characterised and researched [8,9,10]. A complete modern SCADA communication infrastructure can be split into four layers as shown in Figure 1a [4,7,11]:

• Power System Layer: it is the lower end layer. It covers all the electrical units; generators, transformers, converters, feeders, and collector buses.
• Data Acquisition and Monitoring Layer: contains sensors and actuators represented by RTUs. This layer reports measurements in different forms, data rates, and frequency to the communication layer. This layer has circuit breaker controllers and several power protection devices.
• Communication Network Layer: The third layer and the backbone of the SCADA system. It connects the three main levels of the system, as described below and shown in Figure 1b:
  • Controller area network: is the smallest communication network inside the RTU. It is responsible for connecting the microcontrollers with their slaves (sensors and actuators).
  • Power area network: is a narrow area network, positioned above the controller area network and provides a stable connection between the main Master Terminal Unit (MTU) and the connected RTUs. Multiple communication protocols are used depending on the characteristics of the controller type.
  • Station area network: is a station WAN, and it provides the communication link between the power area network and the SCADA command center.
• Human Machine Interface (HMI) Layer: consists of several softwares and graphical user interface applications that support the system operators. Usually, the interface has different menus, options, and screens for each layer of the SCADA.

The main challenge appears in the timescale of the modern SCADA communication network due to the intensive integration of low-carbon technologies. New time requirements are required for a stable and robust SCADA system. Figure 2 shows the bigger picture of the timescales requirements in a modern hybrid HVDC/AC network SCADA [12,13,14,15]. The different timescale requirements between the traditional AC networks and the HVDC demands an update to the communication system. It has to be faster and able to handle more control units and data bandwidth [16,17,18]. In addition, the traditional messaging protocols (such as IEEE C37.118 [19] and IEEE 1646 [18]) have to be restudied to include HVDC messages characteristics.

Furthermore, a heavy load on the communication infrastructure has emerged due to the large number of sensors for each RTU, especially in wind farm generation [20,21,22]. However, a combination between stageable data concentration and back-haul communication mechanism can improve the performance of the centralized approaches [22]. Besides, the distributed control approach of large AC/DC power systems is still under research, and one of the biggest challenges in this field is the global variables [5,13]. The question remains on how to share these global data between different SCADAs, data such as the frequency in the AC system and the voltage in the DC side. Several distributed computing solutions have been proposed such as the Epidemic Propagation [23], but it has not been tested on real SCADA for power systems yet.

This paper aims to estimate the time required by a SCADA system to perform a complete cycle of unified state estimation for a hybrid VSC-HVDC/AC network. The process starts from the RTU sensing time up to transmitting data through the communication network and the estimation processing time. The paper presents if the total time consumed is within the acceptable standards by simulating a case study based on the Cigre B4 network.

The structure of the paper is as follows—Section 2 goes through a detailed review of SCADA components and structure. Section 3 explores the time requirements of modern RTUs and the required upgrades for HVDC integration. Section 4 presents a simulation of a case study to illustrate the time elapsed to complete a unified WLS state estimation cycle. Section 5 concludes the work of this paper.

2. SCADA Components and Structure

In the following subsections, each component of the SCADA is reviewed.

2.1. Remote and Master Terminal Units

2.1.1. Remote Terminal Units

RTUs are considered the slaves of the master units. RTUs are more advanced than Programmable Logic Controllers (PLC)s and considered an extended version of it. They contain sensors, actuators, and a control unit. The RTUs can transmit synchronized data to the master terminal in real-time from the network, periodically or on-demand requests [24,25].

RTUs are geographically distributed, usually with Global Positioning System (GPS) modules installed to label time to the measurements (time-synchroniser). Several technologies can be employed as a communication medium such as radio, Fiber optic, and microwaves [26], the network can be Local Area Network (LAN) or WAN. A generic RTU includes:

• Controllers or PLC are embedded systems with multiple digital and analog Input/Output (I/O) ports, memory, communication interface, Analog-to-Digital (A/D) and Digital-to-Analog (D/A), instrumentation amplifiers and signal filters;
• Sensors: comes with a driver to convert and process the signal, for example, Current Transformer (CT) or Voltage Transformer (VT) with 4–20 mA transducer;
• Actuators: comes with isolated drivers, used to trigger circuit breakers or relays;
• HMI: (Not mandatory) a small-scale interface, screen or display.

Usually, the functional block programming (International Electro Technical Commission (IEC) 61131-3) is used to build up PLCs and RTUs programs. Also, Windows embedded 5.0/6.0 or Unix can be used as an operating system.

The RTUs are presented in the power network under different titles; Phasor Measurements Unit (PMU)s, and Intelligent Electronic Devices (IED)s. In industry, Phasor Data Concentrator (PDC)s (IEEE C37.244) are refereed as a superior RTUs, where data are collected and aggregated from RTUs in a synchronized multi-staged structure. The standard IEC 61850 defines the automation and communication structure between substations, within substations and RTUs [27]. The standard provides constrains for data acquisition, protection and data exchange between different types of RTUs [17]. IED is an emerging technology that is reconfigurable and extendable, since it is based on Field-Programmable Gate Array (FPGA) technology. There are three different types of IEDs; circuit breaker (CB) only, merging unit (MU), and protection and control (P&C) [9,28]. MU-IEDs are used for data acquisition from current and voltage transformers (sensors), while CB-IEDs are used to monitor and control the circuit breaker states [28]. P&C-IEDs combine the protection and data acquisition features with an extended I/O ports [11,17]. With the integration of the HVDC technology, the IEDs became the generic name for the measuring and control devices in AC and DC systems [29].

2.1.2. Master Terminal Units

MTUs are considered the second control unit after the command center and usually installed in substations. MTUs have similar characteristics of the main station, like HMI, servers and they are connected to other MTUs through a communication network, for example, LAN/WAN. The HMI allows the operators to control and monitor the connected RTUs through predefined procedures and some visual presentation of geographical and textual information and shapes [16,25,26].

2.2. Communication Infrastructure

The communication infrastructure is the backbone of the SCADA system, and it is a function of the coverage area, the selected medium and protocols. Different technologies have been already used in the SCADA field such as radiowaves, serial modems, LAN/WAN and Transmission Control Protocol (TCP)/Internet Protocol (IP) [2,3,4]. The common SCADA communication topologies are: Point to Point (PTP), Point to Multiple (PTM), and Multiple to Multiple (MTM) [30,31,32]. The SCADA communication networks have gone through several generations presented below in chronological order:

• Monolithic (1st Generation): this generation was missing the network and queuing systems, such as LAN/WAN protocols. SCADA systems were operated independently and individually on minicomputers. This generation was known for having multiple redundancy systems [30,33].
• Distributed (2nd Generation): the SCADA architecture was a low-traffic semi-real-time LAN network. It connects master stations and remote terminal with the command room. As a result, the reliability has improved, and the processing time and failure rate have been reduced. However, no standards were designed for the LAN protocols, and the security of the protocol was not guaranteed [30,33].
• Networked (3rd Generation): is an extension of the 2nd generation, and it is used widely today. It was introduced to override the protocol’s limitations. It consists of a centralized command center with multiple remote stations connected in one network with a single application interface to access all the stations. This generation breakthrough is the integration of the IP and TCP protocols. This improvement increased network security and extended the accessibility to the I/O devices. The system’s reliability has also been enhanced due to duplicated/redundant components and strong WAN technology. Despite these improvements, there is an uprising risk on system cyber-security due to internet protocols [4,30,33].
• Internet of Things (IoT) (4th Generation): is an under research generation, it came with the so-called Industry 4.0 revolution. Researchers claim that it will be suitable for low voltage distribution networks and microgrids [34]. It has the advantage of dealing with big data using simpler communication equipment; it uses cloud infrastructure and Wi-Fi technology. Also, it proposes cheaper and simpler controllable devices to replace the complicated IEDs and measuring units. Although this generation claims higher stability and robustness system, others stand against it due to privacy and security concerns. IoT is expected to add data aggregation, predictive/prescriptive analytics and unified data structure for the the different SCADA applications [33,34,35].

2.2.1. Communication Mediums

Some of the common communication mediums in the transmission level are described below, along with their advantages and disadvantages [25,30,31,32].

• Fiber Optic cables: is considered the best solid medium for long-distance communication (140+ km), and it is in continuous enhancement since the 1970s. The commercial fiber wire has reached a signal attenuation of less than 0.3 dB/km, and the optical detectors and data modulators have archived a higher accuracy [36].
  The simplified fiber optic cable is built from three layers, a nanometer-diameter glass core, a glass cladding, and a protective sheath (plastic jacket). The cables are manufactured to handle one of the following light propagation technologies: multi-mode graded-index, multi-mode step-index, and single-mode step-index. The single-mode fiber optic can handle a distance of 60+ km with speed up to 10 Gbps, while multi-mode covers lower distance with data rate up to 40 Gbps [37,38].
  The Fiber optic cables are used widely in power systems SCADA, three major modified fiber optic cables are used [36]:

  (a)

  Optical Power Ground Wire: is a cable used between transmission lines, either underground or overhead.

  (b)

  All-Dielectric Self-Supporting: is a completely dielectric cable designed to work alongside the conductor part of the HV transmission lines/towers.

  (c)

  Wrapped Optical Cable: is used in power transmission and distribution lines. Usually, it is wrapped around the grounding wire or the phase conductor.

  Fiber optic technology is considered a secured and straightforward medium, but with high installation and maintenance costs. Underground or offshore installations require complicated permissions compared to other communication mediums. The most two fiber optic standard technologies in power systems are: Synchronous Optical Networking and Synchronous Digital Hierarchy [36,39].
• Power Line Carrier communication (PLCC) (analog/digital): is power-dependent communication technology, it uses the power transmission lines as a communication medium. The analog version is still used up-to-date; it can handle two communication channels, and transmit voice, data, and SCADA commands. The medium is reliable, secure, and uses frequency signals between 30 kHz and 500 kHz, with a baud rate of 9600. PLCC was installed and tested on 220/230, 110/115 and 66 kV power lines [40,41,42]. This technology can cover distance of 200 m with data rate >1 Mbps, or distance >3 km with data rate between 10 and 500 Kbps [37,43]. The digital PLCC is an under research technology. It provides higher reliability and at least 4 channels with higher data rate/speed. It is expected to be less affected by the electrical noise, and more secured than the analog PLCC [44].
• Satellites: are considered the preferred communication technology for the graphically separated receiver and transmitter terminals; it is widely used in remote access systems [30,32]. The signal is sent from one earth-stationary terminal through a special antenna (dish) pointing towards the satellite. The satellite receives, amplifies and retransmits the signal towards another earth-stationary terminal. The antenna has a special low-noise amplifier and works on C-band and Ku-band frequencies. The C-band covers 5.2625–26.0265 GHz uplink and 3.26–4.8 GHz downlink, while Ku-band covers 12.265–18.10 uplink and 10.260–13.25 GHz downlink [30].
  State of the art in satellite communications is the Very Small Aperture Terminal technology. It provides a smaller and cheaper antenna under the KU-band, but the communication can be affected by natural phenomena, for example, solar equinox [25,30].
• Microwave Radio: is part of the ultra-high frequency technologies that operate on frequencies higher than 1 GHz or as a multichannel medium in lower frequencies. Microwave technology was invented as an analog communication medium with high data rates and secured multichannel capabilities [25,30]. Despite the improvements on the analog microwave, such as complexity reduction, the digital microwave leads with cost reduction and high communication flexibility. Also, it provides a higher data rate, new communication protocols, and standards. WiMax technology is one of the ultra-high frequency microwaves, and it can cover a distance up to 50 km with speed up to 75 Mbps [30,37].
  The microwave technology has two main network topologies; the PTP and PTM. The PTP is a directional long-distance communication link, while PTM is an omnidirectional communication that can be structured as star or tree networks [25,30]. PTM is more suitable for on-demand channels. Both topologies are power-independent, but the line of sight between the communications nodes must always be guaranteed.
• Omnidirectional Wireless/Cellular: is part of the radio based technologies which operated in specific frequency bandwidths.

  Table 1. Wireless/Cellular technologies.

  Technology	Channel Bandwidth	Latency	Data Rate	Cell Size
  ZigBee	0.3–2 MHz	<100 ms	250 kbps	<100 m
  Wifi	22 MHz	<100 ms	54 Mbps	<100 m
  2G	0.2–1.25 MHz	300–750 ms	64 kbps–2 Mbps	<20 km
  3G	1.25–20 MHz	40–400 ms	2.4–300 Mbps	<10 km
  4G	<100 MHz	40–50 ms	<3 Gbps	<10 km
  5G	<100 GHz	<=1 ms	>3 Gbps	<1 km

  shows the most common Wireless/Cellular communication technologies along with their specifications and data speeds [39,45,46]:
  The advantage of ZigBee over traditional Wi-Fi is the ability to connect a higher number of cell nodes (approx. 60k instead of 2k). Cellular networks can provide high speed, low latency, and stable communication networks for power systems. However, the security side is still doubtable [39,45]. Further visions are available in [47] for the 6th generation (6G).

2.2.2. SCADA Communication Protocols

Communication protocols are a list of rules, structures, and limitations that control the transfer speed, security, and reliability of the information. The communication in a SCADA system starts in two steps. First, the MTUs initialize the connection, identify all the connected RTUs and perform ping test. Second, all remote access stations initialize the connection with the MTUs and allocate themselves.

SCADA systems use different types of communication protocols which are maintained by several standards such as Modbus, IEC 60870-5-101/104, IEC 61850 and Distributed Network Protocol (DNP3) [6,26]. Some of these protocols can deal with TCP/IP, which provides higher security and on-demand requests. However, it is always recommended that SCADA system should not be connected to the internet to avoid any DOS or cyber attacks [48]. The most common communication protocols for SCADA are listed and described below:

• Modbus: One of the most used protocols in SCADA systems, provides real-time communication using the Open Systems Interconnection (OSI) seven layers model [25,49]. Modbus protocol has four operating modes between the main station and the remote station. It converts the transmitted requests into a protocol data unit (PDU: function code and data request). The data linking layer in the OSI converts the PDU into an application data unit (ADU), which the receiver side can understand. Modbus is considered as a serial communication tool and usually use RS-232, and RS-485 modems [31]. Also, it can be extended to deal with TCP/IP protocols through a new layer to generate the PDU’s “encapsulation” [49].
• DNP3: is a protocol developed by IEC based on a simplified version of the OSI called the Enhanced Performance Architecture (EPA) model, as shown in Figure 3. DNP3 is widely used in SCADA systems to establish a connection between multiple MTUs and RTUs. DNP3 can handle low bandwidth serial and IP communication modems, or TCP/IP over internet connection (WAN) [7,50].
• IEC 60870-5: the protocol is based on EPA model and was developed by IEC. An additional layer is added called the User Application layer. It is a front-end layer that can be used to set up the functions of the telecontrol system to interact with the SCADA hardware devices [30]. There are several versions of the protocol 60870-5-(101 to 104), each with different data objects, functions codes, and specifications. The most generic one is 101, which has basic definitions of the data objects, the geographical areas, and the WAN technology. The 104 version has two additional layers from the OSI model to deal with TCP/IP protocols (Figure 4) [25,30]. This protocol is used in power transmission and distribution SCADA’s, with a data transfer speed of 64 kbits/s depending on the interface of the IEC 60870-5 [30].
• IEC 60870-6: is known as Inter-Control Center Communications Protocol (ICCP), globally used for telecontrol of SCADA. TASE.2 version of the standard has 5–7 layers of the OSI model. It is used to connect command centres of SCADA together, and exchange measurements, time-tagged data and events [51].
• IEC 61850: is considered the most widely adopted protocol in substations automation [27], especially for connecting multiple RTUs/IEDs. The protocol is based on the OSI model and it overcomes the DNP3 by providing higher bandwidth communication and real-time protection and control [27]. The communication architecture of IEC 61850 contains 3 hierarchical levels [7]:

  (a)

  Station Level: contains the main station HMI and computers;

  (b)

  Bay Level: is presented by the different RTUs, such as IEDs and PMU.This level is connected to the upper level through the Station Bus;

  (c)

  Process Level: is the RTU’s terminals such as sensors (CT/VT) and actuators (circuit breaker). This level uses the Process Bus to communicate.

  The IEC 61850 provide five Ethernet/IP communication services, as shown in Figure 5, which makes it superior over IEC 60870-5; these services are (1) Abstract Communication Service Interface (ACSI), (2) Generic Object Oriented Substation Event (GOOSE), (3) Generic Substation Status Event (GSSE), (4) Sampled Measured Value multicast (SMV), and (5) Time Synchronization (TS). The GOOSE and GSSE are the main two services for data and events exchange. GOOSE provides the master/slave multicast messaging and allows the transfer of binary, integers, and analog values from the slave nodes. However, GSSE is more restricted to binary status data. The TS service ensures that all IEDs work in a synchronized timestamp. This process can be implemented through GPS or the IEEE 1588 Precision Time Protocol [27,52]. ASCI is responsible of the system reports and status logging while SMV service transfer sampled analog data and binary status to the IED’s Bay level via the Process bus [27].
• Profibus or the “Process field bus”: is an OSI model protocol used in the industrial monitoring environment. It has a bus control unit that establishes the connection between the different hardware equipment. Usually, it has a type D connector, 127 points of data, speed up to 12 Mbps, and a data message size up to 244 bytes per node [31]. There are three standard versions of the Profibus; Field bus Message Specification, Distributed Peripheral, and Processes Automation. The Distributed Peripheral version data speed can vary between 93.75 kbps and 12 Mbps for a distance of 1.2 Km and 100 m, respectively [31].
• Highway Addressable Remote Transducer (HART): is a protocol developed by Rosemount for smart communication with sensors and actuators. It provides smart digital techniques to collect data and send commands. HART is a hybrid protocol since the physical layer can deal with 4–20 mA scheme and frequency-shift keying for analog and digital communication, respectively. Therefore, it is commonly used between PLCs and RTUs. [31,53]. The protocol has provided higher speed and efficiency, and simplified the maintenance and diagnostics processes. HART protocol can be operated in two topologies: PTP and PTM [31].
• Modbus+ (plus): is proposed to overcome the limitations of the Modbus protocol. The Modbus+ was developed to establish a LAN connection between master stations. Also, it allows the “token” approach with unique addresses for each station in the network up to 64 addresses. Therefore, an additional special cable is used to transmit data, but this came with a price that Modbus+ is not suitable for real-time communication. Modbus+ transceivers are polarities independent, and usually twisted pair cables are used, with an additional shield wire [31].
• Data Highway (DH)-485 and DH+: are standard protocols for Allen Bradley. The general specifications of these protocols are Peer to Peer (P2P), half-duplex LAN, up to 64 nodes, and 57.6 kbaudbit. DH+ supports the “token” approach (floating master), and it is based on 3 layers of the OSI model; the application, the data link, and the physical layers. The DH-485 is a RS-485 serial master/slave protocol [31].
• Foundation Fieldbus: is a protocol based on the OSI model with an extra user application layer, which provides a convenient interface for users. Fieldbus protocol has less wire costs, higher data integrity, and compatibility with several industrial vendors. In addition, it uses HART protocol, making it compatible with analog and digital signals. The protocol can handle up to 32 self-powered or 12 powered devices through the communication link, with 31.25 kb/s transfer speed [31].

2.3. Operator Room

The operator room of the SCADA system has a user interface unit known as HMI, which provides a visual presentation of the hardware and the software components of the SCADA. The system operator can control and monitor the whole system from this room through telecommunication and computerized services. The HMI is connected to a powerful computing architecture that can handle complicated operations, big database and perform tests and simulations before executing commands [1,3,4]. The HMI can be either vendor restricted or compatible with other vendors hardware and software [26].

3. RTUs and Communication Networks Requirements for HVDC/AC

The modern SCADA system for the AC/DC network are presented in Figure 6.

3.1. The HVDC/AC RTUs

The demand on advanced digital RTUs for HVDC opens the door for researchers to investigate the characteristics of an HVDC side RTUs. The challenge, in this context, is to ensure new RTUs (e.g., IEDs) comply with both AC and DC time requirements. In most of the HVDC systems, the internal control procedures do not require communication with a higher-level control system. However, hierarchical control of the converter’s voltages, currents and powers is needed [54]. Besides, the control of multi-type converters DC grids could have different RTUs specifications [14].

On the AC side, there are several RTUs available; most of them have been already standardized. Synchrophasor is one of the major ones; it provides real-time measurements such as phase and frequency. A Synchrophasor block can contain PMUs, multi-stage PDCs, a communication medium and a synchronized clock (by GPS) [15,27].

PDCs are used to collect time-synchronized data from multiple PMUs. Centralized PDCs are commonly used in multi-stage data concentration (hierarchical network) [15]. The PMU is one of the main RTUs in the AC transmission networks, PMUs aim to improve the availability of the system states and maintain system stability [55]. They act 100 times faster than the SCADA by analysing the measured data in real-time and activating built-in/pre-programmed protection functions. The decision-making process (including the reading of data from sensors) inside the PMUs takes between 10 and 100 ms, while PDCs can take 100 ms and 1 s [56].

On the DC side, researchers from industry and academia are modifying the AC side RTUs to meet DC side requirements. Intensive research has been established on the AC/DC RTUs requirements.

Table 2. AC and DC RTUs characteristics and measurements.

	Characteristics	Measurements
AC	I/O: −8 digital input and 8 to 32 analog input with 16–24 bits A/D resolution and sampling frequency of 4 or 8 kHz for a 50 Hz system [57]. −16 to 264 digital output for large scale protection RTU [58].	Voltages, Currents. Active and reactive power/energy. Power factor. Frequency, rate of change of frequency (ROCOF), total vector error (TVE) and the frequency error (FE) [19]. Digital inputs (e.g., CB).
	IEC 61131-3 based or FPGA processor
	Transfer rate: 64 Kbps to 2 Mbps
	Time-Synchronization based on global navigation satellite system or GPS. The standard acceptable error is in range of ±500 nanoseconds [19].
	Reporting rate based on IEC/IEEE 60255-118-1 are {10, 25, 50, 100} frame per second (fps) for 50 Hz system, and {10, 12, 15, 20, 30, 60, 120} fps for 60 Hz [19].
DC	I/O: −32 analog input with 24 bits A/D resolution. −16 to 264 digital output for large scale protection RTU (ABB [8] Open Source [29]).	Voltages, Currents. Real power/energy. Digital inputs (e.g., CB).
	Sensors with Very high sampling frequency (in range of 100 kHz) [57,59,60,61,62,63] due to the voltage fluctuations (approx. 1.6% per minute [13])
	Sensing time interval is within 30 ms. Transmitting time interval between RTU and MTU is within 10 ms [24].

shows a comparison between the characteristics and the collected measurements for AC and DC sides.

At the sensor level, the HVDC RTUs require fast sampling rate (bandwidth) for protection reasons, which some of the traditional AC side sensors cannot meet.

Table 3. Conventional and non-conventional sensors [57].

Category	Type	Bandwidth	Suitable for
Current Transformers/sensors	Electromagnetic (iron-core)	in kHz	AC
	Rogowski coil Integrated with optical signal	in MHz	AC/DC
	Fibre optic CT	in MHz	AC/DC
	Zero-flux (Direct Current or Hall-effect CT)	in hundred kHz	DC
Voltage Transformers/sensors	Inductive/Capacitive VT	in kHz	AC
	Compensated RC-divider	in MHz	AC/DC
	Fibre optic VT	in MHz	AC/DC

shows most of the known conventional and non-conventional sensors [28,57,59,64]. The non-conventional sensors are digital instruments that support fast response and high data bandwidth (from 10’s of kHz to a few MHz) [59]. A novel HVDC contactless magnetic-field sensor CT and VT are proposed in [60,61].

Researchers in [10] proposed an HVDC measuring unit based on PMUs and IEC 61850 protocol, named SynDC. They modified the AC measurement message standard (IEEE C37.118) to add new parameters to differ between AC and DC measurements. The paper also recommends that the time-synchronization error between the HVDC measurement units should not exceed 20 microseconds. Also, the system status update rate with the local command center should be within the 1-s. A recent publication on low cost open source IED is proposed in [29], with the following specifications:

• Friendly user interface, flexible and expandable to meet future requirements.
• Provided state-of-the-art in HVDC protection with a swift reaction. The papers refer to speed requirements for the DC side circuit breakers to act in 2 ms with a sampling rate >50 kHz.
• I/O ports are: (16 analog and 26 digital) Inputs/28 digital Outputs

3.2. The HVDC/AC Communication Network

3.2.1. Inter-Control Center Communications Protocol ICCP

Modern power networks are controlled by different electric utilities (SCADAs), as shown in Figure 7. Therefore, there is a need for an ICCP data exchange agreement (IEC 60870-6/TASE.2) to be defined. It will enable data exchange between the different AC and DC network operators. However, the lack of a standardized version of the protocol has limited and restricted the communication between the network’s operators [54]. ICCP can establish an unified SCADA interface of a hybrid network with several unified toolboxes, for example, hybrid AC/DC state estimator. In addition, it will increase the power availability and reliability during abnormal conditions and blackouts [65,66].

Despite that, the ICCP is still undefined (not standardized) for communication networks between AC and DC grids. Transpower has proposed its own agreement for AC and DC RTUs data exchange; their main ICCP characteristics are [68]:

• DNP3 protocol is used, with the maximum data rate (speed);
• The communication network is serial optical fiber, RS232 Master/Slave architecture;
• The data exchange scheme is defined as shown in

  Table 4. ICCP data exchange scheme for Transpower.

  I/O Type	Data Slots	Description
  Analog	5	Voltages, Currents, Active & Reactive Power and Fault Location
  Flags	10–17	Protection and Trips
  Control	2	Circuit Breakers control
  Status	10–11	Circuit Breakers status

  .

3.2.2. Time Requirements

In HVDC systems, the communication network timescale requirements are still not fully defined in a standard. However, recent publications have estimated these time ranges based on similar standards or experiments. For instance, in

Table 5. Communication timing for wide area power network under IEC 61850 standard.

Protocol	Message Application	Delay Tolerance (ms)
GOOSE	Fast trips	3–10
	Fast commands/messages	20–100
	Measurements/Parameters	100–500
SMV	Raw data	3–10
TS	Station bus	1
	Process bus	0.004–0.025
(Reset)	File Transfer	≥1000
	Low–Medium speed	100–500

, the IEC 61850 standard can be adapted to control the timing constrains of HVDC communications [7].

However, The delivery timing from HVDC RTUs to the SCADA side is not fully clarified by IEC 61850. Therefore, several publications have recommended the IEEE P1646 standard to be used instead. In

Table 6. Data exchange timescale & size requirements (AC based on IEEE P1646 vs. DC).

Data Type	AC Substations (S)		DC Substations (S)		Data Size [7]
Information	Within${\mathbf{S}}_{\mathbf{i}}$ (ms)	Between ${\mathbf{S}}_{\mathbf{i}}$and${\mathbf{S}}_{\mathbf{j}}$(ms)	Within${\mathbf{S}}_{\mathbf{i}}$ (ms)	Between ${\mathbf{S}}_{\mathbf{i}}$and${\mathbf{S}}_{\mathbf{j}}$(ms)	Range
Error Time Synchronization	<0.1 [52], <2 [19]	-	<0.020 [10]	<1000 [10]	Bytes
Protection	5/4 for 50/60 Hz (1/4 cycle)	8–12, 5–10 [28]	0.1–0.5 [62,69]	3–4 [62,69]	10 s of Bytes
Monitoring and control	16	1000	10 [70]	250–500 [70,71]	10 s of Bytes
Operation and Maintenance	1000	10k	1000	10k	100 s of Bytes
Text Data	2000	10k	2000	10k	KB to MB
Files	10k–60k	30k–600k	10k–60k	30k–600k	KB to MB
Data Streams	1000	1000	1000	1000	KB to MB

the AC substations values are obtained from [18]. There are some amends made by researchers such as in [28] However, the DC substations values are gathered from several recent publications, as shown in the table. Some data types have the same delivery time in AC and DC systems.

4. Case Study: HVDC/AC State Estimation Time Requirements

The case study aims to find the total elapsed time for a state estimation cycle (from sensors to SCADA) for an HVDC/AC network. The Cigre B4 network, RTUs, Data concentrators, and SCADA are shown in Figure 8. 43 virtual RTUs are installed in different locations, as shown in Figure 8b. These RTUs are collecting data from power lines, converters, and generators and transmit them to 10 data concentrators (Figure 8c). The centralized SCADA is connected to 9 data concentrators (Figure 8d). The communication medium of the entire network is optical-based - Fiber Optic. This work objectives are to estimate the following components:

• $t_{RTU}$: the measuring time from the sensor to the buffer of the RTUs;
• $t_{DataCon}$ and $t_{Comm}$: the time elapsed in data acquisition, from RTUs to the SCADA;
• $t_{SE}$: the state estimation processing time of the unified WLS from the moment of receiving the data.

The total cycle execution time can be estimated as below:

$$t_{total}=t_{RTU}+t_{DataCon}+t_{Comm}+t_{SE}.$$

(1)

The latency of the P2P communication is found by aggregating the delays of data processing, queuing, transmission, and propagation [72]. In this work, the network is structured in P2P architecture and these delays are calculated/estimated as follows:

• Transmission delay ($T_{P2P}$): is calculated theoretically using Equation (2). However, additional simulations are carried out using Optiwave OptiSystems and OptiSPICE for fiber optic communication simulations [73].

  $$T_{P2P}=\frac{\mathrm{Data}\mathrm{Packets}(\mathrm{in}\mathrm{bits})}{\mathrm{Link}\mathrm{Speed}(\mathrm{in}\mathrm{Mbits}/\mathrm{s})}.$$

  (2)
• Propagation delay ($P_{P2P}$): theoretically it can be calculated using the below equation, for example, 1 km fiber optic link has 3.33 $\mathsf{\mu }$s $P_{P2P}$ delay. Further simulations are carried out for comparison.

  $$P_{P2P}=\frac{\mathrm{Distance}(\mathrm{in}\mathrm{km})}{\mathrm{Medium}\mathrm{Speed}(\mathrm{in}\mathrm{km}/\mathrm{s})}.$$

  (3)

4.1. The 1st Layer: From Sensors to RTUs

In this layer, the communication network is between the sensors and the dedicated RTU. The RTUs (IED/PMU) are distributed in an average of 100 m away from the sensors. Each RTU has multiple sensors based on the number and the type of measurements [21]. This work focuses only on 3 main measurements: voltage, current, and power.

The total data packets from S sensors can be expressed by Equation (4), where $N_{i-ch}$ is the number of channels, $SS{b}_i$ is the sample size, and $F{s}_i$ is the sampling rate.

$$\mathrm{TotalDataPackets}=\sum _{i=1}^S{\mathrm{DataSenosr}}_i=\sum _{i=1}^S{N}_{i-ch}\times SS{b}_i\times F{s}_i.$$

(4)

Modern sensors release the data in float formats (single/double: 32/64 bits). The single float structure (Figure 9) is commonly used to interface power measurements.

A simulation test case is implemented on OptiSPICE. An electrical-optical sensor is implemented that converts the sensor’s 32 bits generated @ 100 Mbit/s into laser beams. The electro-laser converter transmits the beams at 193.4 Thz with a wavelength of 1550 nm. The fiber optic cable is 100 m long with a 0.2 dB/km attenuation rate. The receiver stands at the RTU side and converts the laser beams back into an electrical signal. These signals are amplified to the proper value and read by the RTU input port. Figure 10 shows the OptiSpice schematic of the transmitter, fiber cable, and receiver.

Theoretically, the speed of light in vacuum is 299.792 m/$\mathsf{\mu }$s. However, this speed goes down in fibre material by a refractive index related to the wavelength and optical fiber brands (A and B, G.652 and G.655). It varies between 1.4677 and 1.47 for different wavelengths from 1310 to 1625 nm. Commonly 1550 nm wavelength is used due to its minimum attention ratio in fiber material, with a refractive index of 1.468. Therefore, the theoretical beam speed is close to 204.22 m/$\mathsf{\mu }$s.

Table 7. Time delays from 32-bits sensors to RTU.

Time Delay Type	Theoretical (ns)	Simulation (ns)
Propagation	489.66	491.959
Transmission	306	326.6521
Amplification	-	0.0258
Total	795.66	818.6111

shows the results of the theoretical estimation of the sensor time delays and the simulation estimations.

The simulation results are shown in Figure 11, in (a) the generated 32-bits are shown vs the received data after the amplification. In (b), the propagation delay is presented from the transmitter to the receiver. The data arrived at the receiver side after 818.611 ns.

The sensing time delay of N sensors ($T_s$) can be estimated as follow [21]:

$$t_{RTU}=T_s+N\times T_g,$$

(5)

where $T_g$ is a constant guard time delay represented by the opticl fiber transponders, it is used to save the buffer data and establish a new channel. In ultra low latency transponders, the delay can very between 2 ns and 30 ns [74]. In this work, 20 ns is used, then the sensing time delay $t_{RTU}$ for 4 sensors is: 818.611 + 4 × 20 = 898.611 ns ≈ 0.9 $\mathsf{\mu }$s.

4.2. The 2nd Layer: RTU to Data Concentrator

In this layer, the data from the sensors to the RTU are aggregated and transferred to the concentrators through fiber cables (50 $\mathsf{\mu }$m core radius, and 10 $\mathsf{\mu }$m cladding thickness).

Table 8. Data rate per message application from AC substations based on [7].

Message Application	Data Packets	Sampling Rate (Hz)	Data Rate
Protection Signals	50 Bytes	1	400 bps
Measurements data	102–198 Bytes	1440 @ Non-Sync AC	1147.5–2227.5 Kbps
		4096 @ 50Hz AC	3264–6336 Kbps
Interlocks	150 Bytes	250	293 Kbps
Control Signals	200 Bytes	10	15.26 Kbps
File Transfer	1 Mb	1/3600	248 bps
Total Traffic			1.422–2.477 Mbps
			3.489–6.489 Mbps

shows the RTUs (IEDs) data transfer rate from an AC substation. Considering the HVDC side, a very high sampling rate is used, resulting in higher data rate messages. It can be estimated by changing the AC side sampling rate to 100 kHz and the number of channels to 2 (±lines). Hence, the DC side RTU data rate for measurements is between 5312.5–12890.6 Kbps.

From the previous analysis and [7,11], the generated data for the different RTUs can be estimated as shown in

Table 9. Transmission delays per RTU type.

RTU Location/Type	Generated Data	Theoretical (ms)	Simulated (ms)
Line	0.623 MB	0.004867	0.005226
Converter	1.574 MB	0.012296	0.01320355
Generator (50 Hz)	1.027 MB	0.008023	0.008615

. The distances (medium length) between the RTUs and the data concentrators are shown in

Table A1. RTUs ⇄ Data concentrators.

DataConc. #	RTU #	Distance (km)
#1	#1G	100
	#2G	100
	#3C	100
	#4C	100
	#5L	100
#2	#1G	100
	#2G	100
	#3C	100
	#4C	100
	#5L	100
	#6L	160
#3	#1G	0.5–1
	#2C	0.5–1
#4	#1G	0.5–1
	#2C	0.5–1
#5	#1G	0.5–1
	#2C	0.5–1
#6	#1L	100
	#2L	100
	#3L	100
	#4L	315
	#5L	320
#7	#1L	100
	#2L	100
	#3L	100
	#4L	100
#8	#1L	150
	#2L	100
	#3L	100
	#4L	150
#9	#1L	100
	#2L	100
	#3G	155
	#4G	155
	#5C	155
	#6C	155
#10	#1G	150
	#2G	50
	#3L	50
	#4L	100
	#5L	100
	#6C	155
	#7C	155
	#8 (DC)	150

in Appendix A.1. Figure 12 shows the simulated transmission and propagation delays on OptiSystem for 210 kbits of data transmited in a 100 km fiber-optic cable at 1 Gbps. Similar simulations are carried out for other data sizes and distances as shown in Table 9 and

Table 10. Propagation delays based on fiber cable length.

Distance (km)	Theor. (ms)	Simu. (ms)	Distance (km)	Theor. (ms)	Simu. (ms)
50	0.244834	0.24483625	250	1.224170	1.2241813
100	0.489668	0.4896725	300	1.469004	1.4690175
150	0.734502	0.73450875	350	1.713838	1.7138538
200	0.979336	0.979345	400	1.958672	1.958690

. The fiber cable uses 1550 nm wavelength.

Figure 13 shows a schematic of a 100 km fiber optic P2P communication link in OptiSystems. It shows the transmitter (IED), receiver (Data concentrator), and multiple optical blocks. Each block represents an optical amplifier and 25 km fiber cable.

The IEC 61850 protocol delays are estimated from the studies in

Table 11. IEC 61850 protocol delays.

Reference	Msg. Type	Delay Range ($\mathsf{\mu }$s)
2016 [75]	GOOSE	24–37.4
	SMV	24
2012 [76]	GOOSE	14–35
	SMV	26–23

. The maximum value is used in this work (37.4 $\mathsf{\mu }$s per RTU).

Table 12. P2P total delays: Data concentrators and its corresponding RTUs (in ms).

Type	D1	D2	D3	D4	D5	D6	D7	D8	D9	D10
Prop.	2.44836	3.23183	0.004896	0.004896	0.004896	4.57843	1.95869	2.44836	4.01531	4.45602
Tran.	0.0485	0.05337	0.02181	0.02181	0.02181	0.02433	0.01946	0.01946	0.05337	0.11161
Prot.	0.187	0.2244	0.0748	0.0748	0.0748	0.187	0.1496	0.1496	0.2244	0.2992

shows the communication delays for a P2P architecture from the RTUs to the data concentrator.

The total delays of this layer can be approximated as the maximum of the summation of the propagation, transmission, and protocols delays. The maximum delays occur at D10 due to two-stage data concentrators. As a result, the estimated $t_{DataCon}$ delay is: 4.45602 + 0.11161 + 0.2992 = 4.86683 ms.

However, these delays can be reduced by muxponders. The overall delays is the maximum all links delays plus the muxponder delay per peer connected (RTU). Using OptiSystems, the muxponder required 3.371875 $\mathsf{\mu }$s per fiber cable. In order to find the delays of this case, the D9 and D10 delays have to be calculated separately. D10 has the maximum delays per peer, which comes from D9 to D10.

• D9: The max. transmission and propagation delays in D9 are 0.759 ms and 0.01320355 ms, respectively. D9 has 6 links leading to an overall muxponders and protocols delays of 0.02023 ms and 0.2244 ms. Therefore, the D9 delay is 1.01683 ms.
• D10: The max. transmission and propagation delays in D10 are 0.7345087 ms and 0.053371 ms, respectively. D10 has 8 links leading to an overall muxponders and protocols delays of 0.026975 ms and 0.2992 ms. Then D10 delay is 1.1140547 ms.

The final $t_{DataCon}$ delay is the summation of D9 and D10 delays: 2.130885 ms

4.3. The 3rd Layer: DC to SCADA

Similar to the previous analysis, layer 3 uses the same characteristic. The main changes are in the transmission delays due to the increase of the data transfer blocks. The propagation delay is a function of the fiber cable lengths in

Table A2. SCADA ⇄ #Y data concentrator.

Data Concentrators	Distance to SCADA (km)
#1	250
#2	300
#3	220
#4	200
#5	400
#6	200
#7	300
#8	350
#10	250

. However, Table 10 results are still considered valid. The protocol’s delays are estimated to be 0.3366 ms.

Based on

Table 13. Transmission delays per data concentrator.

Data Conc. #	Generated Data	Theoretical (ms)	Simulated (ms)
#1	5.825 MB	0.045507	0.048863
#2	6.448 MB	0.050375	0.054089
#3	2.601 MB	0.02032	0.021818
#4	2.601 MB	0.02032	0.021818
#5	2.601 MB	0.02032	0.021818
#6	3.115 MB	0.024336	0.0261302
#7	2.492 MB	0.019468	0.020904
#8	2.492 MB	0.019468	0.020904
#10	13.519 MB	0.105617	0.1134045

and

Table 14. P2P total delays: Data concentrators to the SCADA (in ms).

Type	D1	D2	D3	D4	D5	D6	D7	D8	D10
Prop.	1.22418	1.46901	1.07728	0.97934	1.95869	0.97934	1.46901	1.71385	1.22418

, in a P2P system, the total delay is the sum of the propagation, protocol, and transmission delays. As a result, the estimated delay is approximately $t_{Comm}=12.83535$ ms. However, using a server-based network (muxponders) as shown in Figure 14, the overall $t_{Comm}$ delay is reduced to $10.11$ ms.

4.4. The 4th Layer: State Estimation Processing Time

The processing time of unified state estimation for AC and DC systems is affected by the estimation algorithm, and the measurements count, type, and error.

Let $t_{SE-AC}$ and $t_{SE-DC}$ represent the processing time of an AC and DC networks state estimation. Then a centralized decoupled AC/DC state estimator takes time equal to the $max(t_{SE-AC},t_{SE-DC})$. However, in a coupled centralized approach further measurements are added such as voltage and power coupling constraints (available in [67]). It can be presented as shown in equation below:

$$t_{SE}=t_{\mathrm{unified}}\gg t_{P_{\mathrm{coupling}}}>t_{V_{\mathrm{coupling}}}>max(t_{SE-AC},t_{SE-DC}).$$

(6)

In this work, only the unified state estimation is in point of interest. Therefore, the simulations are carried out for $t_{SE}=t_{\mathrm{unified}}$ and for two sets of measurements, power injection only and complete set.

Table 15. Time delays ($t_{SE}$) of unified WLS state estimation.

Data Set	Measurements Count	Elapsed Time (ms)
Power-injection only	89	23.3709
Powerflow and Injection	140	26.3246

shows the time elapsed in WLS state estimation for the Cigre B4 network. The simulations were run on Intel Core i7-8750H, 2.20 GHz, GTX 1070 8 GB, and 16 GB RAM. The processing (estimation) time of an unified WLS is estimated based on the algorithm of this work [67], and implemented on Julia Optimization programming language [77]. The time performance is calculated for a 100 simulations on the Cigre B4 test case and shown in Figure 15 and Table 15.

In WLS algorithm, the bad data detection requires 1 complete estimation process per bad data. Thus, for N bad data measurements, the minimum estimation time is:

$$t_{\mathrm{SE}\text{-}\mathrm{BadData}}\ge t_{SE}\times (1+N).$$

(7)

For example, a single bad data measurement can lead to a processing delay of more than 52.65 ms. However, other estimation algorithms with internal bad data rejection, such as Least Absolute Value, can be slower than WLS. However, the processing time per bad data is not proportional [78].

4.5. Outcomes and Results

The case study presented the expected time elapsed for a complete state estimation cycle based on a fiber-optic communication network. The total delays of a state estimation cycle from the sensors up to the SCADA side is estimated by Equation (1). In a P2P architecture, the total delays are:

$$t_{\mathrm{total}}=0.0009+4.8669+12.8354+26.3246=44.0278\mathrm{ms}$$

(8)

If muxponders are used, then the total delays can be reduced to:

$$t_{\mathrm{total}}=0.0009+2.1309+10.1100+26.3246=38.5664\mathrm{ms}$$

(9)

The noticeable outcomes can be summarized in the following points:

• The total state estimation cycle time is less than 100 ms, which is less than the time requirements for a hybrid HVDC/AC systems.
• Changing the communication medium to wifi or microwave has an impact on the propagation delay.
• The static (snapshot) state estimation can be carried out in higher frequency (seconds range) instead of the traditional 5–15 min. Furthermore, the dynamic state estimation can be implemented at the local level since the accumulated delays are in few milliseconds.

5. Main Conclusions

This paper reviewed the modern SCADA system and highlighted the new challenges of integrating HVDC networks. The SCADA components and structure have been listed and discussed. In addition, a comprehensive review of the communication mediums and protocols is presented. The state-of-the-art on RTUs modifications for HVDC integration has been described along with their time constraints. Furthermore, this work establishes a case study simulation for a unified hybrid AC/DC state estimation time requirements using WLS. It estimates the time of measurements sensing, communication data acquisition, and the state estimator processing time. A Cigre B4 case was studied with 43 RTUs and 10 data concentrators. The communication links are fiber optic based and were simulated using OptiSpice and OptiSystems software. The unified WLS state estimation is implemented in Julia and was tested in two different measurements sets. The simulations were concluded that the estimated delays to complete a unified WLS state estimation from sensors to the SCADA side is around 44.02768 ms in a P2P system and 38.556 ms in a server-based system.