The Vision of Self-Management in Cognitive Organic Power Distribution Systems

Abstract

Due to the ongoing trend towards a decarbonisation of energy use, the power system is expected to become the backbone of all energy sectors and thus the fundamental critical infrastructure. High penetration with distributed energy resources demands the coordination of a large number of prosumers, partly controlled by home energy management systems (HEMS), to be designed in such a way that the power system’s operational limits are not violated. On the grid level, distribution management systems (DMS) seek to keep the power system in the normal operational state. On the prosumer level, distributed HEMS optimise the internal power flows by setpoint specification of batteries, photovoltaic generators, or flexible loads. The vision of the ODiS (Organic Distribution System) initiative is to develop an architecture to operate a distribution grid reliably, with high resiliency, and fully autonomously by developing “organic” HEMS and DMS which possess multiple self-x capabilities, collectively referred to as self-management. Thus, ODiS seeks answers to the following question: How can we create the most appropriate models, techniques, and algorithms to develop novel kinds of self-configuring, self-organising, self-healing, and self-optimising DMS that are integrally coupled with the distributed HEMS? In this concept paper, the vision of ODiS is presented in detail based on a thorough review of the state of the art.

1. Introduction

Climate change and sustainability are the major drivers to decarbonisation in the energy supply system by making use of renewable energy power plants (solar and wind energy, etc.). This goes hand-in-hand with increasing efficiency from primary energy to end-use energy and motivates shifting in the heating sector to heat pumps and in the traffic sector to electromobility. The power system is expected to become the backbone of all energy sectors and consequently, the fundamental critical infrastructure. Components such as photovoltaic (PV) systems, heat pumps, electric vehicles, and stationary battery systems, are connected to the electrical distribution system. This results in a decentralisation of power supply and requires the coordination of millions of components owned by network customers who become more prosumers with HEMS. From a distribution system operator’s point of view, this transformation changes the operational approach of the distribution system completely. Presently, few sensors allow transparency about power flows and voltage profiles in the grid. The dominant design principle of the distribution system are expectations of maximum power injected or absorbed by customers. However, increasing reinforcement costs push system operators to operate the electrical grid more at its capacity limit. Monitoring and state estimation become necessary to gain transparency to know the real safety margin towards the capacity limit. In addition, customers striving for self-sufficiency become more active in controlling their power flows to the public distribution grid, resulting in more volatile power flow patterns which increases requirements of monitoring and further control.

Operating the power system near to its capacity limit motivates one to use flexibilities such as active and reactive power control capabilities of controllable distributed generators (DG), loads, and storage systems. The coordination of these millions of actors in hundreds of thousands of grids can only be realised by full automation, but a large-scale roll-out of automation technology in many countries is many years away due to various reasons:

• Conventional regulatory schemes incentivise grid reinforcement instead of using information and communication technology (ICT);
• Full transparency in low and medium voltage grids is non-existent at present. Over the next years, near to full transparency is planned on the medium voltage level, on the low voltage level, this will be even farther in the future;
• The smart meter roll-out is delayed due to information technology (IT) security requests and a limitation to many customers in the first roll-out phases;
• A resilient and continuously available communication infrastructure to connect all sensors and actors has yet to be built;
• Present “smart energy” products request significant effort to configure, parameterise, connect devices, etc.;
• It is not yet entirely clear if from a full system perspective this shift is beneficial regarding costs, reliability, resiliency, etc.

Solving the technical challenges paves the way for full automation of electrical distribution systems in interaction with home energy management systems (HEMS) in the future. The goal is to construct an automation architecture that is as robust, safe, flexible, and trustworthy as possible. To achieve this goal, the electrical distribution operation architecture has to act more independently, flexibly, and autonomously. Such systems are also called “organic” after the term “Organic Computing” [1,2] because they should adapt dynamically to exogenous and endogenous change. The systems are characterised by the properties of self-organisation, self-configuration, self-optimisation, and self-healing, collectively referred to as self-management, based on self-awareness as well as context awareness. We consider self-awareness as an autonomous estimation of the current conditions and expected developments, while self-reflection would then use this information to reason about the own knowledge, see [3,4]. In this sense they can also be termed “cognitive”. The ODiS (Organic Distribution Systems) initiative, therefore, seeks answers to the following question:

How can we develop the most appropriate models, techniques, and algorithms to create novel self-configuring, self-organising, self-healing, and self-optimising distribution management systems (DMS) that are integrally coupled with the distributed HEMS?

Due to their “organic” nature, these systems will be called Organic DMS (O-DMS) and Organic HEMS (O-HEMS).

So far, there has been no comprehensive systematic consideration of self-x capabilities in power systems, nor has self-management been applied on the interaction of DMS and HEMS. This concept paper seeks to draw attention to this approach and aims at forming a basis for discussion of further research on this problem. Based on a comprehensive review of the state of the art, we provide a concept for interactive self-management of HEMS and DMS, while the process of realising the concept is subject to future work.

The remainder of this concept paper is organised as follows: Section 2 summarises and assesses the current state of the art with a major focus on self-x capabilities in power grids. Afterwards, Section 3 introduces the key research questions of the ODiS initiative, derives a system model allowing for self-x capabilities, and discusses use cases as well as evaluation scenarios. Finally, Section 4 summarises the concepts and gives an outlook to the future work of the ODiS initiative.

2. State of the Art

The smart grid architecture can be divided into layers, as Refaat et al. point out in [5]: On the one hand, the power system layer is responsible for transporting electrical energy from generators to consumers. This layer is made up of electrical lines, transformers, power switches, and other assets used directly to transport electrical energy. On the other hand, the communication layer is responsible for the transmission of relevant data to operate the power grid. It is used to collect instrumentation and measurement data from various measuring devices and transfer them in a fast and reliable manner. The data can be transferred and used by a central instance, i.e., a grid control centre, or used by distributed data collectors in the smart grid paradigm. This layer may be a virtual private network on the public Internet or forms of local communication networks (e.g., WiFi).

The distribution management system (DMS) is a system that makes use of the communication layer to receive and transmit information to other entities (here modelled as agents)in the grid. The goal of the DMS is to keep the power system layer in secure and cost-efficient operation conditions. The DMS interconnects with hems situated in the corresponding power system area.

Advancing on the concepts of DMS and HEMS as part of an Organic Distribution System (ODiS), both an O-DMS and an O-HEMS would possess several self-x capabilities that enable them to adapt and optimise for different situations. The following subsection introduces the relevant terms to describe DMS and HEMS. Afterwards, the state of the art in self-x capable distribution management systems is reviewed.

2.1. Home Energy Management Systems and Distribution Management Systems

Home Energy Management Systems (HEMS) aggregate information and manage the assets connected to it in a home/building. They can optimise the internal energy consumption and provide an economic advantage to the homeowners: Althaher et al. present their HEMS controller in [6]. The goal is to control appliances in response to dynamic price signals so that the electricity bill and the curtailed energy are minimised. Results show that both goals can be achieved given certain flexibility on the user’s comfort. In [7], Ahmad et al. present a HEMS to minimise the electricity bill of the prosumer by optimising energy consumption. The optimisation should also be beneficial for the utility due to the reduction of peak power. Zhang et al. present a demand response strategy for a HEMS that categorises loads in a household into three categories depending on their controllability [8]. The machine learning (ML) based strategy uses data such as weather forecast and thermostat settings to optimise energy consumption.

Enhanced with self-x capabilities, O-HEMS will also play a part in organic DMS. Little research has been done to use HEMS beneficially for the grid operator: Zandi et al. [9] show the implementation of a HEMS as part of a multi-agent system (MAS). Three self-optimisation use cases are provided that could benefit from such a HEMS: First, the HEMS can be used to reduce peak load in the distribution grid as requested by the DMS. Second, it is used to reduce the voltage impact caused by high renewable feed-in. The third use case tries to improve residential resiliency by enabling islanding of the distribution grid while relying on local energy storage. Vázquez-Canteli and Nagy review numerous papers using reinforcement learning (RL) for demand response in HEMS [10]. They identify multi-agent RL, dynamic addition and removal of buildings, and testing RL control systems in physical systems (possibly in the co-simulation context) as open research questions (among others). Similarly, Dimeas et al. propose a detailed concept of HEMS and envision services to assist grid operators in [11], but do not present practical research or simulation results.

A major innovation in research intended by ODiS is the use of several self-x capabilities that allow both an O-DMS and an O-HEMS to select control strategies and exploit flexibilities according to their individual and joint optimisation goals. This is believed to enhance flexibilities in changing situations and configurations and improve resilience to a variety of disturbance scenarios.

A Distribution Management System (DMS) is defined by possessing different characteristics [12]: The system actively monitors the conditions in the distribution system; it can perform controlling measures in real-time (e.g., reconfiguration after faults as well as voltage and reactive power management). Additionally, it can integrate DG, storage, and demand response. The assets can be integrated directly or aggregated in underlying HEMS. The HEMS can distribute aggregated measurements of its underlying assets and additionally it can send information about the current flexibilities to the DMS.

A DMS has to function in the different states a power grid can be in. According to [13], in a normal grid state, all loads are supplied with power and the grid is operated within the permitted conditions. In an emergency state, some operating limits are violated, which requires immediate action by the grid operator or the DMS to come back to a normal grid state. The restorative state appears if not all loads are supplied with power (e.g., partial or total blackout). Measures must be taken to resupply all customers while adhering to the defined operational limits [14].

Slanding can appear in power systems and needs to be managed by the DMS. It can both be unintentional (as part of the restorative grid state), but also intentional/planned to operate parts of the grid autonomously. Islanding is defined as a “condition in which a portion of an Area [sic] electric power system (EPS) is energised solely by one or more Local EPSs through the associated points of common coupling while that portion of the Area EPS is electrically separated from the rest of the Area EPS” [15]. In other words, islanding is the formation of a power grid by DG if the main grid is in a state of blackout. During intentional islanding, the distributed grid-forming generator performs voltage and frequency control to supply the connected loads with energy while respecting key performance indicators such as voltage and frequency bands. Conscious islanding of autonomous parts of a grid has been introduced as adaptive islanding or self-islanding [16].

The parts of the grid operated autonomously in this way are also referred to as Microgrids (MGs). MGs are defined as distribution systems consisting of distributed loads and generation units operable “in a controlled, coordinated way” both in grid-connected and islanded mode [17] and thus can be understood as a part of active distribution grids. The authors of [17] further categorise three types of MG controllers, namely centralised, distributed, and autonomous control, and point out the superior characteristics of distributed approaches regarding robustness and privacy concerns.

Many published works listed in the following section incorporate some form of self-x capability in their DMS design. However, none of the cited publications combines all of the introduced self-x capabilities into a “fully” organic DMS that can flexibly adjust to changing situations in a grid, both on the electrical and IT side.

2.2. Methodical Foundations from the Fields of Autonomic and Organic Computing

Orthogonal to the application area of the smart grid, self-properties have been considered in the literature as its own research field. In the following, we define the desired self-properties according to the terminology used in the organic computing (OC) [1] and autonomic computing (AC) [18] domains. However, there is no commonly agreed notation of these self-terms in literature, which is in turn not fundamentally necessary for the scope of the initiative as we aim at the basic functionality rather than a precise terminology framework. We assume a self-system to consist of a potentially large set of (autonomous) subsystems. Internally, each of these autonomous subsystems distinguishes between a productive part (responsible for the basic purpose of the system) and a control mechanism (CM, responsible for controlling the behaviour of the productive system and deciding about relations to other subsystems). This corresponds to the separation of concerns between System under Observation and Control (SuOC) and Observer/Controller tandem in the terminology of OC [19,20] or Managed Resource and Autonomic Manager in terms of AC [18]. The user guides the behaviour of the autonomous subsystems and the overall system depending on the abstraction level using a utility function and (normally) does not intervene at the decision level. Actual decisions are taken by the productive system and the CM.

According to [18], we define self-configuration as the ability of a CM to change the parameterisation of components and systems following high-level policies. In contrast, self-organisation is the ability of distributed controllers to “modify the overall system’s structure (i.e., relations between components and the corresponding interaction schemes) depending on current conditions and based on the particular system goal” [1]. This can be augmented with self-optimisation capabilities, which means that the CM of the “components and systems continually seek opportunities to improve their own performance and efficiency” [18]. Considering robustness/resilience as a primary goal of self-mechanisms, reactions of the CMs to disturbances or attacks are required. In this context, we distinguish between self-healing (i.e., the CMs being able to “automatically detect, diagnose, and repair localised software and hardware problems” [18]) and self-protection (which refers to the ability of a system to “automatically defend against malicious attacks or cascading failures. It uses early warning to anticipate and prevent system-wide failures.” [18]).

Mapping self-terminology onto the power system field, the overall self-system is represented by the organic distribution system while the autonomous subsystems comprise both, components of the O-DMS and the different O-HEMS. Productive systems in the O-DMS are the grid control strategies, which use flexibilities and actuators to keep the grid in secure conditions. The different CM and their realisation are subject to the work of the ODiS initiative. In the following, we discuss the four self-x capabilities that are investigated in ODiS: (a) Self-configuration, (b) self-organisation, (c) self-optimisation, and (d) self-healing. For all four concepts, we introduce the term, describe the state of the art from an OC/AC perspective, and close with the state of the art in the domain of energy systems.

2.2.1. Self-Configuration

Self-configuration (also called self-adaptation or re-parameterisation [21]) in intelligent systems is either realised statically or adaptive. Static solutions are assumed to operate in fully predictable environments. In contrast, the main challenge is to develop solutions for adaptive self-configuration mechanisms, since this requires guarantees for reliability and accuracy despite the presence of uncertainty and unanticipated conditions [18]. This includes consideration of a set of aspects [22], including (i) the type of adaptation (i.e., response to external behaviour, or closed systems), (ii) the degree of autonomy, (iii) cost-effectiveness, and (iv) the selection of the current ‘best’ adaptation strategy, see [23]. Furthermore, categorisation with respect to real-time capabilities for embedded control mechanisms is used in the literature [24].

In general, there are three categories of decision-making techniques [23,25]: Predefined decisions, decision making driven by ML, and constraint programming. Predefined decisions mean that the behaviour for a specific situation is specified at design-time by engineers, ML approaches to integrate the self-configuration decision within the learning algorithm or use learning techniques to derive predefined decisions, and constraint programming tries to find an optimal solution using solvers. Due to the dynamic nature of the control problem focused in ODiS, we will rely on ML approaches.

Specific to power systems, there is only limited research on self-configuring systems. Currently, the project “Plug’n’Control” [26] aims to validate the concept of a self-configuring system of inverters. By allowing inverters to detect their underlying asset (battery, charging station, etc.) and being able to communicate, the system can be used to avoid overloading in the grid or perform other tasks to retain grid stability. It is unclear which category of decision-making is used.

To our knowledge, no (organic) distribution management system uses ML-based decision-making. Therefore, the initiative provides an ideal testbed for adapting and evaluating recent advances in ML in terms of their use in self-configuring distribution grid management systems.

2.2.2. Self-Organisation

Self-organisation has been defined as “a mechanism or a process which enables a system to change its structure without explicit command during its execution time” [27]. Based on a multi-agent perspective and according to [28], the different concepts can be classified using the main categories of task or resource allocation, relation adaptation, organisational design, and collective decision making (next to some minor classes).

Self-organising task allocation has been primarily studied as a basis for other research issues such as coalition formation [29]. In addition, it heavily relies on the particular application scenario and the corresponding purpose of the allocation – the most prominent algorithms have been considered in the context of wireless sensor networks [30] and multi-robot systems [31]. In all of these cases, specific tasks of the sensor nodes or the robots have been assigned to the individual entities using decentralised algorithms. Although being based on standard communication schemes for distributed systems (such as leader election, echo algorithms, or broadcast [32]), the solutions are not generic.

Relation adaptation typically covers two intertwined questions for the autonomous subsystems [33]: With whom to modify relations and how to modify them. Existing approaches for partner selection are based on the agents’ interaction history [34], sometimes combined with an uncertainty or trust measure [35]. In contrast, the term ‘organisational design’ refers to the question of how society or group members interact and relate with one another [36]. This mainly means work focusing on the assignment of roles to different actors or agents in the overall system [37]. In contrast, we assume given roles in the context of ODiS.

The concept of collective decision making originates from social science, where members of a group have to achieve a consensus that typically reflects a compromise between the different members. As one particular example, Valentini et al. [38] presented a generalised solution that assigns weights to members and considers their votes in a decentralised manner. In general, the idea is mostly to find a collective agreement over the most favourable choice among a set of alternatives. This is usually encoded as the ‘best-of-n’ decision problem, which is characterised by a dynamic n in open environments. For decentralised decisions, we will investigate how these decision mechanisms can be reused for energy systems.

Self-organisation in power systems is only a fringe topic in research, mostly implicitly researched in the form of cooperative MAS. A comprehensive survey on MAS in power system control can be found in [39,40]. This maps the general overview provided, e.g., in [28] mentioned above in the context of general self-organisation approaches, to the domain of power systems but neglects the separation of concerns between a distribution system and home energy management systems along with their autonomous integration – which is the focus of ODiS. For example, Jianfang et al. develop a MAS for self-healing in [41], which relies on self-organisation for fault isolation and resupply. In [42], an agent-based control scheme for the trading of active power and ancillary services is proposed that guarantees the fulfillment of grid constraints. Dimeas et al. apply MAS to MGs, where each of the following elements is represented by an agent: An agent network operator, market operator, and central controller of the individual MGs [43]. The considered actions comprise load-shedding, black start, and the transition between grid-connected and islanded mode. In the context of self-healing, Li et al. develop an agent-based fully decentralised framework for service restoration of a distribution grid with peer-to-peer communication between agents [44]. In the presented strategy, intentional islanding is combined with network reconfiguration. However, the vision of ODiS goes far beyond these approaches by including self-organisation only as one of many self-x capabilities.

2.2.3. Self-Optimisation

Self-optimisation in self-adaptive and self-organising (SASO) systems is typically considered to deal with the improvement of control parameters of the productive system or the management of the overall system structure based on a given set of goals. Depending on the time horizon, two different fields are distinguished: ML-based and optimisation-based.

In [45], the usage of optimisation techniques within SASO systems for generating new system configurations or adaptation plans is analysed. The authors identified the use of 29 different techniques in 51 publications, including techniques from probabilistic, combinatorial, evolutionary, stochastic, mathematical, and meta-heuristic optimisation. The approaches target optimisation in centralised systems with only minor attention to decentralised optimisation. Such centralised optimisation comes with the disadvantages of providing the risk of a single point of failure as well as a potential bottleneck. Furthermore, it contradicts the nature of autonomous, coordinating entities. Besides by applying optimisation heuristics based on a given utility function, constraint solvers are frequently applied.

On the other hand, self-optimisation is achieved by autonomous learning. In a study [46] of our preliminary work, we analysed the usage of learning techniques in SASO systems. Here, RL has been identified as the most prominent variant, mainly realised as simple learning tasks, e.g., using Q-Learning, or more sophisticated approaches, e.g., using Learning Classifier Systems [47]. Alternatively, multi-agent RL (MARL) is employed to solve problems in a distributed manner when centralised control becomes infeasible [48]. MARL is also used in [49] to tackle the complexity emerging in MAS domains. Especially the “Extended Classifier System” (XCS) variant by Wilson [50] (including its variants from the OC domain such as [51,52,53,54]), which has been widely used for implementing self-adaptation with runtime learning capabilities. For instance, XCS can be seen as an integral part of OC systems that are said to exhibit self-learning properties.

Self-learning has not been a major focus in research on self-optimising power systems. While many publications limit their control capabilities to adjusting switching states, other control measures can be used to optimise the power system: Wong, Lim, and Morris publish a so-called “self-intelligent” active distribution management system in [55]. Their system can control key grid variables such as voltage limits by using an energy storage system to mitigate issues in the grid. The reliable and economic operation of a distribution grid is targeted by the optimisation structure presented in [56] by Li et al. Different asset classes (loads, PV generators, wind energy converters, and batteries) are modelled as individual agents of a MAS that can perform certain tasks not only for self-healing, but also for economic optimisation of the grid. Roytelman and Medina present a volt/VAR control algorithm that uses advanced metering infrastructure measurements, distributed capacitors, and inverters to stabilise the grid’s voltage profile [57].

Recently, there have been a few works focusing on cellular network structure approaches. For example, project Zellnetz2050 [58] follows an approach to increase the flexibility of future energy systems in which different sectors of the energy systems (electricity, gas, mobility, and heat) are coupled. The cells are created hierarchically based on geographical regions and comprise all levels of the electricity grid. In contrast, the ODiS initiative focuses on the distribution level of the electricity sector, but also considers the ICT system and other grid states. Another cellular approach is developed in the project C/sells (www.csells.net, accessed on 24 November 2021). Based on autonomous energy cells that can be created both region- and asset-based and manage their own consumption and generation locally, a decentralised power supply is pursued. At the same time, a variety of methods for the operational management of the individual cells is enabled. In this context, the ODiS initiative also aims at answering the question of the communication structure in the automated distribution grid. However, similarly as in Zellnetz2050, C/sells does not provide strategies for events of emergency.

An approach from the Organic Computing domain has been presented by Schmeck et al. in the context of their ‘Smart Home’; a building energy management system (i.e., a HEMS) that combines different communication technologies for monitoring, data recording, and visualisation purposes [59]. This information is used to perform offline optimisations using heuristics to improve the utilisation of flexibilities for goals such as an increased self-consumption or the provisioning of grid-supporting services. The system is based on a hardware demonstrator (‘living lab’) and is evaluated by hardware-in-the-loop simulation [60]. In contrast to ODiS, the focus is mostly on isolated HEMS. In particular, the awareness level is less sophisticated, the configuration of the HEMS is more static, the relation to other HEMS is neglected, and the integration into the DMS is done rather static.

An agent-oriented controller based on RL achieves power balance in a complex smart grid with changing communication topologies [61]. In their study, Rahman et al. propose a MAS in which distributed agents estimate the reactive power based on local information and information of their neighbours [62]. Based on this estimation, controller agents adapt to the current system state and ensure the safe operation of the distribution network. Radhakrishnan et al. present an agent-based energy management system in which the responsibilities for switching between operating modes, the commitment of storage units and operation schedules are assigned to different agent types [63]. In [64], a hierarchical multi-agent architecture performs the energy management of an active distribution grid while being flexible to changes in the grid topology. In further work, the authors also consider self-reconfiguration [65], robustness [66], and self-diagnostics [67].

Current research usually focuses on one aspect of the grid to be optimised such as the switching states or energy storage. ODiS aims to use a holistic approach to self-optimisation, assessing, and weighting not only the different flexibilities and topologies but also the optimal control strategy in the specific scenario. Two different levels—grid-level via O-DMS and home-level via O-HEMS—provide a large action space to operate within—greater than in most similar publications. A major challenge to research in ODiS will be the reduction of the action space depending on the current situation in the grid.

2.2.4. Self-Healing

The basic idea of approaches for self-healing following the initial notion of IBM’s Autonomic Computing initiative aimed at decreasing the required degree of human interaction [18,68]. Self-healing is often examined in combination with self-protection. However, self-protection is usually considered in a security sense, which is not initially a focus of research in ODiS. According to [69], self-healing systems are categorised as being fully supervised, semi-supervised, or unsupervised. This refers mainly to the time and the trigger of performing self-healing mechanisms with a focus on their learning strategies: Manually or automatically. Fully supervised techniques (such as [70,71,72]) typically require continuous availability of human operators and frequent interaction. This is expected to result in reduced uncertainty about system behaviour and controlled re-parameterisation towards stable behaviour [73]. Popular approaches utilise a database of predefined recovery plans, and these plans are mapped towards observed or estimated events. Examples are the “rainbow” framework [70] and the “GPAC” (General Purpose Autonomic Computing) framework [74]. However, this can be mapped on the concept of predefined self-configuration that is triggered by a given set of events that are associated with negative states requiring repair mechanisms.

As an alternative to supervised approaches, semi-supervised concepts have been proposed that work based on event-driven monitoring. In general, this means combining classification of conditions (i.e., detecting events) with a run time analysis to identify anomalies, i.e., distinguishing between normal or known/established patterns of behaviour and unknown or noticeable patterns [75,76]. In literature, different approaches are available, ranging from purely reactive usage of basic exponential smoothing algorithms for time series [77] to the pro-active prediction of states [78]. One particular example is the VieCure framework [79] that combines event detection with direct analysis of metrics. In contrast to approaches that directly map faults to recovery plans, this system tries to determine abnormal behaviour and uses this as a trigger for self-healing decisions. Abnormal behaviour can be, for instance, a series of incidents within a log file—and this is correlated with faults and correspondingly, the most suitable response is determined. However, unknown events and faults require supervision and this is not performed as a fully autonomous process.

There are only a few cases where frameworks are autonomously establishing self-healing properties, see e.g., [80,81,82]. Their realisation is usually highly dependent on the particular application scenario and only seldom follows a generalised approach. Technically, they make use of genetic algorithms [81], neural networks [82], or so-called “totipotent” behaviours [80]. Besides the underlying applied techniques, the approaches differ in terms of how much risk and resource commitment are needed, which again controls how much autonomous behaviour is possible.

Focusing on power systems, numerous self-healing schemes have been published in recent times. For example, Xiu et al. present their research on self-healing in smart distribution grids in [83]. They define several core processes of a self-healing system. They also define different layers for agents in a MAS, similar to [41]. The MAS can reconfigure the grid after a fault by changing switches to disconnect and connect lines. In [84], Duarte et al. present their self-healing system for power systems. Their modelled scheme includes state estimation and prognosis of the power grid based on SCADA [85] data, a self-healing optimisation and a sequence of manoeuvres to change switching states according to the optimisation result. In simulations, they show that the scheme can reduce the SAIDI [85] of different feeders, sometimes by 20% or more. Similar fault detection, isolation, and restoration (FDIR) schemes are presented in [86,87,88]. Liu et al. present a concept for a self-healing urban power grid in [89]. It is based on a MAS, which is structured in three layers (organisation, coordination, and response). The system classifies the state into 5 classes (secure normal, insecure normal, alert, emergency, and restorative) and also defines 4 sub-control tasks: Emergency control, restorative control, corrective control, and preventive control. Several test cases confirm the validity of the concept. Baxevanos and Labridis details a cooperative system for fault and power restoration management in distribution grids [90].

A common theme among published research on self-healing in power systems is the use of MAS. Agents can coordinate with each other to perform self-healing. This is usually done after the occurrence of fault situations, where a new switching state of the grid has to be found [41]. A limitation of such schemes is the outage of parts of the grid if no connection to the substation transformer can be made by switching after a fault.

Self-healing can benefit from the coordination between O-DMS and O-HEMS levels. By making use of local control strategies and DG as well as batteries, grid sections can be operated in a “self-islanding” mode without connection to the substation transformer while the fault is being repaired. Research on such a scheme is a major innovation compared to the state of the art.

3. A Concept for Organic Distribution Management Systems

3.1. Key Research Question

The overall goal of ODiS is to develop and investigate a novel, integrated approach for an organic distribution system that operates completely autonomously. In particular, we have to answer the following basic research question:

How can we create the most appropriate models, techniques, and algorithms to develop novel kinds of self-configuring, self-organising, self-healing, and self-optimising DMS that are integrally coupled with the distributed HEMS?

These new O-DMS and O-HEMS (“O” for “organic” to refer to “organic computing” [1,2] as reference domain) shall be implemented as the management system for a benchmark grid and evaluated by simulations according to a number of use cases. The use cases are chosen in a way that the key self-x capabilities are required.

The upper part of Figure 1 shows a low voltage grid that is outfitted with an O-DMS. The lower part of the figure shows the runtime adaption components of the O-DMS. Sensors provide various measurements, which are the basis for the state estimation process (observer). The optimisation (controller) creates setpoints depending on the current state and the control strategy. Setpoints are sent to the actuators in the grid.

Figure 2 zooms into Figure 1 to reveal the components of an exemplified home. At the center, the O-HEMS connects to the ICT and electrical grids. The O-HEMS can act as an agent with underlying assets such as inverters are part of local control strategies. The runtime adaption component is identical to the O-DMS. However, state estimation and optimisation blocks use different functionality due to the different scope of operation and available measurements and actuators.

Conceptually, this means to improve the robustness against disturbances and dynamics, the accuracy of reactions to changes, as well as the corresponding efficiency of system behaviour compared to existing systems with a strong decoupling of DMS and HEMS functionality. Especially in cases where the control structures of the DMS are not reliably available (e.g., due to disturbances in the basic communication channels) or not working reliably anymore, HEMS can take over parts of the responsibilities and therefore provide the desired basic functionality. Simultaneously, a continuous, local self-optimisation of control behaviour is established, which cannot be realised at a system-wide level using a centralised approach due to limitations in communication latencies and computation efforts. As a result, ODiS is characterised by a hybrid system structure that combines distributed (OHEMS) and centralised (ODMS) elements towards a more integrated approach and therefore adapts the existing, static, and strictly hierarchical concept.

However, a full distribution of functionality is not always desirable as it comes with limitations (e.g., dependencies cannot be detected with local observations, all decisions with impact beyond an isolated element require communication, and conflicts may not be solvable among autonomous entities) and benefits (e.g., more robustness due to avoidance of a single-point-of-failure, locality of problem solving reduces overhead). The CAP (consistency, availability, partition tolerance) theorem [91] formulates the problem that there is always a conflict in distributed system in the sense that all CAP aspects can not be addressed at the same time.

On the one hand, the hybrid system structure defined for ODiS is still impacted by the insights of the CAP theorem due to the partly distributed character. On the other hand, it mitigates these effects due to general availability of centralised DMS services. However, further challenges can result from the system concept as, e.g., the aspects fairness, robustness, and performance may also result in a trade-off that is already observable in hybrid system structure as explained in [92]. Consequently, a major goal of the ODiS initiative is to investigate basic mechanisms that are aware of these influences and make them explicit for the decision-making processes, resulting in a multi-criteria runtime optimisation.

3.2. System Model

3.2.1. Simulation Framework

Figure 1 and Figure 2 show the descriptive system model for the two relevant layers (grid level and home level). The system model is shown in Figure 3 and is visualised within the simulation framework OpSim [93] which enables the exchange of information in a predefined way. All actors as well as a grid and communication network simulator connect to a simulation bus, which routes the information automatically.

OpSim has already been used in a variety of applications, e.g., for co-simulation between different types of components, testing of prototype controllers, and other tools used to aid grid operation. It consists of a “master control program” and a message bus to which components can connect. Using a scenario configuration, the control software routes information between the different components.

For ODiS, OpSim can be utilised to simulate various scenarios in different use cases. All agents (the term is used for either O-DMS and O-HEMS) connect to the message bus as individual OpSim components. Depending on the available functionality, the runtime adaption component is run in parallel for every component. Measurements are observed and processed and setpoints are generated. The setpoints are sent back to the message bus and distributed.

A separate component holds the grid model and corresponding time series. It is used to generate measurements for the other components. The disturbances imposed by the use case, e.g., voltage limit violations or interrupted communication links, are created in the “disturbance creator” component by altering the data going out to the message bus accordingly. Part of the research is to identify and evaluate ways to realistically emulate disturbances for the corresponding use cases. The grid simulator also collects the set points by the O-DMS, O-HEMS so that the next grid state is a reaction to the chosen control strategies.

3.2.2. Observable and Controllable Parameters

Every component observes different variables and adjusts parameters depending on the self-optimisation targets as shown in Figure 4: The O-DMS component has access to measurements from the corresponding power grid and the data from underlying components of the distribution grid. Specifically, the O-DMS reads measurements of active power ($P_t^k$, $P_g^k$), reactive power ($Q_t^k$, $Q_g^k$), current ($I_t^k$, $I_g^k$), voltage magnitude ($V_t^k$, $V_g^k$), and frequency ($f^k$) from the substation transformer t and from DG g connected at the O-DMS voltage level respectively. Further, circuit breaker states (switching states ${\delta }_s^k$) are stored. From each O-HEMS, the O-DMS receives both measurements and trajectories of flexibilities of relevant variables, such as the active and reactive power, storage energy capacity, current, voltage magnitude, and frequency. From each O-HEMS i, the O-DMS receives measurements of the aggregated active power $P_i^k$, reactive power $Q_i^k$, the total storage energy capacity $W_i^k$, the current $I_i^k$, voltage magnitude $V_i^k$, and the frequency $f_i^k$ at the grid connection point of the O-HEMS. Additionally, each O-HEMS i transmits trajectories of flexibilities of active power (${\underline{\mathbf{P}}}_i^k,{\overline{\mathbf{P}}}_i^k$), reactive power (${\underline{\mathbf{Q}}}_i^k,{\overline{\mathbf{Q}}}_i^k$), and storage energy capacity (${\underline{\mathbf{W}}}_i^k,{\overline{\mathbf{W}}}_i^k$), where $k\in \mathbb{N}$ is the current time step and ${\mathbf{X}}^k=[X\left(k\right|k)X(k+1|k)\dots X(k+H|k)]$ is the trajectory of a variable $X\in \mathbb{R}$ computed at k over a time horizon $H\in \mathbb{N}$.

These input data are then processed by the O-DMS to estimate the current state and to generate setpoints to control the grid state. The O-DMS can control transformer tap positions for voltage regulation by setpoints for active and reactive power $P_t^{k,\star }$ and $Q_t^{k,\star }$, respectively. Similarly, setpoints $P_g^{k,\star }$ and $Q_g^{k,\star }$ are computed for generators (e.g., solar panels) directly connected to the distribution grid. It also computes setpoints for the circuit breaker status ${\delta }_s^{\star }$ of each circuit breaker $s\in \mathbb{N}$ in the corresponding grid section to change the grid topology. Finally, it sends parameter ranges for active power ${\underline{P}}_i^k,{\overline{P}}_i^k$, reactive power ${\underline{Q}}_i^k,{\overline{Q}}_i^k$, and storage energy capacity ${\underline{W}}_i^k,{\overline{W}}_i^k$ to the underlying O-HEMS.

The O-HEMS gathers several measurements, depending on the connected assets: Each connected prosumer (e.g., electrical batteries) transmits its available energy. Flexibilities regarding loads are also calculated. The measurements are processed locally for use in local control strategies. If an O-HEMS is acting locally or provided with parameter ranges, it can self-optimise and set parameters to the inverters (for reactive power management), battery (for power storage/generation) or loads depending on available flexibility. Based on the measurements and forecasts, the O-HEMS aggregates its flexibilities of total storage energy capacity, active and reactive power for the current time step, and for a predefined time horizon. The corresponding trajectories of active power, reactive power, and storage energy capacity are transmitted to the O-DMS.

For simulations in ODiS, all measurements can be assumed to be sent periodically in varying intervals which range from seconds to minutes depending on the component. For more specific use cases such as self-healing, the parameters need to be extended.

3.3. Evaluation

To quantify the success of ODiS before actually implementing the techniques in real-world applications, various use cases as proposed in this section must be investigated.

The research in ODiS is categorised along with several self-x capabilities, that should be part of both the O-DMS and the underlying O-HEMS. The self-x capabilities are designed to ensure normal operation under a variety of interference influences both on the side of the electrical grid and of the communication network. The following summary of use cases is classified into mostly self-configuration/-organisation (C1–C3), self-optimisation (O1–O4, C/O5, C/O6), and self-healing (H1–H4). All use cases are described in

Table 1. List of use cases for the organic distribution system. Use cases containing ”C” mainly require the ability to self-configure, ”O” is the ability to self-optimise, and ”H” is the ability to self-heal. Self-organisation is mainly included in the use cases containing ”C”.

Use Case	Description
C1	Manual (Dis)connection of a new electrical asset: Self-configuration is required after detection of new assets, as well as self-configuration of the grid model, operational algorithms, local parameters, communication configuration. The new power grid topology needs to be validated based on self-organisation.
C2	Continuous anomaly detection, automatic background process: The system needs to continuously monitor the data streams for anomalies and inconsistencies as could be caused by mal-parameterisation, human failure, or cyber-attacks. If anomalies are detected, the underlying data or control schemes need to be adjusted accordingly.
C3	(Dis)connection of a new agent: Every O-HEMS or O-DMS (agent) must perform self-configuration to adapt to the new overall system. This further requires self-organisation for determining the best possible system structure.
O1	Self- and environment-awareness (including state estimation and forecasting): The basis for optimisation of electrical grids is the knowledge of the relevant grid variables. A state estimation ensures that such knowledge is available, even when a low number of real-time measurements is available, which is typically the case for distribution grids. Forecasting provides projections into the future, which can be strongly beneficial for control strategies. In principle, both, O-HEMS and O-DMS, must be self- and environment-aware.
O2	Normal operation and self-optimisation: During normal operation, key performance indicators are inside their defined boundaries. Nevertheless, self-optimisation of agents can improve their conditions further, e.g., to minimise system losses or increase self-consumption of energy.
O3	Voltage limit violation (no circuit breaker tripping): High load, high DG, or a sub-optimal switching configuration can cause voltage limit violations in a grid zone. Self-optimising agents need to choose between different measures to push the voltage back inside limits. Measures can be self-optimisation via load management, active and reactive power control of DG, flexibilities/storage, switching state reconfiguration, or transformer tap adjustments.
O4	Thermal line/transformer limit violation (no circuit breaker tripping): The current flowing through grid assets is bound by thermal limits, which should not be violated for longer than a specific period or failures can occur. Similar to the use case O2, agents need to be aware of potential limit violations and use available measures to alleviate the limit violation.
C/O5	Unreliable communication for the asset(s): Communication between the agents can become unreliable, e.g., faulty measurement devices produce unreliable measurements that should not be used further. The communication network can become unstable, e.g., exhibit packet loss. Agents need to identify unreliable data streams and self-adapt accordingly.
C/O6	Interrupted communication for the asset(s): If technical issues appear in the communication network used for self-coordination, the exchange of information becomes unreliable or interrupted completely. Similarly to C/O5, agents need to self-adapt, e.g., by using an alternative channel of communication or by changing their control scheme from central to local control strategies.
H1	Self-protection: In the case of an asset failure or a manual mal-configuration, events such as blackouts can occur. For such cases, a self-protecting O-DMS prepares contingency strategies so that a minimum number of customers are affected.
H2	Self-healing/reconfiguration: If the self-protection is not successfully performed, a self-healing and reconfiguration scheme must be run. The scheme can trigger individual parts of a grid to operate as autonomous islands without connection to the higher voltage level (self-islanding) and may require changes in control strategies of O-HEMS etc. (e.g., switching to local control strategies). The process includes decoupling from the interconnected grid, a black start, islanding operation with suitable control strategies and re-synchronisation when self-islanding is no longer optimal.
H3	Self-healing under impaired ICT: Impaired ICT connections pose an additional challenge for self-healing schemes. O-DMS and O-HEMS need to evaluate the limited control options and prioritise operational limit violations, e.g., allow for thermal overloading up to a certain time frame to avoid a (partial) blackout.
H4	Blackout in the communication grid: A blackout in the communication grid prevents coordination among the different agents. The O-DMS needs to evaluate and activate alternative means of communication. If that is not possible, the agents need to switch to local control strategies automatically.

and sorted according to the service level and power grid operational state in Figure 5.

In ODiS, we focus on the power system layer’s performance while the service level of the communication layer is fixed for the specific use case. The task for O-DMS and O-HEMS is to keep the power system layer in the normal state by using their self-x capabilities while disturbances appear both in the communication layer as well as the power system layer.

4. Conclusions and Outlook

The energy grid undergoes a dramatic change from a previously fully centralised to an increasingly distributed structure containing autonomous subsystems. The operation of this grid requires more observation and control mechanisms at all levels of the grid structure—allowing for fast detection and reaction to unexpected developments, for instance. The Organic Distribution System (ODiS) initiative proposes to master the resulting issues of the control problem (such as robustness, scalability, efficiency, and limited autonomy) by means of introducing self-x capabilities as known from the autonomic and organic computing domains. The main focus is on the four core self-x capabilities—self-configuration, self-organisation, self-optimisation, and self-healing—while methodologically establishing a framework that allows for later integration of further self-mechanisms.

This article established a unified notion of terminology in the field and reviewed the state of the art in the field. We identified a gap in research towards an integrated cross-level solution, i.e., combining the distribution system functionality with those of home energy systems. Based on a system model for both levels, we identified parameters and decision freedom for autonomous or organic control modules operating in a distributed manner. This leads to a definition of the research scope of ODiS in the field.

Current and future work deals with the investigation and implementation of basic self-functionality for ODiS—and the resulting simulation-based proof of the feasibility. We start with the presented most urgent use cases and scenarios that allow the investigation of self-mechanisms in simulation using the OpSim framework. These will be refined, analysed systematically, and extended to cover the entire space of possible cases where robustness can be preserved using decentralised self-concepts. Subsequently, we investigate the appropriate evaluation setup and focus that have already been outlined within this article, which includes the definition of relevant metrics that go beyond standard energy grid aspects and especially consider the adaptation and organisation behaviour. Technically, we follow an incremental approach to realise the different self-functionality, starting with self-configuration, and the corresponding self-awareness (i.e., state estimation) technology in simple use cases, before continuing with more sophisticated scenarios and closing the gap towards self-organisation, self-optimisation, and finally self-healing. This includes a development of approaches to establish self-awareness of current and expected conditions, basic learning-enabled adaptation schemes for the OHEMS and ODMS level, collaboration schemes to allow for a take over of parts of the DMS functionality by collectives of OHEMS, and the resulting self-healing protocols.