Dynamic Overlapping Coalition Formation in Electricity Markets: An Extended Formal Model

Abstract

The future power system will be characterized by many small decentralized power plants—so-called distributed energy resources (DERs). The integration of these DERs is vital from an economic and grid operation point of view. One approach to this is the aggregation of such DERs. The formation of coalitions as an aggregation method has already been examined in the literature and applied in virtual power plants, active distribution networks, and microgrids. The spread of DERs also increases the need for flexibility and dynamics in the power grid. One approach to address this can be overlapping coalitions. Therefore, in this paper, we first performed an analysis of related work and, in this context, found no work on overlapping coalitions for energy use cases in the literature. We then described a method for dynamic coalition formation, called dynamic coalition in electricity markets (DYCE), and analyzed how DYCE would need to be extended to include overlapping coalition formation. The extension includes the phases of product portfolio optimization and the actual coalition formation. Our analysis of DYCE shows that the methods used for the optimization of the DYCE sub-tasks are not suitable for overlapping coalitions and would have to be replaced by other methods in order to be able to form overlapping coalitions.

1. Introduction

In order to achieve a sustainable future for our planet, the power grid is currently undergoing a transformation from a system with a few large energy generators (such as gas or coal-fired power plants) to a system based on distributed generation (DG), mainly based on distributed energy resources (DERs) and renewable energy sources (RESs) [1,2]. DERs are small- or medium-sized plants—typically under 10 MW in capacity. They are connected directly to the distribution grid and are allocated close to the load. They can consist of fuel generators, solar arrays, small wind farms, or battery energy storage systems (BESSs) [3].

With the further increase in DERs and a simultaneous dismantling of traditional generation plants, the power system will also face some challenges. The unpredictability and spatial dispersion of power production by DG systems will increase the complexity of system balancing (in spatial and temporal terms). Further challenges arise from the fact that DERs are mainly connected to low- and medium-voltage distribution systems, which will lead to a paradigm shift in power systems. In “traditional” power systems, a top-down philosophy typically prevails, which implies that power generation is mainly in the form of centrally controlled large power plants in the high-voltage grid. This development implies increased complexity in distribution grid management but also creates new opportunities and needs for overall system optimization. It allows distribution grids to participate in system operation actively and enables DERs to participate in distribution grid management actively. For example, they can provide ancillary services (AS) to the power grid, including balancing energy, frequency regulation, voltage support, congestion management, and reserves [3,4].

At the same time, integrating DERs creates new planning problems for system operators, as the system’s complexity increases significantly due to the many new plants. Operating such a decentralized and dynamic infrastructure will require the ability to solve large-scale problems in real time with hundreds of thousands of DERs simultaneously online [5]. For example, information from a multitude of plants must be collected, processed, and the plants must be planned and controlled afterwards. One approach to reducing this technical and operational complexity is to aggregate DERs into pools of plants.

The aggregation of DERs leads to a reduction in the technical and operational complexity and provides a significant amount of DER capacity to participate in electricity markets by turning many small entities into large market participants. It also enables integrated risk management along with advanced forecasting, resulting in an overall increase in combined reliability and addressing the requirements for the provision of electricity products by DERs [6,7]. Through the aggregation of generation or consumption potential, small-scale participants can meet the minimum quantity requirements of electricity wholesale markets, enabling them to engage in the trading of products surpassing their individual capacities. In addition, aggregating assets with different reliability levels or complementary capabilities provides the opportunity to reduce or offset unpredictable impacts. As a result, product fulfillment reliability increases, leading to cost savings for members. By pooling resources and targeting coordination among members, a collective pool enables a greater degree of adaptability in the face of changing market dynamics. This adaptability, in turn, enables members to compete more effectively in markets [8,9]. With respect to typical power plant characteristics such as minimum/maximum capacity and ramp criteria, aggregated DERs can behave like conventional power plants and participate in markets by selling power and AS or offer their flexibility [7,8].

The formation of coalitions is a concept from game theory and multi-agent system (MAS) research that can be part of a virtual power plant (VPP), microgrid (MG), or active distribution network (ADN) to allow operators to obtain a short-term, robust, and scalable aggregation method through a distributed design, taking into account the requirement for reliable product delivery as well as the increasing dynamics in the markets [9,10]. The coalition formation (CF) is performed in a decentralized manner by self-organized agents, with each agent taking control of one or more DER units. Thus, the agents act as representatives of the DER units. As part of the DER control, agents then form coalitions with other agents to maximize the utility of the units they control. For simplicity, in the further course, it will always be assumed that an agent controls only one DER unit. In the CF process, the agents dynamically group into coalitions, e.g., on a product basis, such that all DERs fulfill the same product. Aggregating DERs into coalitions offers several advantages. Thus, coalitions can be formed in a completely decentralized manner by agents, which takes into account the decentralization of the future energy system. In addition, various constraints regarding the formation of coalitions can be considered, for example, the provision of local energy products such as the provision of reactive power by DERs. Topological information of DERs can be explicitly considered in CF to take advantage of the aggregation of small plants while serving local power products [9]. Finally, CF as an aggregation concept can reduce the uncertainty about expected energy output [11]. An example of aggregation of DERs using CF is shown in Figure 1. In this example, different types of DERs in a low-voltage grid are controlled by agents and are aggregated into coalitions, e.g., to provide various types of electricity products.

CF in MGs aims to optimize energy sharing and management among different entities within the MG. By forming coalitions, these entities can work together more efficiently, share resources, and achieve common objectives. In an ADN, CF aims to optimize the use of DERs and improve the overall efficiency, reliability, and flexibility of the grid. The key benefits of CF in MGs and ADNs are optimal resource utilization, improved resilience and reliability, cost optimization, grid decentralization, and islanded operation.

VPPs are long-established aggregation concepts in which a group of DERs is aggregated, optimized, coordinated, and controlled to behave like a single resource. The VPP can function as a dispatchable unit in power system operations and as a tradeable unit in energy markets [12]. While some concepts on dynamic [13,14], hierarchical [15], holonic [16], and self-organizing [17,18] VPPs have been presented in the literature, VPPs in the field still are typically static systems with a fixed pool of assets [19]. This fixed pool is fully aggregated and marketed for a single purpose at a time, with the result that the flexibility potential of the assets cannot be fully exploited. However, flexibility plays a vital role in the future power system and, therefore, will become more economically viable in the future [20,21]. Hence, it is essential to develop and investigate concepts that exploit the full potential of plant flexibility while meeting the requirements for reliability in providing energy products provided by DERs.

The integration of CF into a VPP as an extended aggregation concept enables multiple products to be offered at one point in time through all the assets of a VPP. However, other concepts are needed to exploit the full flexibility potential of DER plants and, thus, make these aggregation concepts ready for use in the field. For example, to achieve the economical operation of a BESS and to use its full flexibility potential, multi-purpose usage of BESSs is necessary [22,23,24].

A concept that enables the provision of multiple products at one time is the model of cooperative games with overlapping coalitions, which can be described as an overlapping coalition formation (OCF) game [25,26]. This concept allows one DER plant to be in multiple coalitions at one time, which also allows that plant to fulfill various products simultaneously.

All approaches to the formation of coalitions as an aggregation method presented so far consider only disjoint coalitions, and, therefore, limit the approach, especially with regard to the use of flexibility potentials. The contribution of the paper is twofold. On the one hand, an extended formal model for OCF in the energy domain is presented. This formal model provides a clear and precise representation of the underlying system (the energy system), the process of OCF, and the underlying concepts from game theory. On the other hand, based on the formal model, an approach for dynamic CF with overlapping coalitions is presented. For this purpose, the requirements regarding the extensions for overlapping coalitions are discussed on the basis of an already existing approach and the extensions are outlined afterwards.

The remainder of this paper is structured as follows. In the following Section 2, we review the related work on the aggregation of DERs, including overlapping and non-overlapping CF in the energy domain. In Section 3, we will present a formal model for CF and show new formalizations to extend the model to overlapping CF. Using the formalizations from the previous section, in Section 4, we present a model for the dynamic formation of coalitions from the literature and discuss where the model would need to be extended to allow overlapping coalitions to be formed. Finally, the paper closes with a summary and outlook in Section 5.

2. Aggregation of Distributed Energy Resources

Due to the increasing number of DERs and the resulting challenges, the topic of the aggregation of DERs has gained increasing scientific attention in this decade. As a result, there is also a large body of work describing various aggregation concepts based on the formation of coalitions. Therefore, for the following literature review, Section 2.1 first introduces the criteria by which the literature comparison was conducted. Then, in Section 2.2, the relevant papers are described and compared based on the criteria. Finally, a summary of the literature review is given in Section 2.3.

2.1. Literature Review Criteria

The challenge of generating a coalition structure (CS), referred to as the coalition structure generation (CSG) problem, represents a fundamental abstraction of a key issue within MAS [27]. This issue revolves around how agents can organize themselves into cooperative groups to enhance their collective performance. Consequently, CF is a broad academic domain that garners substantial attention, particularly in game theory [10] and MAS research [28]. Nonetheless, deriving optimal CS configurations within MAS poses significant computational complexities [29], spurring researchers to devise an array of algorithms and heuristics to tackle this difficulty [30,31,32]. In addition to research on the process of CF, the equitable distribution of gains [33], and stability states of CS [34], there are also already some applications of CF as an aggregation method of DERs in the energy domain.

Concerning applications addressing the aggregation of DERs based on the concept of CF, a distinction can be made between different characteristics, which are briefly described below and then used to compare different approaches, as summarized in

Table 1. Comparison of the presented literature regarding the aggregation of DERs based on the formation of coalitions. The criterion was fully implemented: ✓. The criterion was not implemented: ✗. The criterion was partially implemented: (✓).

Paper	Objective	Endogenous vs. Exogenous	CSG Solution	Dynamic Aggregation	Reflection of Grid Topology	Flexibility Assessment	Reliability Assessment	Overlapping CF
[35]	Fit a given set of energy products	Endogenous	Combinatorial heuristics	✓	✗	✓	✗	✗
[39]	Reliable provision of primary frequency control	Exogenous	Solving multi-objective optimization problem	(✓)	✓	✗	✓	✗
[9]	Fulfill power products	Endogenous	Heuristic	✓	✓	(✓)	✗	✗
[40]	Integration of DERs	Endogenous	Negotiation mechanism	✓	✗	✗	✗	✗
[41]	Maximize expected profit	Exogenous	Solving an MILP	✗	✗	✗	✗	✗
[42]	Profit maximization	Exogenous	Solving bilevel optimization problem	✗	✗	✓	✗	✗
[43]	Discount through collective buying	Mixed	Integer programming	✗	(✓)	✗	✗	✗
[44]	Reduction in the variability of RES	Exogenous	Top k merit weighting PBIL	✓	✗	✗	✗	✗
[45]	Cost minimization in peer-to-peer energy trading	Exogenous	Deep Q-learning-based coalition formation	✗	(✓)	✗	✗	✗

. The work presented can be distinguished as to the CF’s objective, e.g., the reliable provision of electricity products or the economic optimization of a coalition of DERs, leading to different requirements regarding uncertainty handling or guarantees. Furthermore, a distinction can be made in how coalitions are formed, how the CS is created, i.e., how the membership of an agent in a coalition is determined. The CS can be initiated endogenously, e.g., by initiator agents and built up in a process, or the CS initiation can be exogenous, by an external central entity. This also concerns the coalition value calculation mechanism, which can be executed in a distributed manner by the agents or calculated centrally. It can also be distinguished how the problem of finding an optimal CS can be solved. For example, this can be formulated as an optimization problem and solved by suitable solvers [35], or learned by learning algorithms [36]. It can also be distinguished by whether the CF is temporally static or dynamic [37], that is, whether the CS is fixed or whether the structure can change continuously, even after the CSG process. The presented approaches also differ in how the topology of the power grid is reflected in the CSG concept [38], which is necessary for specific application purposes, such as the provision of reactive power. In addition, the approaches can be distinguished by whether the flexibility of the individual systems is taken into account in the CF process and whether a reliability assessment or reliability requirements about the coalitions are taken into account. Finally, the approaches can be distinguished in terms of whether overlapping coalitions are allowed when creating a CS.

2.2. Coalition Formation Literature

In [35], Bremer and Lehnhoff present a decentralized CF approach based on the combinatorial optimization heuristics for distributed agents (COHDA) for predictive scheduling with the characteristic of keeping all information about the local decision base and local operational constraints private. Two interlocking optimization processes orchestrate an overall procedure that adapts a CS to fit a given set of energy products best while considering the DER flexibilities. The approach is then evaluated in several simulation scenarios with different types of established DER integration models and extended to the induced surplus distribution use case. During the CF process with COHDA, no topological information about the power system or the coalition reliability is considered, and the results of CF are disjoint coalitions.

Lehnhoff et al. present a methodology in [39] to uncover decentralized coalitions comprising units operated by agents capable of providing frequency response reserve to meet specific reliability goals. In their approach, they introduce the concepts of base and core coalitions, wherein the core coalition agents actively partake in providing AS. The task of identifying the most suitable agent coalitions is cast as an optimization problem. The goal is to select a core coalition with the longest operational duration and minimal costs, guided by the attributes of the agents while ensuring a predefined level of reliability. To tackle this challenge, they employ a multi-objective optimization approach facilitated by solvers such as IBM ILOG CPLEX. It is important to note, however, that this approach does not account for power plant flexibilities and exclusively forms non-overlapping coalitions.

The doctoral thesis of Beer [9] introduces a game-theoretic framework for non- overlapping CF named dynamic coalition formation in electricity markets, abbreviated as DYCE. This approach, centered around agents, addresses dynamic CF within electricity markets. The core objective of DYCE is to optimize distributed energy resources (DERs) by efficiently fulfilling aggregated power products. What sets DYCE apart from existing aggregation methods is its fully distributed nature, which takes into account the individual preferences of units as well as the broader supply system. DYCE incorporates the grid’s topological information, facilitating the trading of electricity products localized to specific areas in the power grid. Since coalitions are structured to fulfill singular products, they exhibit temporal flexibility to adapt to changing circumstances. Once a successful trade of a product occurs, the surplus gained is shared equitably among the coalition members, guided by a game-theoretic distribution model. The process of CF is decentralized in DYCE, initiating with agents forming individual coalitions and progressively evolving these coalitions to better fulfill their intended products. This iterative process involves agents regrouping based on a heuristic approach. Inter-agent communication adheres to the ContractNet protocol, with each agent aiming to maximize the average coalition value, thus contributing to the overall value of the CS. Due to the inherent complexity of the problem, the implemented heuristics prevent the formation of overlapping coalitions. While flexibility is introduced in the formal model of DYCE, it is not considered further for CF.

Another approach for dynamic CF of DERs is presented in [40] by Ossowski et al. Finding an optimal CS is first modeled as a dynamic CF game. The actual CF process is achieved by opportunistic aggregation of self-organized agents using a negotiation mechanism, which makes the approach fully distributed. The formation process consists of three phases: coalition initiation, provider aggregation phase, and consumer aggregation phase. The formation and coalition self-adaptation mechanisms introduced perform an open-ended adaptation of groups of organizational agents, converging towards stable configurations. In the context of the experiments conducted, no information about the grid topology and only non-overlapping coalitions were considered. While the approach is motivated by maximizing reliability and the benefits of flexibility, neither is explicitly considered in the approach presented by the authors.

In their work [41], Shabanzadeh et al. introduce a medium-term model for CF involving diverse DERs within a VPP. The main aim is to maximize the anticipated profit of the VPP. Their model comprises two distinct stages: The initial stage focuses on optimal decision making concerning the selection of DERs for VPP coalitions, the agreed-upon quantities through bilateral negotiation, and the determination of the type and quantity of forward market contracts. The subsequent stage involves decisions based on the most likely outcomes of stochastic pricing in the day-ahead market. Both stages are orchestrated by a VPP manager, leading to predetermined CS. The authors frame the CSG problem as a mixed-integer programming challenge that can be tackled using high-performance solvers. The approach they propose solely accommodates non-overlapping coalitions. No consideration is given to incorporating information about the power grid’s topology during the coalition formation process. Furthermore, neither the flexibilities of DERs nor the reliability of coalitions is taken into account within their proposed framework.

In [42], Valinejad et al. introduce a CF approach tailored for microgrids (MGs) equipped with DERs and BESSs in energy markets. The authors present a hierarchical structure consisting of two levels to design and plan the integration of DERs and BESS within home-microgrids (H-MGs). Their approach leverages demand response strategies to harness the flexibility potential within these systems. At the upper level of this hierarchical structure, a competitive scenario unfolds among H-MGs, aiming to maximize their individual and collective earnings through the process of CF with other H-MGs. To realize this, the upper-level problem is deconstructed into a series of lower-level market-clearing challenges. This comprehensive model addresses both electricity and heat markets concurrently. Subsequently, the intricate algorithm of this bilevel hierarchical framework is resolved by transforming it into an equivalent single-level model, utilizing the Karush–Kuhn–Tucker optimization conditions. The approach is adept at accommodating various forms of DERs but does not cater to coalitions with overlapping memberships. The quest for an optimal CS is formulated as an optimization problem, which is tackled using a central solver. The findings underscore that the establishment of coalitions among H-MGs within a grid not only influences the scheduling and control of generated power but also has repercussions on consumer behavior and demand within demand response initiatives. No topology information was used during the CF process, and the reliability was not assessed.

In [43], Vinyals et al. present an approach using social relationships of customers to organize them into coalitions of virtual electricity consumers to gain discounts through the collective buying of electricity. Therefore, the problem of finding an optimal CS is modeled as a coalitional game, and an algorithm based on linear programming to form coalitions is provided. The CF process consists of three differentiated activities determining which coalitions can be formed and the coalition value calculation, finding a CS with maximal value, and dividing the payments generated among the coalition members. Multiple agents calculate the coalition values while creating a CS is solved using an integer programming approach by a single entity. No overlapping coalitions are considered, no information about the topology of the power system is considered during the formation of the coalitions, and neither the plant’s flexibilities nor the reliability of the resulting coalitions are considered in their approach.

An approach to finding optimal large-scale CS in smart grids based on hierarchical population-based learning is proposed by Lee et al. [44]. The objective of the CF is the aggregation to reduce fluctuations in RES feed-in. To form the coalitions, a Top k merit weighting population-based incremental learning (PBIL-MW) algorithm is used, which can consider many plants in the CF process. The learning algorithm is part of a central agent that subsequently specifies a CS. Different types of plants, such as BESS, PV plants, or wind turbines, were considered as part of the experiments. In the experiments conducted, the authors show that the learning-based approach presented scales well in terms of runtime for solving the CSG up to a number of 80 agents, whereas previous methods based on dynamic programming scale only up to a size of 20 agents. The CS is formed only from disjoint coalitions, and no topological information, flexibility, or reliability are considered in the formation process.

Another learning-based approach is proposed by Sadeghi et al. in [45]. Coalitions are formed to address the problem of minimizing costs in MG communities while considering the dynamic nature of the system. The objective of one coalition is to efficiently group buyers and sellers in a way that minimizes the cost of energy transactions compared to buying or selling energy to the macrogrid. To create suitable CS, the deep-learning-based algorithm deep Q-learning-based CF (DQN-CF) is introduced. Only households in the MG are considered for CF, so the approach only allows for homogeneous plants. In the approach presented, plant flexibility and reliability of coalitions are not considered.

2.3. Summary

In Section 2.1, we first described various criteria for reviewing CF approaches from the literature. In Section 2.2, different approaches to aggregate DERs by CF were presented and compared based on the criteria introduced before. An overview of the presented CF approaches is given in Table 1.

In the brief literature review, we examined nine papers in the context of CF for power plant aggregation. The literature review shows that the objectives of CF are mostly to minimize costs or reduce the variability in RES. In four of the nine approaches, the CS is centrally or externally specified (exogenous). In three approaches, the CS is formed decentrally, and one approach pursues both. Finding a CS is usually defined in the papers as an optimization problem and solved by, e.g., a heuristic. Most approaches do not consider asset flexibility for CF (six of nine), and only one paper considers asset and coalition reliability. None of the approaches allow for the formation of overlapping coalitions. During the brief literature search, no work on OCF in the energy context could be found either.

3. Formalizing Overlapping Coalition Formation

In this section, some formalizations for CF are introduced, which will later serve as the basis for describing a method for CF. For this purpose, basic formalizations of CF in the context of CPES are first introduced in Section 3.1. In Section 3.2, these formalizations are extended with respect to overlapping coalitions.

3.1. Basic Formal Model

The following definitions are mostly taken from [9,13] and modified by us. The formalizations can be divided into four domains: physical device representation in Section 3.1.1, coalition in Section 3.1.2, product in Section 3.1.3, and optimization problem in Section 3.1.4.

3.1.1. Domain 1—Physical Device Representation

The power grid is relevant in the context of CF for assessing the neighborhood of units, e.g., for determining the sensitivity of reactive power products. A power grid is denoted as $G=(V_G,E_G)$, represented through a weighted graph structure. Here, $V_G=\{v_{G,1},\dots ,v_{G,n}\}$ constitutes a collection of vertices corresponding to the nodes within the grid, while $E_G$ comprises a set of edges $\{v_{G,i},v_{G,j}\}$, where $v_{G,i},v_{G,j}\in V_G$, and $i\ne j$, signifying the power transmission lines. Each specific edge $\{v_{G,i},v_{G,j}\}$ possesses a numerical weight termed as the distance weight, denoted by $dw(v_{G,i},v_{G,j})$ which quantifies the spatial separation, with higher values indicating greater distances between the respective nodes. The calculation of each distance weight relies on a predefined assortment of distance criteria referred to as $c_{dis}$, which encompasses parameters such as line impedance.

Units are atomic elements connected to the power grid and are of the type producers, consumers, or both. A unit u is a technical installation physically connected to a power grid G via a grid connection point and can influence current or voltage. Let U be the set of all units.

Units are represented in terms of information technology via agents assigned to them. Therefore, there is an agent for each unit considered in the CF process. Let $A=\{a_1,\dots ,a_n\}$ be a finite non-empty set of agents. Each agent $a_i$ is assigned a non-empty subset $U_{a_i}$ of units $u_i\in U$ according to the assignment $ua$ (unit assignment): $ua:A\to \mathcal{P}\left(U\right),ua\left(a_i\right)\mapsto U_{a_i},U_{a_i}\subseteq U$. According to their intended purpose, agents are also referred to as participants in an electricity market. For ease of understanding, we assume that an agent represents exactly one unit, i.e., $ua\left(a_i\right)=u_i$. From an agent’s perspective, the task of CF is directly related to planning the operations of its associated units. In general, all planning activities take place concerning a planning horizon, which is a discrete-time frame for which agents plan in advance.

Let $t_{bu}$ represent a time base unit, and $T\subseteq {\mathbb{N}}_0$ denote the set of time values in $t_{bu}$. The planning horizon is defined as $T_{pl}=\left\{t_{pl}^{\left(i\right)}\right|0\le i\le i_{max}\}$, with $t_{pl}^{\left(i\right)}=[t_{i·j},t_{(i+1)·j})$ as a planning interval. Here, $t_{max}$ is the planning horizon length in $\mathbb{N}$, $\Delta t$ is the planning interval length in $\mathbb{N}$, both measured in $t_{bu}$. We have $t_{i·j}\in T$, mapped to a real-time point $t_{rt}$ through a time mapping function $\tau :T\to \mathbb{R},t\mapsto t_{rt}$, where i is an interval index and $i_{max}=\frac{t_{max}}{j·t_u}-1$ is the maximum interval index within a planning horizon. Here, $t_u\in \mathbb{N}$ is a constant in $t_{bu}$, determining the time unit of a planning interval. $\Delta t$ is a multiple of $t_u$, and $j=\frac{\Delta t}{t_u}$ is the interval length, measured in the time unit defined by $t_u$.

Given the definition of a planning horizon $T_{pl}$, the operation schedule of a unit u is defined as a function $o{s}_u:T_{pl}\to {\mathbb{R}}^3$, where $o{s}_u\left(t_{pl}^{\left(i\right)}\right)=(e_u^{\left(i\right)},er{r}_u^{\left(i\right)},c_u^{\left(i\right)})$ specifies the amount of electrical energy $e_u^{\left(i\right)}$ which is produced or consumed by unit u with an error $er{r}_u^{\left(i\right)}$ at cost $c_u^{\left(i\right)}$ in planning interval $t_{pl}^{\left(i\right)}$. The electricity amount $e_u^{\left(i\right)}$ can be equivalently expressed as amount of electric power $po{w}_u^{\left(i\right)}=\frac{e_u^{\left(i\right)}}{\Delta t}$ which is provided or demanded by unit u over a planning interval $t_{pl}^{\left(i\right)}$. Let $e_{o{s}_u}$ denote the electrical energy produced or consumed by unit u, accompanied by error $er{r}_{o{s}_u}$ at a cost of $c_{o{s}_u}$ predefined by $o{s}_u$ within $T_{pl}$. The collection of all operation schedules is $O{S}_U$, while the feasible operating schedules of agent $a_i$ are defined as $O{S}_{a_i}$.

Specific unit types can offer flexibility about their scheduled amounts of electrical energy. Given a planning horizon $T_{pl}$ and an operation schedule $o{s}_u$, the operational flexibility of a unit u within a planning interval $t_{pl}^{\left(i\right)}$ is denoted by $o{f}_u^{\left(i\right)}=(e_{u,-}^{\left(i\right)},e_{u,+}^{\left(i\right)})\in {\mathbb{R}}^2$. This tuple indicates the adjustable range of produced or consumed electrical energy. Here, $e_{u,-}^{\left(i\right)}$ and $e_{u,+}^{\left(i\right)}$ are relative to $e_u^{\left(i\right)}$ and define the negative and positive bounds for modifications. These bounds represent the extent of energy that can be added to or subtracted from $e_u^{\left(i\right)}$. The coalitional flexibility of a coalition $\mathcal{C}$ during a planning interval $t_{pl}^{\left(i\right)}$, termed as $o{f}_{\mathcal{C}}^{\left(i\right)}$, is the summation of individual unit flexibilities within $\mathcal{C}$:

$$o{f}_{\mathcal{C}}^{\left(i\right)}=(e_{\mathcal{C},-}^{\left(i\right)},e_{\mathcal{C},+}^{\left(i\right)}),\mathrm{where}{e}_{\mathcal{C},-}^{\left(i\right)}=\sum _{u\in U_{\mathcal{C}}}{e}_{u,-}^{\left(i\right)}\mathrm{and}{e}_{\mathcal{C},+}^{\left(i\right)}=\sum _{u\in U_{\mathcal{C}}}{e}_{u,+}^{\left(i\right)}$$

(1)

This aggregation accounts for the operational flexibility of the member units (it is important to note that this simplified approach does not incorporate temporal and technical dependencies in flexibility modeling, whereas more intricate models, such as [22], do encompass these complexities). Given this definition, a unit u can assume multiple types with respect to its capacity to generate or consume electrical energy and its operational flexibility. The operational flexibility of a unit extends certain advantages to the controlling agent during CF. This expanded array of choices enables the agent to decide on trading products or contributing to a coalition with greater flexibility. The integration of flexibility within coalitions generally leads to enhanced dependability in fulfilling product commitments. This is attributed to the capability to adapt to unexpected deviations from scheduled electricity amounts. In the context of CF, operational flexibilities acquire even more significance. Coalition members gain the ability to compensate for errors committed by others, thus averting the need for costly balancing reserves.

3.1.2. Domain 2—Coalition

Given a set of agents A, a coalition is defined as a finite, non-empty set $\mathcal{C}\subseteq A$. The set of all possible coalitions is denoted as $\tilde{\mathcal{C}}=\mathcal{P}\left(A\right)\setminus ⌀$. Agents $a\in \mathcal{C}$ are also acknowledged as members of $\mathcal{C}$. The size of a coalition is defined as the number of its members $\left|\mathcal{C}\right|$, where in the case of $\left|\mathcal{C}\right|=1$, it can also be denoted as a singleton coalition. Let $U_{\mathcal{C}}$ be the set of all assigned units to $\mathcal{C}$. A coalition is an organizational aggregation of one or more agents for the trade of a power product. The commitment of each member to a coalition is limited by time, as a coalition dissolves once a product has been physically fulfilled. CF results in a partition that divides the entire set of agents into distinct subsets. A coalition structure $\mathcal{CS}$ is a set of coalitions that partitions the set of all agents A into disjunct coalitions, i.e.,

$$\forall \mathcal{C},{\mathcal{C}}^{\prime }\in \mathcal{CS}with\mathcal{C}\ne {\mathcal{C}}^{\prime }:\mathcal{C}\cap {\mathcal{C}}^{\prime }=\varnothing \wedge \bigcup _{\mathcal{C}\in \mathcal{CS}}\mathcal{C}=A$$

(2)

The size of CS is defined as the number of comprised coalitions $\left|\mathcal{CS}\right|$. The set of all possible CS is given by $\widetilde{\mathcal{CS}}$, where the number of valid coalition structures $|\widetilde{\mathcal{CS}}|=B_n$ for non-OCF is equal to the Bell number $B_n$ with $n=\left|A\right|$ being the number of agents.

The requirement that all coalitions of a given $\mathcal{CS}$ have to be disjunct (see Equation (2)) guarantees that these are independent of each other with regard to their products, leading to only non-overlapping $\mathcal{CS}$. The value v of a coalition $\mathcal{C}$ is defined by the coalition value and is defined as a characteristic function $v:\tilde{\mathcal{C}}\to \mathbb{R}$, where $v\left(\mathcal{C}\right)$ is also referred as the value (or worth) of $\mathcal{C}$. The function maps each coalition $\mathcal{C}$ to its value $v\left(\mathcal{C}\right)$. While a higher coalition value typically results in higher revenues or lower expenses for a coalition, it does not directly correspond to the amount of money it ultimately receives or pays when trading a power product. Therefore, the characteristic function describes the payoff available to coalition members but does not define a way of distributing these payoffs.

3.1.3. Domain 3—Product

Based on a planning horizon $T_{pl}$, we define a product horizon $T_{pr}^{\left(p\right)}\subseteq T_{pl}$ of a product p as a set $T_{pr}^{\left(p\right)}=\{t_{pr}^{\left(0\right)},\dots ,t_{pr}^{\left(j_{max}\right)}\}$ with $0\le j_{max}\le i_{max}$, where $t^{\left(j\right)}\in T_{pr}^{\left(p\right)}$ is termed as the product interval. Thus, a product horizon $T_{pr}^{\left(p\right)}$ is a set of product intervals corresponding to the planning intervals of a particular planning horizon $T_{pl}$ in which a product p is to be physically fulfilled.

Given the definition of the product horizon, a product p determines the amount of electrical energy $e_p^{\left(i\right)}$ which is produced or consumed by a coalition $\mathcal{C}$ with a particular error $er{r}_p^{\left(i\right)}$ at a particular cost $c_p^{\left(i\right)}$ in each product interval $t_{pr}^{\left(i\right)}$, such that $p:T_{pr}^{\left(p\right)}\to {\mathbb{R}}^3$, where $p\left(t_{pr}^{\left(i\right)}\right)=(e_p^{\left(i\right)},er{r}_p^{\left(i\right)},c_p^{\left(i\right)})$. Let $e_{p_{\mathcal{C}}}$ be the amount of electrical energy consumed or produced by a coalition $\mathcal{C}$ with error $er{r}_{p_{\mathcal{C}}}$ at cost $c_{p_{\mathcal{C}}}$ in $T_{pr}^{\left(p\right)}$. The set of all potential products is represented as $\tilde{P}$.

To fulfill its intended target products, an agent $a_i$ joins or initiates coalitions to which it contributes according to the technical capabilities of its units $U_{a_i}$. Given an agent $a_i$ controlling a unit $ua\left(a_i\right)$ = $U_{a_i}=u$, the contribution of a to a product p is defined as the function $co{n}_{a,p}:T_{pr}^{\left(p\right)}\to {\mathbb{R}}^3$, where $co{n}_{a,p}\left(t_{pr}^{\left(i\right)}\right)=(e_u^{\left(i\right)},er{r}_u^{\left(i\right)},c_u^{\left(i\right)})$ specifies the amount of electrical energy $e_u^{\left(i\right)}$ which is produced or consumed by u with error $er{r}_u^{\left(i\right)}$ at cost $c_u^{\left(i\right)}$ in the product interval $t_{pr}^{\left(i\right)}$ as determined by its operation schedule $o{s}_u\left(t_{pl}^{\left(i\right)}\right)$. Let $CO{N}_{\mathcal{C},p}$ be the set of all contributions of the members of a coalition $\mathcal{C}$. Since, so far, only non-overlapping coalitions can be formed, an agent a can only contribute to one product in each product horizon $T_{pr}^{\left(p\right)}$.

The payoff $\breve{\rho }$ resulting from a trade of p by a coalition $\mathcal{C}$ is defined as the final total cost which the members of $\mathcal{C}$ receive from or pays to its trading partner. Given a coalition’s payoff $\breve{\rho }$, the payoff distribution is defined by a function $\psi :\tilde{\mathcal{C}}\times \mathbb{R}\to {\mathbb{R}}^{\left|\mathcal{C}\right|}$, where the allocation vector $\psi \left((\mathcal{C},\breve{\rho })\right)={\mathbf{x}}_{\mathcal{C}}$ specifies the individual shares $(x_1,\dots ,x_n)$ for the coalition members $\{a_1,\dots ,a_n\}$. The computation of ${\mathbf{x}}_{\mathcal{C}}$ is predicated on a set of criteria, denoted as $c_{\psi }$. The distribution function $\psi$ allocates the coalition’s payoff $\breve{\rho }$ among its members. The shares $x_i$ delineate individual revenues or expenses. This distribution hinges on dedicated criteria $c_{\psi }$ that are relevant for its calculation, such as a member’s contributed electrical energy amount or its reliability in ensuring physical product fulfillment.

Given a planning horizon $T_{pl}$ and a template portfolio $TP=\{p_{tmp,1},\dots ,p_{tmp,n}\}$ containing product templates $p_{tmp}$, a product portfolio $PP=\{p_{tar,1},\dots ,p_{tar,n}\}$ is defined as a set of n template-compliant target products $p_{tar}$ which an agent intends to trade in $T_{pl}$ at market M by forming coalitions. In this context, a target product $p_{tar}$ is defined as the function $p_{tar}:T_{pr}^{\left(p_{tar}\right)}\to {\mathbb{R}}^3$, where $p_{tar}\left(t_{pr}^{\left(i\right)}\right)=(e_{p_{tar}}^{\left(i\right)},er{r}_{p_{tar}}^{\left(i\right)},c_{p_{tar}}^{\left(i\right)})$ specifies the target amount of electrical energy $e_{p_{tar}}^{\left(i\right)}$, which is produced or consumed with target error $er{r}_{p_{tar}}^{\left(i\right)}$ at target cost $c_{p_{tar}}^{\left(i\right)}$ in $t_{pr}^{\left(i\right)}$.

The time model, in particular the relationship between the real time and the product, and the planning time, as well as the notation used for the different times, is shown in Figure 2.

3.1.4. Domain 4—Optimization Problem

Given these formalizations, the problem of finding an optimal coalition structure ${\mathcal{CS}}^{\ast }$ can be seen as a characteristic function game (CFG) or non-OCF game $\mathsf{\Gamma }$. In a CFG, a coalition’s value depends solely on its members’ identities. Such a game is given by a tuple $\mathsf{\Gamma }=(A,v)$, where A is a set of agents and v is the characteristic value function. In CFGs, for any coalition $\mathcal{C}\subseteq A$, the value of a coalition structure $\mathcal{CS}\in \widetilde{\mathcal{CS}}$ is denoted by $V\left(\mathcal{CS}\right)$ and is given by $V\left(\mathcal{CS}\right)={\sum }_{{\mathcal{C}}_i\in \mathcal{CS}}v\left({\mathcal{C}}_i\right)$. The coalition structure generation problem is the problem of finding a $\mathcal{CS}$ over A whose value $V\left(\mathcal{CS}\right)$ is maximal. In CFGs, the problem of finding ${\mathcal{CS}}^{\ast }$ can be defined as:

$${\mathcal{CS}}^{\ast }=\underset{\mathcal{CS}\in \widetilde{\mathcal{CS}},s.t.H{C}_{\tilde{P}}}{arg\; max}V\left(\mathcal{CS}\right),V\left(\mathcal{CS}\right)=\sum _{{\mathcal{C}}_i\in \mathcal{CS}}v({\mathcal{C}}_i)$$

(3)

This is a combinatorial optimization problem, which in principle, could be solved to optimality using a brute-force search. However, this is not practicable, as the number of possible coalition structures over n agents is $\mathcal{O}(2^n-1)$. The CSG problem is NP-complete, and research mainly investigates concepts that provide an efficient approximation to an optimal solution [46]. In particular, algorithms from the field of dynamic programming (DP), anytime exact algorithms such as integer-partition-based search (IP) or a combination of both approaches, such as the ODP-IP algorithm [30], are used to find optimal CS [27].

3.2. Extended Formal Model

In the following, we extend the model of a non-OCF game from the previous subsection to a cooperative game with overlapping coalitions. The game-theoretic definitions of overlapping coalitions are mainly taken from [25]. The presented model can be applied when agents allocate different portions of their associated units to contribute to multiple products as members of different coalitions simultaneously. We define overlapping coalition formation as the process of forming coalitions of entities with shared goals or interests to achieve a specific objective. These coalitions may overlap, meaning that some entities may belong to multiple coalitions simultaneously.

Extending Agents Contributions

In the game theory literature, a game with overlapping coalitions is typically defined as an OCF game with a set of players $N=\{1,\dots ,n\}$ and a function $v:{[0,1]}^n\to \mathbb{R}$, where $v\left(0^n\right)=0$ [25]. An overlapping coalition is now given by a vector $\mathbf{r}=(r_1,\dots ,r_n)$ where $r_j$ is the fraction of agent j’s resources contribution and the value function v maps each partial coalition $\mathbf{r}$ to its corresponding payoff. This approach allows not only the pure membership in a coalition to be mapped (which would be the case if always $r_j\in \{0,1\}$) but also to what extent an agent $a_j$ distributes its resource among different coalitions, i.e., $r_j\in [0,1]$.

An agent’s a contribution to a coalition is already explicit in the formalization for non-overlapping CF. The definition of a non-overlapping coalition includes only membership information over its members, but agents explicitly specify a contribution $co{n}_{a,p}$ to a product p. Since, in our context, products are provided by coalitions, there is already a link between the faction of an agent’s resource and the contribution to a product.

In Equation (2), we described the constraint for disjoint coalitions that applies to non-OCF games. To make the previous concepts applicable to overlapping coalitions or OCF games, the condition in Equation (2) does not apply to OCF. This means that the intersection of both coalitions now does not necessarily have to be an empty set but can also include the agents that contribute to both coalitions.

Since an agent a can now participate in more than one coalition in a planning horizon $T_{pl}$, its contributions $co{n}_{a,p}$ and $co{n}_{a,p^{\prime }}$ fulfilling products p and $p^{\prime }$ having the same product horizon $T_{pr}^p=T_{pr}^{p^{\prime }}$ or at least overlapping product intervals $t_{pr}\in T_{pr}^p\wedge t_{pr}\in T_{pr}^{p^{\prime }}$, is bounded by the operation schedule considering a’s operational flexibility, i.e., the amount of electrical energy that a contributes to all products in a time interval must not exceed the amount determined by the operational schedule, including the possible positive flexibility bound. The extended contribution function

$$\begin{array}{c}\hfill co{n}_a^{+}:T_{pl}\to {\mathbb{R}}^{3\times n},\mathrm{where}co{n}_a^{+}\left(t_{pl}^{\left(i\right)}\right)=({\mathbf{e}}_u^{\left(i\right)},{\mathbf{err}}_u^{\left(i\right)},{\mathbf{c}}_u^{\left(i\right)})\end{array}$$

(4)

now determines a tuple of vectors containing the electrical amount ${\mathbf{e}}_u^{\left(i\right)}$, with error ${\mathbf{err}}_u^{\left(i\right)}$ at cost ${\mathbf{c}}_u^{\left(i\right)}$ for all n products for which a contributes in $t_{pl}^{\left(i\right)}$:

$$\begin{array}{cccccc}\hfill {\mathbf{e}}_u^{\left(i\right)}& =\left[\begin{array}{c}{e}_{p_1}^{\left(i\right)}\\ e_{p_2}^{\left(i\right)}\\ ⋮\\ e_{p_n}^{\left(i\right)}\end{array}\right],\hfill & \hfill {\mathbf{err}}_u^{\left(i\right)}& =\left[\begin{array}{c}er{r}_{p_1}^{\left(i\right)}\\ er{r}_{p_2}^{\left(i\right)}\\ ⋮\\ er{r}_{p_n}^{\left(i\right)}\end{array}\right],\hfill & \hfill {\mathbf{c}}_u^{\left(i\right)}& =\left[\begin{array}{c}{c}_{p_1}^{\left(i\right)}\\ c_{p_2}^{\left(i\right)}\\ ⋮\\ c_{p_n}^{\left(i\right)}\end{array}\right].\hfill \end{array}$$

(5)

It must hold that:

$$\forall t^{\left(i\right)}\in T_{pl}:{\kappa }_a^{\left(i\right)}\le (e_u^{\left(i\right)}+e_{u,+}^{\left(i\right)}),\mathrm{where}{\kappa }_a^{\left(i\right)}=\sum _{e\in {\mathbf{e}}_u^{\left(i\right)}}{e}_{p_j}^{\left(i\right)}$$

(6)

4. Discussion of Overlapping Coalition Formation in DYCE

The CF process can be divided into three phases: (1) forming a coalition structure involves each agent joining a coalition. This can be achieved either through the agents making decisions independently using a bargaining process, or through external means such as a system designer. The result of this phase is a CS. (2) Solving the (optimization) problem of each coalition—this includes fulfilling the coalition’s purpose, such as solving an optimization problem or providing an electricity product. (3) Dividing the reward of each coalition among its members—if a coalition receives benefits from cooperative action, the members of the coalition will need to decide how to (e.g., fairly) distribute these benefits among themselves [27].

In the following Section 4.1, we first provide an overview of the DYCE methodology. In Section 4.2, we briefly explain the different value maximization levels in DYCE before we discuss for each of the phases in DYCE what problems exist within the phase regarding the formation of overlapping coalitions in Section 4.3.

4.1. Introducing the DYCE Method

The DYCE method presented by Beer in [9] is a method for dynamic CF in electricity markets, including the three phases above. DYCE involves an agent performing four activities in an iterative and partially parallel manner, as shown in Figure 3. Each activity addresses a specific sub-problem of the overall problem and involves solving an associated optimization problem. The DYCE method improves both local and global utility by considering the individual benefits of agents during product portfolio management and optimizing the value of the CS when forming coalitions.

4.1.1. Product Portfolio Management

In this phase, the agents in the DYCE activity have three main tasks. The first is to build a portfolio of target products that it will trade within its planning horizon, using product templates and price predictions. The second is to choose an operation schedule from its controlled units, which will be used to make contributions to coalitions during formation processes. The portfolio and schedule are designed to optimize local utility. Finally, the agent plans and carries out CF processes for its identified target products according to the created schedule.

4.1.2. Neighborhood Formation

If an agent initiates negotiations during CF, it first creates a group of nearest neighbors based on the physical distance between units in the grid. This group, called a neighborhood, is limited in size to reduce communication and computational costs while considering the layout of the grid. If the current neighbors are not successful in forming a coalition, the agent can expand its neighborhood to include a larger number of potential partners.

4.1.3. Coalition Formation

During the third DYCE activity, agents form coalitions with their neighbors to fulfill their target products jointly. The formation process aims to maximize the overall value of the CS. An agent has two options if a coalition cannot be formed with the current neighbors. The first is to expand its neighborhood to include more potential partners and increase the chances of forming a successful coalition (see condition $c_1$ in Figure 3). As a second option, if the neighborhood already covers the whole grid, the agent can try to create a new portfolio of target products as an alternative (see condition $c_2$ in Figure 3). If neither of these options is possible, the agent will end the formation process and schedule the following target product for which to form a coalition (see condition $c_3$ in Figure 3). If the agent is a member of a successful coalition, it will complete the final activity of the overall process before ending the process (see condition $c_4$ in Figure 3).

4.1.4. Payoff Distribution

If a coalition is formed successfully, it will seek to trade its fulfilled target product and distribute the resulting payoff fairly among its members. The distribution is typically based on the Shapley value, a game-theoretical model that determines the individual shares of the coalition’s utility. The resulting distribution vector specifies the share that each member receives from the cooperation.

4.2. Value Maximization Level

When forming coalitions, agents can maximize value with regard to different levels. Therefore, we first introduce the different levels of value maximization for both disjoint and overlapping CF. Depending on the agents’ utilized objective function, value optimization within the context of CF can occur on three different levels [9]:

• At the agent level (a-level), each agent $a_i$ strives to maximize its personal share $x_i$ acquired as a member of a coalition $\mathcal{C}\in \mathcal{CS}$, as indicated by the objective function

  $$\underset{\mathcal{C}\in \mathcal{CS}}{max}{x}_{i,\mathcal{C}}$$

  (7)
• At the coalition level (C-level), the agents try to maximize the value function of single coalitions $\mathcal{C}\in \mathcal{CS}$ as indicated by the objective function

  $$\underset{\mathcal{C}\in \mathcal{CS}}{max}v\left(\mathcal{C}\right)$$

  (8)
• At the coalition structure level (CS-level), the agents try to maximize the value of a coalition structure $\mathcal{CS}$ as reflected by the objective function

  $$\underset{\mathcal{C}\in \mathcal{CS}}{max}\overline{v\left(\mathcal{C}\right)},\mathrm{where}\overline{v\left(\mathcal{C}\right)}=\frac{{\sum }_{\mathcal{C}\in \mathcal{CS}}v\left(\mathcal{C}\right)}{\left|\mathcal{CS}\right|}$$

  (9)

The choice of objective function an agent employs for its specific task is contingent upon its optimization objective. This can entail optimizing its individual payoff (a-level), the value of individual coalitions (C-level), or the value of the entire coalition structure (CS-level). From a more pragmatic perspective, these choices align with optimizing the value for the unit owner, unit pools, or the entire system. Figure 4 shows the different levels of value maximization in abstract terms, including the differences between non-overlapping and overlapping CF. The figure shows that at the C- and CS-levels, overlapping coalitions create dependencies on other coalitions that do not exist for disjoint coalitions.

The objectives pursued across distinct levels might conflict. For example, when maximizing value on the C-level, an agent might opt to join a coalition even if it leads to a reduction in its own share or the overall value of the CS. The choice of objective function for an agent ultimately hinges on the specific use case and regulatory constraints defined by hard regulations.

4.3. Overlapping Coalition Formation in DYCE

In the remainder of this section, we will systematically examine the four stages of DYCE and identify the aspects within the model that necessitate a modification to facilitate the emergence of overlapping coalitions.

4.3.1. Product Portfolio Management for OCF

In the phase of product portfolio management (PPM), each agent tries to maximize its utility function by selecting an optimal template portfolio $TP$ and operation schedule $o{s}_u$ to create a product portfolio $PP$. The expected utility of a producer $u{t}_p$ or consumer $u{t}_p$ for a given operation schedule $o{s}_u$ and product template $p_{tmp}$ is given by

$$utility(p_{tmp},o{s}_u)=\left\{\begin{array}{cc}\sum _{i=0}^{j_{max}}{e}_u^{\left(i\right)}·c_{p_{tmp}}^{\left(i\right)}-e_u^{\left(i\right)}·c_u^{\left(i\right)}\hfill & \mathrm{if}u\mathrm{is}\mathrm{of}\mathrm{type}u{t}_p,\hfill \\ \sum _{i=0}^{j_{max}}{e}_u^{\left(i\right)}·c_u^{\left(i\right)}-e_u^{\left(i\right)}·c_{p_{tmp}}^{\left(i\right)}\hfill & \mathrm{if}u\mathrm{is}\mathrm{of}\mathrm{type}u{t}_c.\hfill \end{array}\right.$$

(10)

Given Equation (10), the utility of a template portfolio $TP$ and an operation schedule $o{s}_u$ is defined as

$$utility(TP,o{s}_u)=\sum _{p_{tmp}\in TP}utility(p_{tmp},o{s}_u)$$

(11)

The above Formula (11) enables an agent to identify an optimal pair of template portfolio $TP$ and operation schedule $o{s}_u$. Assuming that template portfolio $TP$ comprises n product templates, the corresponding optimization problem is given by

$$\begin{array}{cccc}\hfill & \mathrm{maximize}\hfill & \hfill & utility(TP,o{s}_u)\hfill \end{array}$$

(12)

$$\begin{array}{cccc}\hfill & \mathrm{subject}\mathrm{to}\hfill & \hfill & TP\in \mathcal{P}\left(P_{tmp}\right),\hfill \end{array}$$

(13)

$$\begin{array}{cccc}\hfill & \hfill & & o{s}_u\in OS{S}_u,\hfill \end{array}$$

(14)

$$\begin{array}{cccc}\hfill & \hfill & & \bigcup T_{pr}^{(p_{tmp},i)}\setminus T_{pl}=\varnothing \wedge \bigcap T_{pr}^{(p_{tmp},i)}=\varnothing ,1\le i\le n,\hfill \end{array}$$

(15)

$$\begin{array}{cccc}\hfill & \hfill & & sa{t}_{h{c}_u}\left(o{s}_u\right)=1\forall h{c}_u\in H{C}_u.\hfill \end{array}$$

(16)

As each agent in the context of the PPM seeks to maximize its utility, optimization occurs at the a-level. The first phase of DYCE results in the creation of a product portfolio $PP$, which comprises all target products $p_{tar}$ that the agent aims to trade within a given planning horizon $T_{pl}$ across one or more sub-markets $M_s$. The specification of $PP$ draws upon the template portfolio $TP$, which represents a selection of product templates $p_{tmp}$ from a catalog of templates $P_{tmp}$ that maximizes the agent’s expected utility.

To create a product portfolio $PP$, an agent performs six consecutive tasks. First, the agent establishes the template catalog $P_{tmp}$, which serves as a basis for making a utility-maximizing template portfolio $TP$. If an agent trades only standardized products, e.g., on an energy exchange, $TP$ is predefined. Subsequently, an agent creates price forecasts $c_{p_{tmp}}^{\left(i\right)}$ for all product templates $p_{tmp}$, which are used to determine the expected utility from $TP$ and $o{s}_U$. The next task of the agent is to determine the operation schedule space $OS{S}_u$. For this purpose, the agent forecasts the operational behavior of the unit u by identifying all feasible plans within the considered planning horizon. Depending on the type of plant, these forecasts must take into account controllable factors, such as planned events in a production process, or uncontrollable factors, such as future weather conditions. Before the agent determines $TP$ and $o{s}_u$ using a heuristic, it creates a global formation schedule that includes time intervals in which to form the coalitions for the target products. The template catalog $P_{tmp}$, the price predictions $C_{P_{tmp}}$, and the operation schedule space $OS{S}_u$ from the previous tasks then serve as input to the combinatorial optimization for portfolio enhancement (COPE) heuristic. The agent next runs COPE to obtain $PP$ and $o{s}_u$ to maximize its expected utility. Finally, the agent schedules the next of its target products according to the previously created formation schedule and starts the activity of neighborhood formation when the corresponding formation period begins.

4.3.2. Extended Product Portfolio $P{P}^{+}$

The main result of the PPM is a product portfolio $PP$, which for a planning horizon $T_{pl}$ comprises several target products $p_{tar}\in PP$ and an associated operation schedule $o{s}_u\in OS{S}_u$. However, the $PP$ comprises only one product at a time. For an agent to trade multiple products at a time, its capacity based on an $o{s}_u$ must be proportionally allocated to one or more products, resulting in an advanced product portfolio $P{P}^{+}$. Figure 5 shows an example representation of a $PP$ for only one product per time in a $T_{pl}$ with a total of 11 target products, including the associated $o{s}_u$, as well as an extended $P{P}^{+}$ that allows multiple target products $p_{tar}$ per time and a total of 15 target products. Both product portfolios $PP$ and $P{P}^{+}$ use the same operation schedule $o{s}_u$ and planning interval $T_{pl}$.

When trading multiple products simultaneously is allowed, the problem of finding an optimal $PP$ is no longer a combinatorial problem. Instead of selecting one product for a period, the resource must now be allocated to one or more products as optimally as possible. This type of problem can be described as a resource allocation problem, a particular case of nonlinear programming problems. Therefore, searching for an optimal $P{P}^{+}$ is no longer possible with the heuristic COPE and other methods must be used to find an optimal $P{P}^{+}$.

4.3.3. Neighborhood Formation for OCF

After an agent $a_i$ chooses a product portfolio $PP$ and operation schedule $o{s}_u$ that maximizes its expected utility, the agent performs the neighborhood formation to limit the set of potential interaction partners. This activity consists of four individual sub-tasks which are executed sequentially by the agent $a_i$. Initiating the procedure for a predetermined target product $p_{tar}$, the agent constructs an intrinsic model of the provided power grid G, encompassing all linked units $U_{a_i}$ and assigned agents $a_j$. Following this, principles from graph theory are employed to estimate distances between grid nodes, relying on the previously established grid model (in this context, the distance between two grid nodes $v_{G,i}$ and $v_{G,j}$ is represented by the impedance of the connecting power lines). The created internal grid model and the distances to the neighbors of $a_i$ are used in the next sub-task to calculate the corresponding abstract trust values by the trust function $trust(t,a_i,a_j)$ to evaluate the trustworthiness of potential cooperation partners $a_j$ of $a_i$ for a specific point in time t. Finally, the agent selects its initial role for the forthcoming CF negotiations. This involves deciding whether to proactively initiate (the $initiator\mathrm{role}$) or respond (¬initiator role) to formation requests. The agent determines the choice of role randomly based on the given probabilities $P\left(A\right)=r_{init}$ (probability of choosing the role initiator) and $P\left(\overline{A}\right)=1-r_{init}$ (probability of choosing the role ¬initiator). This leads to having a certain number of initiator agents $|A_{init}|$ since the ratio $r_{init}=\frac{|A_{init}|}{\left|A\right|}$ has implications for the global communication cost and runtime of the CF.

Since the Neighborhood Formation is performed separately for each target product $p_{tar}$, the sub-tasks apply to both classical product portfolios $PP$ as well as extended product portfolios $P{P}^{+}$ with overlapping products. However, trading multiple products in one period may also have different requirements for determining different types of neighborhoods. For example, for the formation of a coalition to deliver a reactive power product, the local proximity of the agents to each other plays an important role, whereas for the delivery of an active power product, the dispersed nature of the agents may be beneficial for reliable fulfillment of the product. In addition, there may be other requirements regarding the ratio $r_{init}$, as overlapping coalitions may lead to a larger number of coalitions, and, therefore, there is a larger communication overhead. It would, therefore, be conceivable, for example, that an agent who is in two coalitions and is the initiator in one coalition also takes on this role in the second coalition.

4.3.4. Coalition Formation for OCF

The activity of coalition formation is the third step of the DYCE method. It is structured as a decentralized procedure where agents initially form singleton coalitions and, through an interaction protocol and heuristic, progressively create optimized coalitions to achieve their target products. Throughout this process, agents aim to maximize the mean coalition value $\overline{v\left(\mathcal{C}\right)}$, consequently enhancing the overall value of the coalition structure $\mathcal{CS}$. Thus, the optimization problem that agents address during the formation process is:

$$\begin{array}{cc}\hfill \mathrm{maximize}& \overline{v\left(\mathcal{C}\right)}\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill \mathrm{subject} \mathrm{to}& \mathcal{C}\in \widetilde{CS},\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill & \mathcal{C},\mathcal{C}‘\in \mathcal{CS}\mathrm{with}\mathcal{C}\ne \mathcal{C}‘:\mathcal{C}\cap \mathcal{C}‘=\varnothing \wedge \bigcup _{\mathcal{C}\in \mathcal{CS}}\mathcal{C}=A.\hfill \end{array}$$

(19)

The problem can be solved optimally by finding a coalition structure $\mathcal{CS}$ in which all the coalitions involved achieve their desired product perfectly. To accomplish this, agents aggregate the technical capabilities of the units they control to fulfill an intended target product $p_{tar}$ and perform regroupings to improve the mean coalition values.

The contribution an agent a makes to a product p with product horizon $T_{pr}^{\left(p\right)}=\{t_{pr}^{\left(0\right)},\dots ,t_{pr}^{\left(j_{max}\right)}\}$ depends mainly on the operation schedule $o{s}_u$ generated in the previous phase and is given by the function $co{n}_{a,p}:T_{pr}^{\left(p\right)}\to {\mathbb{R}}^3$, whose values $co{n}_{a,p}\left(t_{pr}^{\left(i\right)}\right)=(e_u^{\left(i\right)},er{r}_u^{\left(i\right)},c_u^{\left(i\right)})$ specify the electricity amount $e_u^{\left(i\right)}$ produced or consumed by unit u with error $er{r}_u^{\left(i\right)}$ at cost $c_u^{\left(i\right)}$ in each product interval $t_{pr}^{\left(i\right)}\in T_{pr}^{\left(p\right)}$. Since DYCE only allows each agent to be a member in one coalition in each product interval, it follows that

$$\begin{array}{c}\hfill \forall t_{pr}^{\left(i\right)}\in T_{pr}^{\left(p\right)}\mathrm{with}{t}_{pr}^{\left(i\right)}=t_{pl}^{\left(i\right)}:co{n}_{a,p}\left(t_{pr}^{\left(i\right)}\right)=o{s}_u\left(t_{pl}^{\left(i\right)}\right)\end{array}$$

(20)

Aggregating the contributions of all participating agents of a coalition $\mathcal{C}$ results in the cumulative contribution $co{n}_{\mathcal{C},p}\left(t_{pr}^{\left(i\right)}\right)=(e_{\mathcal{C}}^{\left(i\right)},er{r}_{\mathcal{C}}^{\left(i\right)},c_{\mathcal{C}}^{\left(i\right)})$. To achieve meaningful outcomes, it is necessary to aggregate these attributes suitably. Given a coalition $\mathcal{C}=\{a_1,\dots ,a_n\}$ with associated units $U_{\mathcal{C}}=\{u_1,\dots ,u_n\}$, the cumulative electricity amount (Equation (21)), error (Equation (22)), and cost (Equation (23)) in a product interval $t_{pr}^{\left(i\right)}$ are calculated as

$$\begin{array}{cc}\hfill e_{\mathcal{C}}^{\left(i\right)}=& \sum _{u\in U_{\mathcal{C}}}{e}_u^{\left(i\right)}\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill er{r}_{\mathcal{C}}^{\left(i\right)}=& \frac{\sqrt{{\sum }_{u\in U_{\mathcal{C}}}(er{r}_u^{\left(i\right)}·|e_u^{\left(i\right)}{\left|\right)}^2}}{|e_{\mathcal{C}}^{\left(i\right)}|}\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill c_{\mathcal{C}}^{\left(i\right)}=& \frac{{\sum }_{u\in U_{\mathcal{C}}}{c}_u^{\left(i\right)}·e_u^{\left(i\right)}}{|e_{\mathcal{C}}^{\left(i\right)}|}\hfill \end{array}$$

(23)

During the process of forming coalitions, agents must evaluate the worth of their coalition to determine whether a regrouping enhances the overall value of the $\mathcal{CS}$, as defined by optimization problem (17). A regrouping $({\widehat{\mathcal{C}}}_{old},{\widehat{\mathcal{C}}}_{new})$ is generally considered as beneficial if

$$\begin{array}{c}\hfill \overline{v\left({\mathcal{C}}_{new}\right)}>\overline{v\left({\mathcal{C}}_{old}\right)},\end{array}$$

(24)

i.e., if the mean coalition value of a coalition resulting from regrouping $\overline{v\left({\mathcal{C}}_{new}\right)}$ has a higher value than the previous coalition $\overline{v\left({\mathcal{C}}_{old}\right)}$. The main objective of a coalition $\mathcal{C}$ is given by the trade of its intended product $p_{tar}$. Therefore, a reasonable way to specify the worth of $\mathcal{C}$ is to determine its ability to fulfill $p_{tar}$. Given the cumulative contribution $co{n}_{\mathcal{C},p_{tar}}$ for a target product $p_{tar}$ it is optimally fulfilled if

$$\begin{array}{c}\hfill \forall t_{pr}^{\left(i\right)}\in T_{pr}^{\left(p_{tar}\right)}:co{n}_{\mathcal{C},p_{tar}}\left(t_{pr}^{\left(i\right)}\right)=(e_{\mathcal{C}}^{\left(i\right)},er{r}_{\mathcal{C}}^{\left(i\right)},c_{\mathcal{C}}^{\left(i\right)})=(e_{p_{tar}}^{\left(i\right)},er{r}_{p_{tar}}^{\left(i\right)},c_{p_{tar}}^{\left(i\right)})=p_{tar}\left(t_{pr}^{\left(i\right)}\right)\end{array}$$

(25)

The approximation of a cumulative contribution to a target product is then defined in terms of the following attribute-based formulas:

$$\begin{array}{cc}\hfill e_{\mathcal{C},f}^{\left(i\right)}& =\varrho (1-\frac{e_{\mathcal{C}}^{\left(i\right)}}{e_{p_{tar}}^{\left(i\right)}}),\hfill \end{array}$$

(26)

$$\begin{array}{cc}\hfill er{r}_{\mathcal{C},f}^{\left(i\right)}& =\varrho (er{r}_{p_{tar}}^{\left(i\right)}-er{r}_{\mathcal{C}}^{\left(i\right)}),\hfill \end{array}$$

(27)

$$\begin{array}{cc}\hfill c_{\mathcal{C},f}^{\left(i\right)}& =\varrho (c_{p_{tar}}^{\left(i\right)}-c_{\mathcal{C}}^{\left(i\right)}),\hfill \end{array}$$

(28)

with

$$\begin{array}{c}\hfill \varrho :\mathbb{R}\to ]0,1],\varrho \left(x\right)=\frac{1}{1+|3·x|}.\end{array}$$

(29)

The approximation values from Equations (26) to (28) indicate the degree of fulfillment of the target electricity, target error, and target cost. Their assessment involves computing the real approximations of the target values. These results are then standardized by mapping them onto the interval $]0,1]$ through the application of the function $\varrho$ (Equation (29)). As a result, optimal values with no deviation ($x=0$) from the target values will have a value of 1, which indicates that the pursued target value is exactly met.

Given the above definitions, the worth of a coalition $\mathcal{C}$ can now be assessed by the value function

$$\begin{array}{c}\hfill v\left(\mathcal{C}\right)=\left\{\begin{array}{cc}{w}_e·\overline{e_{\mathcal{C},f}}+w_{err}·\overline{er{r}_{\mathcal{C},f}}+w_c·\overline{c_{\mathcal{C},f}}\hfill & \mathrm{if}\overline{e_{\mathcal{C},f}}\ne 0,\hfill \\ 0\hfill & \mathrm{else},\hfill \end{array}\right.\end{array}\sum _{i\in attr}{w}_i=1.$$

(30)

The variables $w_i$ denote weights employed to prioritize distinct attributes, while $\overline{e_{C,f}},\overline{er{r}_{C,f}},\overline{c_{C,f}}$ indicate the average degrees of fulfillment computed across all product intervals $t_{pr}^{\left(i\right)}\in T_{pr}^{\left(p_{tar}\right)}$ using Equations (26) to (28). Therefore, the formula (in Equation (30)) allows rating a coalition according to its ability to fulfill the average interval-related target values of the pursued product and enables agents to reasonably argue about the worth of different coalitions during the interactive formation process.

Given the ability to assess the value of a coalition, agents conduct the activity of CF to cooperate to fulfill their pursued target products optimally. The agents begin as singleton coalitions and then use an interaction protocol based on the well-established ContractNet protocol to evaluate possible regroupings. Through this iterative process, they optimize their coalitions in a fully decentralized and self-organized manner. Based on the role (initiator $a_I$ or responder $a_R$) chosen by an agent during the neighborhood formation, the agent initiates the formation of a coalition or waits for requests from other agents. When an agent $a_I$ has the initiator role, it first removes from its list neighboring agents who are already in its coalition, those who fall below a minimum trust value, or those who are in the set of unlikely cooperation partners $ucp$. Then, $a_I$ sends a call for proposal (CFP) to all remaining agents, with a payload that includes its coalition ${\mathcal{C}}_I$, the individual contributions of its members $CO{N}_{{\mathcal{C}}_I,p_{tar}}$, the target product $p_{tar}$, and a corresponding tolerance band $p_{tar,tol}$, as well as a timestamp $t_{d,CFP}$ as a deadline for responses to the CFP. A responder agent $a_R$ receiving such CFPs first checks whether the requesting agent $a_I$ is in its neighborhood, whether the target product $p_{tar}$ matches, for local power products whether agents in its coalition ${\mathcal{C}}_R$ match for that product, and whether agents from the requesting coalition ${\mathcal{C}}_I$ have sufficient trust values. If all preconditions are met, the responding agent $a_R$ identifies a potential regrouping using a heuristic called combined coalition regrouping (CCOR).

CCOR enables a member transfer between an initiating coalition $C_{I,old}$ and a set of responding coalitions ${\tilde{C}}_{R,old}$. This is achieved by iterating over ${\tilde{C}}_{R,old}$ and performing regrouping in each cycle if the value of the new coalition is greater than the old one. More precisely, the method essentially consists of two complementary steps. First, members are unilaterally transferred from responding to the initiating coalition to accumulate potential. Then, members are exchanged between the two coalitions to approximate the target values of the target product. The outputs of CCOR are two new optimized initiator and responder coalitions $({\widehat{C}}_{I,new},{\widehat{C}}_{R,new})$. It is generally true, that $0\le |{\widehat{C}}_{R,new}|\le |{\widehat{C}}_{I,new}|\le 1$ and ${\widehat{C}}_{R,new}=\varnothing \wedge {\widehat{C}}_{I,new}=\varnothing$ if no optimization was possible.

After an agent $a_R$ has checked the preconditions and executed CCOR, it responds to the CFP to $a_I$ with either a refusal or proposal. The refusal always includes the reason and is sent, for example, if the trust values of the requesting coalitions are insufficient. The proposal consists of the new optimized coalitions $({\widehat{C}}_{I,new},{\widehat{C}}_{R,new})$ as well as the optimization cycle parameter $oc$, which stores the cycles CCOR needed to find an optimal solution. When $a_I$ receives a refusal, it adds the agents from the coalition ${\mathcal{C}}_R$ to its list of $ucp$. After $a_I$ has processed all refusals, it checks the trust values of all new members it received from $a_R$. Subsequently, $a_R$ executes CCOR with the replying coalitions and obtains coalitions ${\widehat{C}}_{I,com}$ and ${\widehat{C}}_{R,com}$ as a result. After each cycle, a new initiator agent may need to be decided for each coalition, as the previous initiator agent may have been transferred to another coalition.

The interaction of the agents and the heuristic CCOR allows the agents to optimize the coalition structure $\mathcal{CS}$ in a decentralized and fully self-organized manner. An important constraint of CCOR is that the transfer of members from one coalition to another can never degrade the quality of the fulfillment of a product. In addition, various parameters, such as $oc$, are also used to prevent dead- and livelocks. Finding a $\mathcal{CS}$ ends when a certain number of cycles have been run or an optimal coalition structure ${\mathcal{CS}}^{\ast }$ has been found. The fulfillment of a coalition is considered successful if each unit in a coalition delivers the agreed contribution so that the target product can be fulfilled as planned (see Equation (25)).

Extended Coalition Formation

The CF in DYCE in the form of the interaction protocol and the heuristic is designed in such a way that the result can only be disjoint coalitions. This is also reflected in the constraints of the optimization problem, which ensures that all coalitions are disjoint (Equation (19)) and the assumption that the contribution of a unit equals its operation schedule (Equation (20)). However, since coalitions are formed individually for each target product, an extension of the current CF process is possible. Given an extended $P{P}^{+}$ that already allows agents to contribute to multiple products in a time interval, as illustrated in Formula (4), the expansion mainly concerns the heuristic CCOR in which the mean coalition values are maximized.

CCOR maximizes the mean coalition values by transferring agents between coalitions, leading to new coalitions, while never diminishing the value of a coalition by member transfers. However, due to overlapping contributions by an agent a, dependencies on other coalitions in which a is also a member can arise. These transitive dependencies must be taken into account by CCOR during optimization. At the same time, overlapping contributions also create other possibilities. For example, during the formation, an agent can explicitly use information during the optimization of $\mathcal{CS}$ from both coalitions in which it is located for the optimization.

4.3.5. Payoff Distribution for OCF

The payoff distribution is the last of the four phases in DYCE. Here, the added values of a coalition generated by a trade of a product are fairly distributed among the coalition members. If the trade of a product p with a product horizon $T_{pr}^{\left(p\right)}=\{t_{pr}^{\left(0\right)},\dots ,t_{pr}^{\left(j_{max}\right)}\}$ was successful, the resulting payoff $\breve{\rho }$ is defined as:

$$\begin{array}{c}\hfill \breve{\rho }=\sum _{i=0}^{j_{max}}{e}_p^{\left(i\right)}·{\breve{c}}^{\left(i\right)}\end{array}$$

(31)

where ${\breve{c}}^{\left(i\right)}$ are the actually contracted costs per unit of electrical energy in a product interval $t^{\left(i\right)}$. In reality, the achieved payoff $\breve{\rho }$ usually differs from the originally determined payoff $\rho$. The payoff $\rho$ determined by the agents in a coalition for a product p is defined as follows:

$$\begin{array}{c}\hfill \rho =\sum _{i=0}^{j_{max}}{e}_p^{\left(i\right)}·c_p^{\left(i\right)}\end{array}$$

(32)

The payoff ultimately establishes the utility attained by a coalition through the trading of a product p. This utility is defined as follows for a coalition of producer agents ${\mathcal{C}}_p$ and a coalition of consumer agents ${\mathcal{C}}_c$:

$$\begin{array}{c}\hfill utilit{y}_{\mathcal{C}}\left(p\right)=\sum _{i=0}^{j_{max}}utilit{y}_{\mathcal{C}}^{\left(i\right)}\left(p\right)=\left\{\begin{array}{cc}\sum _{i=0}^{j_{max}}{e}_p^{\left(i\right)}·{\breve{c}}^{\left(i\right)}-e_p^{\left(i\right)}·c_p^{\left(i\right)}=\breve{\rho }-\rho \hfill & \mathrm{if}\mathcal{C}\mathrm{is}{\mathcal{C}}_p,\hfill \\ \sum _{i=0}^{j_{max}}{e}_p^{\left(i\right)}·c_p^{\left(i\right)}-e_p^{\left(i\right)}·{\breve{c}}^{\left(i\right)}=\rho -\breve{\rho }\hfill & \mathrm{if}\mathcal{C}\mathrm{is}{\mathcal{C}}_c.\hfill \end{array}\right.\end{array}$$

(33)

The coalition’s utility is established as the sum of interval-associated utilities $utilit{y}_{\mathcal{C}}^{\left(i\right)}\left(p\right)$. For a producer coalition ${\mathcal{C}}_p$, the utilities are computed by subtracting the initial payoffs from the final payoffs $(\breve{\rho }-\rho )$. For a consumer coalition ${\mathcal{C}}_c$, the utilities are determined by subtracting the final payoffs from the initial payoffs $(\rho -\breve{\rho })$, as depicted in Equation (33).

All organizational and contractual tasks within a coalition $\mathcal{C}$ are performed by a representative agent $a_{\mathcal{C},rep}$, which is a member of $\mathcal{C}$. This includes, among other things, placing orders on the electricity markets for a trade of the target product. If the product trade was successful, $a_{\mathcal{C},rep}$ informs all coalition members to control their units accordingly. After the physical fulfillment of a product, $a_{\mathcal{C},rep}$ takes over the fair distribution of the payoff. For this purpose, $a_{\mathcal{C},rep}$ asks for the actual contributions of the members of $\mathcal{C}$, as they may differ from the agreed values due to unforeseen reasons. Based on the error an agent had in its contribution to a coalition, the agents calculate new trust values for that agent. If an agent has a low trust value, this may make future collaborations with other agents difficult. Agents, therefore, are interested in fulfilling their agreed contributions to a coalition, which positively impacts the reliability of coalitions. Subsequently, $a_{\mathcal{C},rep}$ determines the fair distribution of the payoff for all members of $\mathcal{C}$. For this purpose, $a_{\mathcal{C},rep}$ uses a distribution model based on cooperative game theory methods to determine the distribution vector ${\mathbf{x}}_{\mathcal{C}}=(x_1,\dots ,x_n)$, which indicates the individual shares of all members. Given a product horizon $T_{pr}^{\left(p\right)}=\{t_{pr}^{\left(0\right)},\dots ,t_{pr}^{\left(j_{max}\right)}\}$ of a successfully traded product p, the individual share $x_k$ of a coalition member $a_k$ is calculated as

$$\begin{array}{c}\hfill x_k=\sum _{i=0}^{j_{max}}{x}_k^{\left(i\right)}=\sum _{i=0}^{j_{max}}{c}_u^{\left(i\right)}·e_u^{\left(i\right)}+{\varphi }_k^{\left(i\right)}(\mathcal{C},v)·utilit{y}_{\mathcal{C}}^{\left(i\right)}\left(p\right),\end{array}$$

(34)

where ${\varphi }_k^{\left(i\right)}(\mathcal{C},v)$ is a fair percentage in terms of the Shapley value, and v is referred to as the characteristic function. Hence, the agent $a_k$’s total share is outlined as the sum of interval-associated shares $x_k^{\left(i\right)}$, wherein an interval-linked share constitutes the accumulation of the corresponding schedule cost $c_u^{\left(i\right)}·e_u^{\left(i\right)}$ alongside an equitable portion of the coalition’s utility ${\varphi }^{\left(i\right)}k(\mathcal{C},v)·utility{\mathcal{C}}^{\left(i\right)}\left(p\right)$. The equitable proportion ${\varphi }_k^{\left(i\right)}(\mathcal{C},v)$ is determined through the utilization of Shapley values, a concept drawn from the realm of cooperative game theory. Shapley values serve to create a just distribution vector in a given coalition game $\mathsf{\Gamma }$, where the notion of fairness is characterized by specific axioms that encapsulate attributes of the resultant distribution [47].

An extension or adaptation for overlapping coalitions is not necessary because the distribution of payoffs in DYCE happens purely on the C-level.

5. Conclusions and Outlook

5.1. Conclusions

The future energy system will be characterized by a variety of DERs because of its transformation driven by the energy transition. Due to their nature, these DERs place various requirements on the future power grid, such as a volatile feed-in due to weather-dependent power generation. For an economical and grid-serving use of DERs, they must be integrated into the power system. One type of integration that addresses various grid integration and economical requirements is the aggregation of DERs into pools. The formation of coalitions is a method of aggregation of DERs that enables dynamic product-based aggregation of DERs. Due to an increasing need for dynamics and a flexibilization of DERs in the future, further requirements for the aggregation of DERs arise. By forming overlapping coalitions, these requirements potentially can be addressed. Here, DERs can be members of more than one coalition at a time.

In this paper, we first investigated the related work in the context of the CF of DERs. In the analysis of related work, we found that a large body of work studies the aggregation of DERs for different purposes. The approaches to the formation of coalitions are primarily based on heuristics for finding an optimal CS or agent-based negotiations. There are approaches to both the centralized and decentralized formation of coalitions. However, very few works (none in the energy context) study the formation of overlapping coalitions.

Therefore, we presented the DYCE model and analyzed where DYCE would need to be extended to allow overlapping coalitions to be formed. For this purpose, we first described the formal model of DYCE and expanded it to include the formalisms for overlapping coalitions. These extended formalizations mainly concern the game-theoretic modeling of overlapping coalitions as an overlapping CF game and the coalition structure generation problem, as well as the extension of the contribution of a unit to coalitions. We then described the four phases of DYCE and, for each phase, examined the extensions needed to allow overlapping coalitions. The most extensions are necessary for the first phase, the product portfolio management. In this phase, an agent creates a product portfolio $PP$ for a planning horizon, comprising the products the agent wants to trade. An optimal $PP$ for an agent is searched using a heuristic. For an agent to be able to trade overlapping products, an extended product portfolio $P{P}^{+}$ is necessary, which, however, can no longer be determined with the previous heuristic. Further adjustments are required for the third phase, the actual CF. In this phase, agents decentrally form coalitions based on an interaction protocol to maximize the value of the CS. To achieve this, they can perform member transfers between coalitions. As part of the member transfer, the agents optimize the values of the coalitions using a heuristic. However, this optimization does not allow the formation of overlapping coalitions since, among other things, transitive dependencies between coalitions and the membership of an agent in several coalitions are not considered in the optimization.

Finally,

Table 2. Summary of all extensions required for overlapping coalition formation in DYCE, grouped by phase in DYCE.

Phase	Extension
1. Product Portfolio Management	Extended product portfolio $P{P}^{+}$ as part of the formal model to formalize the possibility of trading multiple products at a time. Method for solving the resource allocation problem, which considers several products at a time and provides a product portfolio as a result.
2. Neighborhood Formation	Maintaining different neighborhoods to consider different constraints for different types of electricity products. Investigation of different values for the ratio $r_{init}$, since the communication overhead for overlapping coalitions could be larger.
3. Coalition Formation	Extended agent contribution function $co{n}_a^{+}$ as part of the formal model to formalize the possibility of agent contributions to multiple coalitions at a time. Heuristic for finding an optimal CS with overlapping coalitions that takes into account the characteristics of OCF, such as transitive dependencies agents in multiple coalitions.
4. Payoff Distribution	No extensions necessary.

summarizes all of the previously described extensions required for OCF in DYCE, grouped by phase in DYCE.

5.2. Outlook

The formation of overlapping coalitions increases the overall complexity of the optimizations in the different phases. Therefore, the heuristics used in DYCE are no longer applicable. Future work should consequently investigate different approaches for the creation of a product portfolio with overlapping products as well as for the process of optimizing the CS in the context of agent interaction, while keeping the requirements such as the decentralized formation. For example, the investigation of methods from the field of multi-agent learning for the formation of overlapping coalitions would be conceivable.

The flexibility of the units is already rudimentarily taken into account in the DYCE formalizations but not in the formation of coalitions. However, with the increasing importance of flexibility in the future energy system, the need for modeling flexibility also increases. An extension of DYCE is, therefore, also conceivable with regard to the consideration of unit flexibility in the process of CF.

Since AS will also have to be provided by aggregates of DERs in the future, the reliability of product delivery plays an important role. Although there are already some approaches to assessing the reliability of coalitions (e.g., in [48]), these approaches only assess the reliability of an already existing CS. Therefore, DYCE could be extended to consider maximizing the reliability of a CS already during the formation process.

Within the CF process, the agents communicate based on the ContractNet protocol. ContractNet is a very general, application-unspecific protocol. However, due to the increasing complexity during the CF process, caused by overlapping coalitions, the use of an application specific protocol may be necessary, such as those presented in [49].

Overlapping coalitions offer additional advantages which could be analyzed in an extended version of DYCE. For example, an agent’s membership in two coalitions could be used to identify unneeded capacities from units in one coalition and make it available in the other coalition if it is needed there in the short term to improve coalition reliability.