New Clearing Model to Mitigate the Non-Convexity in European Day-ahead Electricity Market

Abstract

Nowadays, the payment scheme of European Day-Ahead Market is based on the market clearing price by running the Pan-European Hybrid Electricity Market Integration Algorithm. However, this conventional payment scheme is challenging because of the non-convexity and the short computation time requirement. Thus, the aim of this work is to propose a new clearing model in order to mitigate this challenge. The model is based on make-whole payment mechanism and it includes two major steps: (i) maximizing social welfare and (ii) achieving a Walrasian equilibrium by the “minimum-uplift approach”. The proposed model is validated and investigated by two case studies: one is an artificially created Day-Ahead Market session containing all type of bids encountered in Europe and containing a very large number of bids to stress the algorithm and the other is a reduced, but realistic, model of European market where real data from February to December of 2017 were considered. The tests show a consistent improvement of the numerical performances of the proposed model with respect to the conventional one while the economic performance is not altered, but is slightly improved. Moreover, because the tests are based on real data during a long period of time, the results show that proposed model is very promising for the real application.

1. Introduction

Nowadays, the European DAM coupling contains seven PXs: GME (Italy), EPEX Spot (France, Germany, and Austria, United Kingdom, Belgium, and Netherlands), Nordpool Spot (Scandinavian countries), OMIE (Spain and Portugal), OPCOM (Romania), TGE (Poland), and OTE (Czech Republic). In fact, this mechanism brings many benefits [1], such as periods of convergence of the electricity price and the efficiency in using the interconnectors between countries. To clear the market across Europe, PXs run an official model, namely EUPHEMIA [2]. Following the success of DAM, the European Intra-Day Market integration is in the process, with some proposed mathematical model presented in [3].

Nowadays, the European PXs are adopting a bidding areas approach to represent the electric grid in the market. Here, areas of the network where the trade can be performed without violating network constraints are linked through equivalent interconnectors that model possible network bottlenecks [4]. Moreover, the MCP (also called uniform price) is adopted as payment mechanism: this price characterize each bidding area and is imposed to all market participants in the area. However, the presence of interconnectors congestions will negatively affect the MCP; for this reason, the definition of the bidding areas is crucial [5]. Overall, this mechanism absolutely ensures that the total payment of demand and supply is balanced.

1.1. Non-Convexities in the European Day-Ahead Electricity Market Model

1.1.1. Conventional Non-Convexity Issues

To find the solution of the European electricity market, a non-convex optimization model (EUPHEMIA) is solved [2] due to the presence of non-convex bids. Indeed, today, the high penetration of renewable energy sources leads to thermal units stop and run frequently. Therefore, non-convex bids play an important role in modelling the new way the traditional thermal units are dispatched.

In Europe, two particular types of order representing this purpose are block orders implemented in EPEX Spot, Nordpool Spot, OPCOM, TGE, and OTE, and MIC orders adopted in OMIE. The block order is accepted if the VWAP (1) is greater (lower) than the submitted price of supply (demand), respectively. Meanwhile, the MIC (2) ensures the selling agent recovers all specified costs (minimum income with Fixed Term and Variable Term), disregarding that the specific schedule obtained for each hour can be variable:

$$\begin{array}{cc}VWA{P}_b=\frac{{\displaystyle \sum _{t\in T}}{\displaystyle \sum _{pb\in b}}{x}_b·Q_b·(\underline{U_{pb}^t}-\overline{U_{pb}^t})·{\lambda }_a^t}{\sum _{t\in T}\sum _{pb\in b}{x}_b·Q_b·(\underline{U_{pb}^t}-\overline{U_{pb}^t})}\hfill & \hfill \hspace{1em}\forall b\in B_a,a\in A\end{array}$$

(1)

$$\begin{array}{cc}\sum _{a\in A}\sum _{t\in T}[{\lambda }_a^t·(\sum _{ss\in p}|Q_{ss}^t·x_{ss}^t\left|\right)]\ge F{T}_p+V{T}_p·(\sum _{t\in T}\sum _{ss\in p}|Q_{ss}^t·x_{ss}^t\left|\right)\hfill & \hfill \hspace{1em}\forall p\in P_a^{mic}\end{array}$$

(2)

where $VWA{P}_b$ is the VWAP of block order b, and ${\lambda }_a^t$ is the MCP of area a in period t.

Obviously, these conditions can be guaranteed only if MCP is known. Normally in EUPHEMIA, the “fill-or-kill” condition of block orders is formulated using binary variables. The DAM is solved by maximizing the social welfare in one day. Subsequently, the MCP is obtained from the dual variable of the power balancing equations. Generally, this method will provide an MCP which does not support the market rules, e.g., the VWAP condition of block orders due to “fill-or-kill”constraint condition, or the MIC. This results in paradoxical orders present in the market solution: the non-convexity problem has appeared. In particular, it is possible to find a market solution with PAMICs, PABs, PRMICs, and PRBs [2]. The PAMICs and PABs lead to the loss-incurred of bids, in which the acceptance of these bids gives a negative impact on the total SW. Meanwhile, the PRMICs and PRBs are cause of a sub-optimal solution, thus the final market result is not the optimal one in terms of maximizing SW.

1.1.2. National Uniform Price in Italy

Because of the long shape of Italy, its transmission system is characterized by several bottlenecks along the peninsula and, therefore, the market solution is characterized by different zonal prices. For this reason, when the market was liberalized in 1999, the PUN was introduced in the market rules. This means, that the same price is used for buyers, independent of their location. In particular, the PUN is computed as a weighted average of MCPs (also a decision variable) with respect to the accepted volume of all geographical Italian areas for each period t [1]:

$$\begin{array}{cc}PU{N}^t={\displaystyle \frac{{\displaystyle \sum _{a\in A}}{\displaystyle \sum _{ss\in S{S}_a^{pun}}}{\lambda }_a^t·Q_{ss}^t·x_{ss}^t}{{\displaystyle \sum _{a\in A}}{\displaystyle \sum _{ss\in S{S}_a^{pun}}}{Q}_{ss}^t·x_{ss}^t}}\hfill & \hfill \hspace{1em}\forall t\in T\end{array}$$

(3)

where $PU{N}^t$ is the PUN in period t. Here, the subset $S{S}_a^{pun}$ only contains the demand stepwise orders $ss$ belonging to the geographical areas of Italy, so the orders in this subset are called PUN orders and represent another element of non-convexity of the market model since the total payment of the buyers needs to assure the payment of the sellers at MCP of the bidding zones they belong to.

1.1.3. Current Solution

The single European DAM has to look for a solution to deal with these non-convexity issues since the number of non-convex bids is increasing, e.g., from 2011–2015, the number of block and MIC orders increased 2 times, while the number of PUN orders increased 81% [6]. Today, the European PXs are using the EUPHEMIA [2] model to solve the market. EUPHEMIA is solving an optimization problem, i.e., the Social Welfare Maximization Problem (SWMP), in order to calculate the MCPs; then, the block orders, MIC and PUN market rules are checked and if they are not satisfied, i.e., if a non-convexity is detected, the SWMP is solved iteratively according to a branch-and-bound strategy where the constraints of SWMP are locally adjusted to satisfy the violated market rules. In fact, the description of EUPHEMIA in [2] only gives the main characteristics of EUPHEMIA, but not the detailed mathematical model. However, paper [7] presents a full European DAM model based on the specifications of EUPHEMIA document, which gives results that are very close to the officially reported ones. It includes the main problem, i.e., the SWMP and five SPs, connected as in Figure 1, that iteratively change the constraints of SWMP to mitigate the non-convexities. The SWMP is formulated as MIQP, because it requires the binary variables to model the acceptance rule of the “fill-or-kill” condition and the quadratic function to model the piecewise orders. Besides, the four SPs, which mitigate the non-convexity issues of MIC and block orders are: (i) PAMIC-SP, (ii) PAB-SP), (iii) PRMIC-SP, and (iv) PRB-SP. In which, PAMIC-SP and PAB-SP aim to clear the PAMICs and PABs, meanwhile PRMIC-SP and PRB-SP are used to save PRMICs and PRBs, thus these SPs try to enhance the fairness of the final market solution. The last SP, which is the PUN-SP, looks for the PUN in Italy and consists of two main steps: (i) vertical search and (ii) horizontal search (based on the parametric optimization of [8]) along the step-wise curve of the demand offers.

Although paper [7] proposed many improvements to reduce the computation time, such as sequentially updating the set of PRBs or optimized PUN initialization, the problem still takes too much time to converge due to the complex iterative process it is using. Whereas, the threshold of computation time is within 10 min. [6] since it starts running, thus it potentially provides a sub-optimal solution. Thus, the main goal of this paper is to mitigate the issue described in this part while not altering the market performance with a negative impact on its players.

1.2. Literature Review

Nowadays, the number of non-convex bids in Europe is increasing, leading to economic challenges [9] regarding the increase in the number of paradoxically accepted and rejected bids. That is why there are many proposals to handle them, but all the works presented so far only limit to the block orders and PUN. Paper [10] introduces “adjustable profile blocks”, in which the indivisibility of block orders condition is relaxed. In particular, an iterative heuristic procedure is proposed for handling them. Paper [11] proposes a model which is formulated as a Mixed Integer Problem (MIP) that does not use the complementarity constraints to express market rules. Moreover, binary variables for “fill-or-kill” condition are not adopted. The other market rules are partially considered in [12] and fully implemented in [7] using the iterative process to mitigate the non-convex issues of the European DAM. Paper [13] avoids the impact of LGC on non-convex orders by introducing shadow prices between two consecutive periods of LGC orders, for each LGC order. Here, paper [13] only considers complex orders (MIC, LGC, and SSC orders). Paper [14] proposes specific constraints that ensure producers achieve positive profits by minimizing the duality gap of primary and dual Linear Problems to guarantee a competitive partial equilibrium [15]. However, this adoption will increase the market price to ensure positive profit of all producers, thus it may not be fair for the demand. A bi-level approach used to mitigate the non-convexity is proposed in [16] for PUN in Italy and then it is expanded to block orders in [17]. In paper [18], to mitigate the non-convexity issues of PUN order in Italy, a heuristic approach [7] and a complementary theory approach [19] are compared in terms of MCP, computation time and accepted volume. The conclusion of [18] is the heuristic model of [7] is more efficient, as it converges much faster and gives the same outcome. Therefore, in general, the approaches presented here are limited to solving in a rather limiting manner specific non-convexity issues. Except for approach [7], they are difficult or impossible to adapt to an integrated model where all non-convexity issues (and, therefore, all market rules) are considered. While [7], as shown in the previous subsection, does not mitigate the computation time issue.

On the other hand, paper [15] proposes the use of non-uniform price (make-whole payment) to mitigate the loss-incurred of non-convexity bids. Here, the non-convexity is due to the unit commitment of the generating unit that is represented through binary variables and the related constraints: ramp constraints on generator production, minimum continuous operating time, etc. Basically, this pricing scheme involves the MCP (€/MWh) and the “uplift” payment (€/MWh), and consists of two main steps: (i) the MCP is found by solving the maximum social welfare problem and (ii) this price requires an “uplift” payment to fulfil the non-convex acceptance conditions. This means that the uplift contribution of accepted bids that are in-the-money (A supply (demand) order is in-the-money if its price is smaller (greater) than the market price [2].) compensates the loss-incurred of paradoxically accepted bids. Moreover, papers [9,20] proved that this pricing scheme can support the competitive partial equilibrium and the competitive Walrasian equilibrium [21] always exists, in which each participant is perfectly satisfied. Obviously, the special treatment of non-convex bids can be negative because the generation unit will give rather than receive the “uplift” payments. Thus, a “minimum-uplift approach” allows generator units not only keep their profit, but also contribute to reducing the loss-incurred in a profitable way [15]. Moreover, paper [22] proposes a new approach to minimize the uplift by considering both the “uplift” payments and MCP as variables in one problem. Besides, this approach is being considered for using in the Europe DAM [23] (see Appendix A). Particularly, paper [21] shows the advantages and drawbacks of this approach in Europe. Here, the major drawback is the “truth-telling”, in which sellers can make money easily by submitting a large volume at a low price with the non-convex requirement such as MIC orders in OMIE. Additionally, in market solution, PRBs can hold a small volume and be deep-in-money; the so called the welfare sub-optimality issue. Because these issues concern the market regulator, it is necessary to provide market rules to impose the players to submit reasonable bids. In conclusion, the papers reviewed in the last paragraph show that the “uplift” approach is a promising method to solve the non-convexities in the European DAM. However, these papers does not consider all the market rules in the European DAM, as they limit to study much simpler cases, like generator unit commitment in [15], or only specific rules separately, like MIC or simple block orders in [21]. For this reason, in this work the authors propose a complete “uplift” approach that includes all European DAM rules and mitigates all disadvantages shown here.

1.3. Approach and Contributions

To derive the advanced European DAM model that provides a fair market solution in the minimal computation time and ensures no missing money as well, the proposed paper proceeds, as follows:

• defines market rules for the input data to overcome the drawback of the make-whole payment mechanism in Europe, e.g., the “truth-telling” and the welfare sub-optimality;
• reduces the number of iterations of the algorithm reported in paper [7] (see Figure 1) by solving only SWMP and PUN-SP to obtain the pure market solution. The PAMIC-SP, PAB-SP, PRMIC-SP, and PRB-SP and the parametric optimization of the horizontal search of the PUN-SP are ignored;
• achieves the Walrasian equilibrium by “uplift” payment, and minimizes “uplift” payments based on the “minimum-uplift approach” [15] and MCP as variable [22] in one problem; and,
• keeps shadow prices of the pure market solution to avoid the missing money since MCP is variable in the “minimum uplift approach” payments.

The main contributions of the paper are summarized below:

• The new European DAM model: the paper proposes an advanced model which contains all the market rules in Europe and provides the fair market solution in the minimum computation time;
• Improve the PUN-SP: because PUN is used for payment in for all PUN orders, it is impossible to introduce the “uplift” for each sub-order. Thus, the PUN-SP is improved by replacing the parametric optimization of the horizontal search with the cooperation between the “uplift” payments and final electricity prices to overcome the drawback of this optimization method that is the large computation time;
• Compute CR: provides a general formula to compute CR of interconnectors in Europe based on the dual variables of the network constraints where the losses of DC interconnectors and the meshed network are explicitly considered; and,
• Economic evaluation: the new payment scheme is carefully investigated and compared to the conventional payment method of [7] using the Western European case study, which is realistic case study in term of type and size of data. The analysis is performed considering various economic aspects to show that the proposal is acceptable at European level.

2. Mathematic Model

The proposed model presented in Figure 2 includes two main problems and one SP that are SWMP, NUPP, and PUN-SP, respectively. Firstly, the input data is verified, aiming to ensure that all submitted bids are reasonable. Then, SWMP and PUN-SP are executed in order to find the pure market solution. Finally, the NUPP implemented using the “minimum-uplift approach” [15,22] has the goal to achieve the Walrasian equilibrium. It should be noted that the NUPP just modifies the payment of bids, but it does not change the accepted volume that resulted from the pure market solution.

2.1. Verify Data

2.1.1. The Welfare Sub-Optimality

Today, in Europe, the size and number of block orders are limited, e.g., 40 blocks for France and 100 blocks for Germany per portfolio and maximum size is 600 MW [24]. Indeed, this limitation is imposed precisely to avoid the large loss of welfare. Moreover, this limitation is also due to the small threshold of computation time of EUPHEMIA [6].

2.1.2. The Truth-Telling

In OMIE, MIC is used to guarantee the minimum income of the generating unit. Here, the unit will submit two parts: (i) bids for energy sale (volume and price) and (ii) required minimum income terms (Fixed Term (€) and Variable Term (€/MWh)). As discussed in Section 1.1.1, the loss-incurred of MIC (2) happens for the paradoxically accepted bids when the MIC of the unit is violated and is recovered by the side-payment (€). The income and side-payment of unit subscribing MIC are computed by Equations (4a) and (4b), respectively. It should be noted that Equations (4a) and (4b) are derived from (2):

$$\begin{array}{cc}{\pi }_p=\sum _{a\in A}\sum _{t\in T}\sum _{ss\in p}({\lambda }_a^t-V{T}_p)·|Q_{ss}^t·x_{ss}^t|-F{T}_p+{\sigma }_p\hfill & \hfill \forall p\in P_a^{mic}\end{array}$$

(4a)

$$\begin{array}{cc}{\sigma }_p=max\{0,\sum _{a\in A}\sum _{t\in T}\sum _{ss\in p}F{T}_p+(V{T}_p-{\lambda }_a^t)·|Q_{ss}^t·x_{ss}^t\left|\right\}\hfill & \hfill \forall p\in P_a^{mic}\end{array}$$

(4b)

where ${\pi }_p$ is the income of unit p in €, ${\sigma }_p$ is the side-payment to recover the loss-incurred in €.

If MIC (2) is violated, the side-payment will be non-null in (4b). Hence, the income ${\pi }_p$ in (4a) also increases. Consequently, the production unit can submit either a large Fixed Term or Variable Term to gain ${\sigma }_p$, and increase its income. Obviously, this is an unexpected situation in a transparent market.

Today, OMIE defines a market rule [25] in order to prevent MIC orders manipulate the market. Here, MIC must not be used in production unit that has more than 50% of bids submitted at zero price. However, this rule does not solve the problem, but it only tries to reduce its impact. To solve the problem, it is necessary to define an extra condition (5) that involves only the submitted parameters to prevent the suppliers submit an unreasonable Fixed Term and Variable Term. This does not only enhances the “truth-telling” of MIC orders, but also it does not conflict with the minimum income of producer. In particular, the condition (5) is introduced:

$$\begin{array}{c}[{\displaystyle \sum _{t\in T}}{\displaystyle \sum _{ss\in p}}(P_{ss}^t-V{T}_p)·|Q_{ss}^t|]-F{T}_p\ge 0,\hspace{1em}\forall p\in P_a^{mic}\hfill \end{array}$$

(5)

Condition (5) imposes that MIC is satisfied if the production unit is accepted at maximum quantity and paid at bid price. This condition should cover the majority of scenarios, since, in most of the cases, the supply bids are either completely accepted—where rule (5) guarantees the MIC for the bid prices, while, in general, market prices are higher; or, completely rejected—where MIC is not needed. In some rare cases the production unit could be partially accepted at bid price (marginal production unit) case when condition (5) does not guarantee the MIC, but the convexity of the Minimum Income function and bid prices together with (5) deeply limits the violation of MIC (and, hence, of the side-payment).

2.2. Social Welfare Maximize Problem (SWMP)

2.2.1. Objective Function

The objective function (6) maximizes the social welfare. It is defined as the total cost of supply minus the total offer of demand because the minimizing function is used. Moreover, there is a penalty term in (6) that is used for two purposes [7]: (i) correctly decompose the flow into positive ($f_l^{t,+}$) and negative ($f_l^{t,-}$), and (ii) optimize the flow to reduce the loss of welfare due to the transmission tariff of DC interconnectors. Here, the cost functions of orders are clearly presented in Appendix B for the hourly stepwise orders (A1), hourly piecewise orders (A2), block orders (A3), and flexible hourly orders (A4). As mentioned in the nomenclature, the demand and supply orders are distinguished by their quantity sign; thus, the costs of demand side are actually positive, while the costs of supplier side are negative and, thus, the sum of the two provides the total social welfare.

$$\begin{array}{c}\hfill \mathrm{Min}[-\sum _{a\in A}[\sum _{b\in B_a}{c}_b+\sum _{t\in T}(\sum _{ss\in S{S}_a}{c}_{ss}^t+\sum _{sp\in S{P}_a}{c}_{sp}^t+\sum _{fh\in F{H}_a}{c}_{fh}^t)]+\sum _{l\in B{r}_{fa.ta}}\sum _{t\in T}{c}_l·(f_l^{t,+}-f_l^{t,-})\end{array}$$

(6)

2.2.2. Market Order Constraints

Constraints (7)–(12) model the market rules of orders used in the European electricity market as paper [7]. Here, the market rules of block orders are modeled by constraints (7)–(10) and include the profile (7), flexible hourly (8), exclusive (9), and link (10) block orders. Notably, the parameter $R_b^{min}$ models the “fill-or-kill” condition if it equals to 1, otherwise it is in the range [0,1). The market rules of complex orders are introduced in (11)–(13) that includes the Load Gradient Condition (11) and (12) and Schedule Stop Condition (13). Inequality constraints (11) and (12) guarantee the accepted volume can not vary too much between two consecutive periods.

$$\begin{array}{cc}{Y}_b·R_b^{min}\le x_b\le Y_b\hfill & \hfill \hspace{1em}\forall b\in B_a\end{array}$$

(7)

$$\begin{array}{cc}{x}_{fh}^t-Y_{fh}^t=0\hfill & \hfill \hspace{1em}\forall fh\in F{H}_a,t\in T\end{array}$$

(8)

$$\begin{array}{cc}\sum _{b\in eg}{x}_b\le 1\hfill & \hfill \forall eg\in E{G}_a\end{array}$$

(9)

$$\begin{array}{cc}{x}_{lb}-\sum _{b\in B_a}I{M}_b^{lb}·x_b\le 0\hfill & \hfill \forall lb\in L{B}_a\end{array}$$

(10)

$$\begin{array}{cc}-D{G}_p^t-\sum _{ss\in p}(x_{ss}^t·Q_{ss}^t-x_{ss}^{t-1}·Q_{ss}^{t-1})\le 0\hfill & \hfill \forall p\in P_a^{lgc},t\in T\end{array}$$

(11)

$$\begin{array}{cc}\sum _{ss\in p}(x_{ss}^t·Q_{ss}^t-x_{ss}^{t-1}·Q_{ss}^{t-1})-I{G}_p^t\le 0\hfill & \hfill \forall p\in P_a^{lgc},t\in T\end{array}$$

(12)

Meanwhile, the inequality constraints (13) and (14) are different from [7], because of the simplification of SPs performed on EUPHEMIA model. The inequality constrain (13) imposes the accepted ratio of the first hourly sub-order of producer subject to SSC always equals 1 for the first three periods because the lower bound is always 1. Meanwhile, paper [7] updates the upper bound of sub-orders after each iteration of the PAMIC-SP, SP no longer present in the proposed model. The last constraint (14) optimizes the accepted volume of the PUN orders belonging to the set $PU{N}_{eq}^t$, which contains all PUN orders have the submitted price equal to the PUN. Not likewise, paper [7] where the parameter $T\_PU{N}^t$ is in the range [0,1], here $T\_PU{N}^t$ only receives 0 or 1 due to the elimination of the horizontal search from PUN-SP, where it was updated iteratively. It should be noted that the initial set $PU{N}_{eq}^t$ is empty, and it is updated in the PUN-SP.

$$\begin{array}{cc}{B}_{ss}^{t,ssc}-x_{ss}^t\le 0\hfill & \hfill \hspace{1em}\hspace{1em}\hspace{1em}\forall t\in T,ss\in p,p\in P_a^{ssc}\end{array}$$

(13)

$$\begin{array}{cc}{x}_{ss}^t-T\_PU{N}^t\le 0\hfill & \hfill \hspace{1em}\hspace{1em}\hspace{1em}\forall t\in T,ss\in PU{N}_{eq}^t\end{array}$$

(14)

2.2.3. Power Balancing Equations

The power balancing conditions are modelled with the constraints (15)–(17) [7]. The symbols between square brackets at the end of equality constraints (15) are their dual variables.

$$\begin{array}{cc}{\displaystyle \sum _{b\in B_a}}{\displaystyle \sum _{pb\in b}}{x}_b·Q_{pb}·({\displaystyle \underline{U_{pb}^t}}-\overline{U_{pb}^t})+{\displaystyle \sum _{ss\in S{S}_a}}{Q}_{ss}^t·x_{ss}^t+{\displaystyle \sum _{sp\in S{P}_a}}{Q}_{sp}^t·x_{sp}^t\hfill & \\ +\sum _{fh\in F{H}_a}{x}_{fh}^t·Q_{fh}^t+P_a^t=0\left[{\lambda }_a^t\right]\hfill & \forall a\in A,t\in T\hfill \end{array}$$

(15)

$$\begin{array}{cc}\sum _{l\in B{r}_{fa.a}^{ATC+FB\_AC}}{f}_l^t+\sum _{l\in B{r}_{fa.a}^{ATC+FB\_DC}}(f_{fa.a}^{t,+}·(1-los{s}_l)+f_{fa.a}^{t,-})-\sum _{l\in B{r}_{a.ta}^{ATC+FB\_AC}}{f}_l^t\hfill & \\ -\sum _{l\in B{r}_{a.ta}^{ATC+FB\_DC}}(f_{a,ta}^{t,-}·(1-los{s}_l)+f_{a.ta}^{t,+})+P_a^{t,tem}=0\hfill & \forall a\in A,t\in T\hfill \end{array}$$

(16)

$$\begin{array}{cc}\sum _{l\in B{r}_{fa.a}^{ATC.FB\_AC}}{f}_l^t+\sum _{l\in B{r}_{fa.a}^{ATC.FB\_DC}}(f_{fa.a}^{t,+}·(1-los{s}_l)+f_{fa.a}^{t,-})-\sum _{l\in B{r}_{a.ta}^{ATC.FB\_AC}}{f}_l^t\hfill & \\ -\sum _{l\in B{r}_{a.ta}^{ATC.FB\_DC}}(f_{a,ta}^{t,-}·(1-los{s}_l)+f_{a.ta}^{t,+})+P_a^t-P_a^{t,tem}=0\hfill & \forall a\in A,t\in T\hfill \end{array}$$

(17)

The equality constraints (15)–(17) compute the power flow and ensure that the power is balanced for each area a at each period t. Here, the MCP is obtained from the dual variable of (15), denoted as ${\lambda }_a^t$. These constraints are known as the hybrid model where the ATC and the FB approaches are modeled. The exchange power between ATC and FB model is presented in (17) for $l\in B{r}_l^{ATC.FB}$, while Equation (16) computes the exchanged energy inside ATC and FB models, respectively, for $l\in B{r}_l^{ATC+FB}=B{r}_l\backslash B{r}_l^{ATC.FB}$.

2.2.4. Network Constraints

The network constraints are modelled with the equations (18)–(35) [7]. The symbols between square brackets at the end of inequality constraints (18)–(35) are their dual variables.

Power flow constraints

$$\begin{array}{ccc}{f}_l^{t,+}-B{r}_{l,max}^t\le 0\hfill & \hfill \left[{\mu }_{l,max}^{t,+}\right]& \forall l\in B{r}_l,t\in T\hfill \end{array}$$

(18)

$$\begin{array}{ccc}0-f_l^{t,+}\le 0\hfill & \hfill \left[{\mu }_{l,min}^{t,+}\right]& \forall l\in B{r}_l,t\in T\hfill \end{array}$$

(19)

$$\begin{array}{ccc}B{r}_{l,min}^t-f_l^{t,-}\le 0\hfill & \hfill \left[{\mu }_{l,min}^{t,-}\right]& \forall l\in B{r}_l,t\in T\hfill \end{array}$$

(20)

$$\begin{array}{ccc}0-f_l^{t,-}\le 0\hfill & \hfill \left[{\mu }_{l,max}^{t,-}\right]& \forall l\in B{r}_l,t\in T\hfill \end{array}$$

(21)

$$\begin{array}{ccc}{f}_l^t=f_l^{t,-}+f_l^{t,+}\hfill & \hfill \left[{\mu }_l^t\right]& \forall l\in B{r}_l,t\in T\hfill \end{array}$$

(22)

$$\begin{array}{ccc}B{r}_{l,min}^t-f_l^t\le 0\hfill & \hfill \left[{\beta }_{l,min}^t\right]& \forall l\in B{r}_l^{ATC},t\in T\hfill \end{array}$$

(23)

$$\begin{array}{ccc}{f}_l^t-B{r}_{l,max}^t\le 0\hfill & \hfill \left[{\beta }_{l,max}^t\right]& \forall l\in B{r}_l^{ATC},t\in T\hfill \end{array}$$

(24)

$$\begin{array}{ccc}\sum _{a\in A^{FB}}PTD{F}_a^{t,cb}·P_a^{t,tem}-RA{M}_{cb}^t\le 0\hfill & \hfill \left[{\beta }_{max}^{t,cb}\right]& \forall cb\in (CB\backslash CBA),t\in T\hfill \end{array}$$

(25)

$$\begin{array}{ccc}-RA{M}_{cb}^t-\sum _{a\in A^{FB}}PTD{F}_a^{t,cb}·P_a^{t,tem}\le 0\hfill & \hfill \left[{\beta }_{min}^{t,cb}\right]& \forall cb\in (CB\backslash CBA),t\in T\hfill \end{array}$$

(26)

$$\begin{array}{cc}\sum _{l\in B{r}_l^{FB}}[(max(PTD{F}_{fa}^{t,cb}-PTD{F}_{ta}^{t,cb},0)·f_l^{t,+})-\hfill & \hfill (min(PTD{F}_{fa}^{t,cb}-PTD{F}_{ta}^{t,cb},0)·f_l^{t,-})]\le RA{M}_{cb}^t\\ \left[{\beta }_{cb}^t\right]\forall cb\in CBA,t\in T\hfill \end{array}$$

(27)

The constraints (18)–(22) in combination with the coefficient $c_l^t$ in (6) are used in order to correctly decompose the power flow $f_l^t$ into two components: $f_l^{t,+}$ and $f_l^{t,-}$ [12]. The constraints of ATC model are described in (23) and (24), while constraints (25) and (26) represent the FB model. As detailed in [2,7], the ATC model considers an equivalent interconnector between two areas, when these areas share a border crossed by at least one physical link. The power on this interconnector will not flow according to the Kirchhoff’s laws, but following the commercial path from the low price area to the high price area. This power is limited by (23) and (24). As detailed in [2,7], the FB model is more advanced as it approximates power flows on actual AC physical links as linear functions of the net exports of the bidding areas. For this, the areas PTDFs are used as the constant coefficient of the linear constraints. The flow on a limited set of critical branches is limited in (25) and (26) in order to contain the size of the problem. The equivalent links of the ATC model are used also in the flow-based model to calculate the equivalent commercial power flow between bidding areas.

Inequality constraint (27) is used to clear the non-intuitive commercial power flows that can appear between FB model bidding zones due to the FB constraints [2,7] by updating the set $CBA$ in the intuitive patch. The intuitive patch runs parallel with SWMP and it aims to clear the non-intuitive commercial power flow that goes from high price area to low price area; hence, commercially non-intuitive, due to the formulation of the FB model [26]. First, the intuitive patch checks the welfare of each branch to find the non-intuitive flow (the welfare is negative); then, if non-intuitive flows are realized, the patch eliminates all elements $cb\in CB$ that reach the upper bound of inequality constraints (25) or (26), then adds them to $CBA$ and re-run SWMP; otherwise, the patch stops. It means that inequality constraint (27) replaces inequality constraints (25) and (26) for all $cb\in CBA$.

Ramping up and down constraints

$$\begin{array}{ccc}{f}_l^t-f_l^{t-1}-R_l^{t,U}\le 0\hfill & \hfill \left[{\gamma }_l^{t,U}\right]& \forall l\in B{r}_l^R,t\in T\hfill \end{array}$$

(28)

$$\begin{array}{ccc}{f}_l^{t-1}-f_l^t+R_l^{t,D}\le 0\hfill & \hfill \left[{\gamma }_l^{t,D}\right]& \forall l\in B{r}_l^R,t\in T\hfill \end{array}$$

(29)

$$\begin{array}{ccc}\sum _{l\in ls}[f_l^t-f_l^{t-1}]-R_{ls}^{t,U}\le 0\hfill & \hfill \left[{\gamma }_{ls}^{t,U}\right]& \forall ls\in LS,t\in T\hfill \end{array}$$

(30)

$$\begin{array}{ccc}\sum _{l\in ls}[f_l^{t-1}-f_l^t]+R_{ls}^{t,D}\le 0\hfill & \hfill \left[{\gamma }_l^{t,D}\right]& \forall ls\in LS,t\in T\hfill \end{array}$$

(31)

$$\begin{array}{ccc}{P}_a^t-P_a^{t-1}-R{U}_a^t\le 0\hfill & \hfill \left[{\gamma }_a^{t,U}\right]& \forall a\in A,t\in T\hfill \end{array}$$

(32)

$$\begin{array}{ccc}{P}_a^{t-1}-P_a^t+R{D}_a^t\le 0\hfill & \hfill \left[{\gamma }_a^{t,D}\right]& \forall a\in A,t\in T\hfill \end{array}$$

(33)

$$\begin{array}{ccc}{R}_a^D-\sum _{t\in T}{P}_a^t-In{p}_a\le 0\hfill & \hfill \left[{\gamma }_a^D\right]& \forall a\in A\hfill \end{array}$$

(34)

$$\begin{array}{ccc}\sum _{t\in T}{P}_a^t-In{p}_a-R_a^U\le 0\hfill & \hfill \left[{\gamma }_a^U\right]& \forall a\in A\hfill \end{array}$$

(35)

Finally, inequality constraints (28)–(35) limit the hourly ramping of lines (28) and (29), line sets (30) and (31), and hourly and daily ramping net position (32)–(35). The line set is a group of lines that represent a critical section of the grid for which the total power flow is subject to constraints. These inequality constraints reduce the threats to the security of supply due to the frequent large changes in production and power flows in the grid.

2.3. PUN Search Sub-Problem (PUN-SP)

As presented above, the PUN orders are paid at the same price, no matter the location of orders and congestions in the network. Therefore, the PUN balance condition is imposed in order to guaranty the balance of payment:

$$\begin{array}{cc}{\Delta }^t+{\displaystyle \sum _{a\in A}}{\displaystyle \sum _{ss\in S{S}_a^{pun}}}{\lambda }_a^t·Q_{ss}^t·x_{ss}^t=PU{N}^t·\sum _{a\in A}\sum _{ss\in S{S}_a^{pun}}{Q}_{ss}^t·x_{ss}^t\hfill & \hfill \forall t\in T\end{array}$$

(36)

where ${\Delta }^t$ is the PUN imbalance at period t and the threshold of ${\Delta }^t$ is in the range [−1,5] (€) according to the market rules [6]. Here, the PUN orders are fully accepted if the submitted price $P_{ss}^t$ is greater than PUN, being fully rejected if the submitted price $P_{ss}^t$ is lower than PUN or partially accepted ($x_{ss}^t\in [0,1]$) if the submitted price $P_{ss}^t$ equals PUN.

Therefore, this SP is developed to find $PU{N}^t$, such that the PUN balance condition (36) is satisfied within the defined threshold. Normally, the objective is to minimize ${\Delta }^t$ by parametric optimization [8], but this is very time-consuming [7,18]. In general, the parametric optimization consists in iteratively changing the value of $PU{N}^t$ while fixing the demand of PUN orders as accepted if the submitted price $P_s{s}^t$ is greater or equal to $PU{N}^t$; otherwise, fix the demand PUN orders as rejected, and solve again the SWMP problem to obtain new MCPs and new ${\Delta }^t$. Then, based on the new sign of ${\Delta }^t$ the value of $PU{N}^t$ is changed according to the cumulated curve of the step-wise PUN demand orders where it represents a point on the price-quantity characteristic: the value of $PU{N}^t$ is moved from vertical segment to vertical segment, backward or forward, or on the same horizontal price-quantity line to minimize ${\Delta }^t$. The horizontal search is performed through manipulating the parameters of Equation (14), it is the most time consuming step of the search and it aims at refining an already contained value of ${\Delta }^t$. In this paper, the disadvantage of parametric optimization is overcome by adopting the “uplift” approach. In details, an “uplift” on PUN can be introduced to compensate the PUN imbalance allowing for the simplification of the PUN-SP and the reduction of the computation times by eliminating the horizontal search from the process.

The flowchart of the new PUN-SP is presented in Figure 3 (the continuous line). Here, the PUN-SP, only contains three main steps instead of five main steps in [7]:

$$\begin{array}{cc}PU{N}_{int}^t={\displaystyle \frac{{\displaystyle \sum _{a\in A}}{\displaystyle \sum _{ss\in S{S}_a^{pun}}}{\lambda }_a^t·Q_{ss}^t·x_{ss}^t}{{\displaystyle \sum _{a\in A}}{\displaystyle \sum _{ss\in S{S}_a^{pun}}}{Q}_{ss}^t·x_{ss}^t}}\hfill & \hfill \forall t\in T\end{array}$$

(37)

• Starting point: Initialization: $PU{N}_{eq}^t$ is initialized as an empty set and the first run of SWMP is executed to obtain, among others, the accepted ratio $x_{ss}^t$ and the zonal MCPs, i.e., ${\lambda }_a^t$; following $PU{N}^t$ is initialize to $PU{N}_{int}^t$ (37) and ${\Delta }^t$ is computed by (36). If ${\Delta }^t$ is outside the tolerance ($-1\le {\Delta }^t\le 5$ [6]) initialize the Vertical search: (i) if ${\Delta }^t$ is positive, select the backward vertical segment on the cumulated price-quantity curve of PUN orders with respect to the last accepted PUN order, while (ii) if ${\Delta }^t$ is negative, select the forward segment on the cumulated price-quantity curve of PUN orders with respect to the last accepted PUN order and go to the Vertical search. If ${\Delta }^t$ is inside the tolerance, exit PUN-SP.
• Vertical search: fix the demand in the PUN areas such that the bid price of accepted demand is higher or equal than $PU{N}^t$, solve SWMP and compute ${\Delta }^t$ using (36). (i) if the sign of ${\Delta }^t$ is changed with respect to the previous iteration (from negative to positive side), the Horizontal search is executed and $T\_PU{N}^t$ is set to 1 (14), else (ii) if ${\Delta }^t$ is negative, select the forward segment, else select the backward segment; further, based on the selected vertical segment, update $PU{N}^t$ to a value within the selected vertical segment that minimizes ${\Delta }^t$; finally, repeat Vertical search. If ${\Delta }^t$ is inside the tolerance, PUN-SP stops.
• Horizontal search: fix $PU{N}^t$ to the bid price of the selected horizontal segment and define $PU{N}_{eq}^t$ as the set of all demand segments for which $P_{ss}^t=PU{N}^t$. Subsequently, solve SWMP and stop PUN-SP (the dashed line in Figure 2).

2.4. Congestion Rent (CR)

Normally, if there are no losses on interconnector, CR can be directly obtained from the shadow prices [27]. However, the shadow prices also present the payment of demand to cover the real energy losses. In Europe, the losses are introduced on DC interconnectors, e.g., NorNed (Norway–Netherlands), Baltic Cable (Germany–Sweden), BritNed (UK–Netherlands), IFA (The UK–France), while there are many DC interconectors under construction [28]. Moreover, the method of [27] cannot be applied for loop flows, therefore for meshed grids which is the case of the European market. Thus, the method to compute the CR from the MCPs at two sides of an interconnector in paper [27] is not sufficient for the real situation. In order to obtain exactly the CR, the model generalizes the approach from [27] and computes the CR from the dual variables of all the network constraints (18)–(35), as below:

$$\begin{array}{cc}\hfill & C{r}_l^t=Cr{t}_l^t+Cr{t}_l^{t,ls}\forall l\in B{r}_l,t\in T\hfill \\ \hfill & Cr{t}_l^t={\beta }_{l,min}^t-{\beta }_{l,max}^t+({\gamma }_l^{t,D}-{\gamma }_l^{t+1,D})-({\gamma }_l^{t,U}-{\gamma }_l^{t+1,U})\hfill \\ \hfill & +\sum _{cb\in CB-CBA}(PTD{F}_{fa}^{t,cb}-PTD{F}_{ta}^{t,cb})·({\beta }_{max}^{t,cb}-{\beta }_{min}^{t,cb})\hfill \\ \hfill & +D_l^{t,+}·[c_l^t+{\mu }_{l,min}^{t,+}-{\mu }_{l,max}^{t,+}-\sum _{cb\in CBA}max(PTD{F}_{fa}^{t,cb}-PTD{F}_{ta}^{t,cb})·{\beta }_{cb}^t\hfill \\ \hfill & +{\gamma }_{fa}^D-{\gamma }_{fa}^U+{\gamma }_{fa}^{t,D}-{\gamma }_{fa}^{t+1,D}-{\gamma }_{fa}^{t,U}+{\gamma }_{fa}^{t+1,U}+\hfill \\ \hfill & ({\gamma }_{ta}^U-{\gamma }_{ta}^D-{\gamma }_{ta}^{t,D}+{\gamma }_{ta}^{t+1,D}+{\gamma }_{ta}^{t,U}-{\gamma }_{ta}^{t+1,U})·(1-los{s}_l)]\hfill \\ \hfill & +D_l^{t,-}·[{\mu }_{l,min}^{t,-}-{\mu }_{l,max}^{t,-}-c_l^t-\sum _{cb\in CBA}min(PTD{F}_{fa}^{t,cb}-PTD{F}_{ta}^{t,cb})·{\beta }_{cb}^t\hfill \\ \hfill & -{\gamma }_{ta}^D+{\gamma }_{ta}^U-{\gamma }_{ta}^{t,D}+{\gamma }_{ta}^{t+1,D}+{\gamma }_{ta}^{t,U}-{\gamma }_{ta}^{t+1,U}+\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill & ({\gamma }_{fa}^D-{\gamma }_{fa}^U+{\gamma }_{fa}^{t,D}-{\gamma }_{fa}^{t+1,D}-{\gamma }_{fa}^{t,U}+{\gamma }_{fa}^{t+1,U})·(1-los{s}_l)]\forall l\in B{r}_l,t\in T\hfill \end{array}$$

(39)

$$\begin{array}{cc}\hfill & Cr{t}_l^{t,ls}=({\gamma }_{ls}^{t,D}-{\gamma }_{ls}^{t+1,D})-({\gamma }_{ls}^{t,U}-{\gamma }_{ls}^{t+1,U})\forall l\in ls,ls\in LS,t\in T\hfill \end{array}$$

(40)

Equation (38), presents the CR, as the sum of (39) and (40) for all interconnectors. In Equation (39), the $Cr{t}_l^t$ of ATC model is obtained by the dual variables of constraints (18)–(24) and (28)–(35), while in the FB model, the $Cr{t}_l^t$ is computed by the dual variables of (18)–(20) and (25)–(35). Moreover, the interconnectors belonging to line set $ls$ are treated in a special way by (40), in which the $Cr{t}_l^{t,ls}$ is computed by dual variables of (30) and (31) for two consecutive periods.

2.5. Non-Uniform Price Problem (NUPP)

Now, the accepted ratio x of submitted orders and power flow $f_l^t$ are fixed and equal to the final solution of PUN-SP i.e., the pure market solution, and the set $F_l^t$ is generated by considering all the lines for which $f_l^t$ is not zero. This pure market solution can contain non-convexities as it was solved in the absence of the iterative SPs that mitigate paradoxically accepted or rejected orders. Thus, in the following, “uplift” payments are defined as additional payments with respect to the zonal prices, compulsory for the market participants and with the purpose to compensate the financial consequences of the non-convexities of the pure market solution (compensate the side-payments of MIC etc.). The “uplift” prices can also be negative for some market players and, therefore, represent a gain for them. Under these circumstances, it is necessary to minimize the positive “uplift” prices while respecting the market rules. Thus, starting from this market solution point, the “minimum-uplift approach” problem is built and solved according to the following principles:

• although the non-convex bids are block and MIC orders, the model introduces “uplift” payment for all bids, i.e., for PUN orders and the CR of the TSO. The PUN orders need to be introduced as a insignificant variation of PUN can lead to very high variation in the total payment from PUN orders, bringing an important positive contribution to the payment scheme. Additionally, TSO can refuse an insignificant amount of congestion cost as an award for the new payment scheme;
• minimize the maximum “uplift” payment ($\xi$), thus guarantying the consensus of all market players;
• an “uplift” payment can be a contribution or a subtraction: at the level of the market the net effect needs to be null so that the sum of all side-payments is equal to zero;
• the new payment must satisfy all the acceptance conditions; and,
• electricity price is a variable in this problem. Moreover, the relationship between the electricity prices of two areas needs to be preserved through the shadow price of the pure market solution.

2.5.1. Minimizing the Maximum Uplift Payment

The objective function (41) minimizes the maximum “uplift” payment, $\xi$, while the inequality constraints (42)–(44) impose that all “uplift” payments are smaller than $\xi$. The inequality constraint (42) also forces the “uplift” payment of CR of TSO with respect to the sign of $C{r}_l^t$, as the sign of $C{r}_l^t$ follows the direction of $f_l^t$. The inequality constraint (43) associates the maximum “uplift” payment $\xi$ to the hourly stepwise (except PUN orders), piecewise, profile block, flexible hourly block orders, while inequality constraint (44) is for PUN orders.

$$\begin{array}{c}\hfill \mathrm{Min}\xi \end{array}$$

(41)

$$\begin{array}{c}{u}_l^t·C{r}_l^t-\xi ·\left|C{r}_l^t\right|\le 0\forall l\in F_l^t,t\in T\hfill \end{array}$$

(42)

$$\begin{array}{c}{u}_i^t-\xi \le 0\forall i=ss\cup sp\cup pb\cup fh,a\in A,t\in T\hfill \end{array}$$

(43)

$$\begin{array}{c}{u}_{pun}^t-\xi \le 0\forall t\in T\hfill \end{array}$$

(44)

2.5.2. Payment Balancing Equation

The equality constraint (45) guarantees the make-whole payment mechanism has no missing money issue, even if the “uplift” payment is added, and the “uplift” payment is proportional with the accepted volume. Here, the positive uplift indicates that supply receives less and demand pays more (contribution) as compared to the pure market solution; vice-versa for negative “uplift”.

$$\begin{array}{c}{\displaystyle \sum _{b\in B_a}}{\displaystyle \sum _{pb\in b}}{x}_b·|Q_{pb}|·u_{pb}^t·(\underline{U_{pb}^t}-\overline{U_{pb}^t})+{\displaystyle \sum _{ss\in S{S}_a\backslash S{S}_a^{pun}}}{x}_s^t·\left|Q_s^t\right|·u_s^t\hfill \\ +{\displaystyle \sum _{ss\in S{S}_a^{pun}}}{x}_{ss}^t·|Q_{ss}^t|·u_{pun}^t+{\displaystyle \sum _{fh\in F{H}_a}}{x}_{fh}^t·\left|Q_{fh}^t\right|·u_{fh}^t=0\forall a\in A,t\in T\hfill \end{array}$$

(45)

2.5.3. The Shadow Price and the Contribution of TSO

The electricity price $p{r}_a^t$ is considered as the new electricity price and it is an auxiliary variable. The difference between these variables at the two sides of an interconnector ($p{r}_{fa}^t$ and $p{r}_{ta}^t$) is guaranteed by the shadow price. In equality constraint (46) the shadow price is given by the CR, the “uplift” of CR and the loss coefficient. In particular, the payment for losses is proportional to the electricity price, thus the loss coefficient is added to the importing area. Moreover, the contribution of TSO is defined by the CR minus the “uplift” payment of the interconnector. Thus, this constraint guarantees that without the “uplift” payment, equality constraint (45) is naturally satisfied (or no missing money).

$$\begin{array}{c}(p{r}_{fa}^t·(max(\frac{f_l^t·los{s}_l}{|f_l^t·(1-los{s}_l)|},0)+1)-p{r}_{ta}^t·|min(\frac{f_l^t·los{s}_l}{|f_l^t·(1-los{s}_l)|},0)-1|)+\hfill \\ (C{r}_l^t-u_l^t)·(1+{\displaystyle \frac{los{s}_l}{1-los{s}_l}})=0\forall a\in A,l\in F_l^t,t\in T\hfill \end{array}$$

(46)

2.5.4. Constraints on the Payment

Inequality constraints (47) and (48) model the upper and lower bound of the “uplift” payment of CR of TSO. Here, it can not be higher than the CR and lower than zero because TSO can not contribute or receive more than the CR of the pure market solution. This is in accordance with the market rules where the TSO, on one hand, cannot receive more than the CR of the pure market solution and, on the other hand, it cannot spend money on the market, as it is not a market player. Moreover, the CR has its own sign according to the power flow direction. This means that the absolute value of the “uplift” needs to be lower than the CR—Equation (47)—while its sign needs to follow the sign of the CR—Equation (48).

$$\begin{array}{cc}{u}_l^t·C{r}_l^t-{\left(C{r}_l^t\right)}^2\le 0\hfill & \hfill \forall l\in F_l^t,t\in T\end{array}$$

(47)

$$\begin{array}{cc}-u_l^t·C{r}_l^t\le 0\hfill & \hfill \forall l\in F_l^t,t\in T\end{array}$$

(48)

Inequality constraints (49)–(51) limit the “uplift” payment of hourly step orders with respect to the bid price, including (49) for stepwise orders (without PUN orders), (50) for piecewise orders, and (51) for PUN orders. It should be noted that, in these equations, the submitted quantities are in absolute values when they multiply the uplift, $u_{ss}^t$, because here the negative and positive value of the submitted quantity are used to model the supply and the demand, respectively. Therefore, the inequality constraints (49) and (50) bound the payment of demand not to be higher than the submitted price, while it is vice-versa for the supply. Additionally, the positive value of $u_{ss}^t$ means that demand pays more and supply receives less, thus these inequality constraints combine with the objective function (41) to minimize the loss-surplus of market players in comparison to the pure market solution. This logic is in-reverse for the negative value of $u_{ss}^t$. Therefore, the model does not have any limitations for the negative value of $u_{ss}^t$, since the market players are satisfied if this happens. Obviously, the logic of $u_{ss}^t$ for demand is also applied for the $u_{pun}^t$ of PUN in the inequality constraint (51) as PUN orders are on the demand side only.

$$\begin{array}{cc}{u}_{ss}^t·\left|Q_{ss}^t\right|·x_{ss}^t+(p{r}_a^t-P_{ss}^t)·Q_{ss}^t·x_{ss}^t\le 0\hfill & \hfill \forall ss\in (S{S}_a\backslash S{S}_a^{pun}),t\in T\end{array}$$

(49)

$$\begin{array}{cc}{u}_{sp}^t·\left|Q_{ss}^t\right|·x_{sp}^t+[p{r}_a^t-(P{0}_{sp}^t+\frac{P{1}_{sp}^t-P{0}_{sp}^t}{2}·x_{sp}^t)]·Q_{ss}^t·x_{ss}^t\le 0\hfill & \hfill \forall sp\in S{P}_a,t\in T\end{array}$$

(50)

$$\begin{array}{cc}{u}_{pun}^t+PU{N}^t-P_{ss}^t\le 0\hfill & \hfill \forall ss\in S{S}_a^{pun},t\in T\end{array}$$

(51)

At this point, an important clarification is required. Today, conventionally, the payment is done at the area’s MCPs (${\lambda }_a^t$) for all participants in the respective area. The payment scheme of the proposed model does not change this paradigm, but it is more complex as it is a combination of auxiliary variable $p{r}_a^t$ quantifying the new area price and “uplift” payment u of each order: the payment of each order is established as the sum between the two elements. This is compatible with the conventional payment, since, at limit when all the uplifts are null, constraints like (49) in combination with shadow price constraint (46) make $p{r}_a^t$ be equal to the MCP. In the general case, $p{r}_a^t$ is negatively affected by the uplift, but the minimization of the maximum uplift in the objective function (41) also minimizes this impact; moreover, constraints, like (49), assure that $p{r}_a^t$ will not be higher than the MCP of the pure market solution.

Inequality constraints (52) and (53) model MIC and PUN balance condition, respectively. Inequality constraint (52) is imposing the MIC to always be satisfied, while inequality constraint (53) is guaranteeing that the PUN imbalance is null for all periods. Notably, MIC orders are only for supply and PUN orders are only for demand. Therefore, the accepted quantity in inequality constraint (52) is in absolute values, while it is not an inequality constraint (53).

$$\begin{array}{cc}F{T}_p+(V{T}_p·\sum _{t\in T}\sum _{ss\in p}|Q_{ss}^t·x_{ss}^t\left|\right)-\sum _{t\in T}\sum _{ss\in p}(p{r}_a^t-u_{ss}^t)·|Q_{ss}^t·x_{ss}^t\left|\right)\le 0\hfill & \hfill \forall p\in P_a^{mic}\end{array}$$

(52)

$$\begin{array}{cc}\sum _{a\in A}\sum _{ss\in S{S}_a^{pun}}(p{r}_a^t+u_{pun}^t)·Q_{ss}^t·x_{ss}^t=\sum _{a\in A}\sum _{ss\in S{S}_a^{pun}}(PU{N}^t+u_{pun}^t)·Q_{ss}^t·x_{ss}^t\hfill & \hfill \forall t\in T\end{array}$$

(53)

Inequality constraints (54)–(56) model the constraints of “uplift” payments of block orders: constraint (54) for block orders (without link block orders), constraint (55) for link block orders, and constraint (56) for hourly flexible block orders. Likewise, inequality constraints (49) and (50), the logic of $u_{pb}$ and of the accepted volume are also used. Inequality constraint (54) forces the VWAP of profile block order to not violate the acceptance rule (1), while inequality constraint (55) guarantees that the total payment of the link block order belonging to one family is positive. Especially, according to the acceptance rule of link block order, equation (55) can not guarantee that the child will not reduce the surplus which is created by parent [2]. Thus, the set $LL{B}_a$ is created for the last accepted bid of the family using the pure market solution and added to (54).

$$\begin{array}{c}{\displaystyle \sum _{t\in T}}{\displaystyle \sum _{pb\in b}}\left[\right((u_{pb}^t-P_b)·|Q_{pb}|+p{r}_a^t·Q_{pb})·x_{pb}·(\underline{U_{pb}^t}-\overline{U_{pb}^t})]\le 0\hfill \\ \forall b\in (B_a\backslash L{B}_a)\cup LL{B}_a,t\in T\hfill \end{array}$$

(54)

$$\begin{array}{cc}\sum _{t\in T}[\sum _{b\in B_a}\sum _{pb\in b}[I{M}_{lb}^b·((u_{pb}^t-P_b)·|Q_{pb}|+p{r}_a^t·Q_{pb})·x_{pb}·(\underline{U_{pb}^t}-\overline{U_{pb}^t})]\hfill & \\ +\sum _{pb\in lb}((u_{pb}^t-P_b)·|Q_{pb}|+p{r}_a^t·Q_{pb})·x_{pb}·(\underline{U_{pb}^t}-\overline{U_{pb}^t})]\le 0\hfill & \hfill \forall lb\in L{B}_a,t\in T\end{array}$$

(55)

$$\begin{array}{cc}(u_{fh}^t·|Q_{fh}^t|+(p{r}_a^t-P_{fh}^t)·Q_{fh}^t)·x_{fh}^t\le 0\hfill & \hfill \forall fh\in F{H}_a,t\in T\end{array}$$

(56)

Finally, it is clear that the proposed NUPP model is an optimization model described by linear objective function and constraints and where all of the variables are continuous: the NUPP model is a linear programming problem and, hence, it will converge in computation times much smaller than the SWMP with PUN-SP.

3. Results

3.1. Case Studies Description

The Western European topology represented in Figure 4 includes 51 bidding areas and 66 lines [6]. Here, the network constraints reflect the reality, e.g., the losses are introduced for the DC links NorNed, Baltic Cable, BritNed, IFA (the discrete lines in Figure 4), and the hourly net position ramping constraint is implemented in the FB coupling region (France, Germany, Netherlands, and Belgium). Moreover, the line set conditions are used in bidding area NO1A for the NO5-NO1A and NO2-NO1A links and DK1A for the NO2-DK1A and SE3-DK1A links [29]. In this paper, the parameter of interconnectors, such as the capacity for all interconnectors and the ramping up and down of DC links, are obtained from [30]. It should be noted that the hourly ramping up and down for each hour is determined for DC interconnectors by network operators and this parameter is equal for all hours with 300 MW/h for IFA and BritNed, or 600 MW/h for NorNed and Baltic Cable. Finally, the data regarding the transmission tariff of the DC interconnectors is not public for the European DC links and, hence, the main purpose of parameter $c_l$ is to properly decompose the the flow in the FB interconnectors. For this reason and when considering that the MCP and bids are rounded to two decimals in the real market the parameter $c_l$ was set to 0.001 €/MWh, such that it acts as a penalty term in objective function (6), but it does not impact on the MCP.

Theproposed model is validated by two case studies, which are: (i) the artificial case and (ii) the real data case. The artificial case aims to test the robustness of the model and evaluates the new payment scheme in Europe with all types of bids. Therefore, it contains 117,492 randomly generated bids of hourly stepwise (all PXs), piecewise (EPEX Spot), and block orders (all PXs except OMIE and GME), 22 producers in OMIE are subjected to to MIC, SSC, and LGC (one producer can combine two or three conditions), while PUN orders only appear in the six geographical areas of GME [1] shown in Figure 4 in grey color (CNOR, CSUD, NORD, SARD, SICI, and SUD). All block orders are on the supply side in order to effectively analyze the results.

The real data case contains the actual bids of Central West of Europe (block orders and piecewise orders) [31], OMIE (only step orders) [32], and GME (PUN orders) [33] from February to December of 2017. It is used to evaluate the economic performance of applying the proposed model in real life in terms of SW, AV, VWAP, computation time and maximum “uplift” payment.

All of the case studies are run by a personal computer equipped with an Intel(R) E5-1620 v3 @ 3.5 Ghz processor and 32 Gb RAM. The models were implemented in GAMS environment [34] and solved with CPLEX solver 24.0.2 with the default settings (the optimality gap is 10${}^{-6}$).

3.2. Artificial Case

The case study is run by the conventional model [7] and the proposed model. Subsequently, the results are compared in order to evaluate the numerical performance and impact of the new payment scheme on the single European DAM from a global perspective. Moreover, the impact of make-whole payment mechanism in the proposed model is analyzed from various aspects by running four configurations representing the level of applying the “uplift” in Europe. Here, the “uplift” payment can be introduced only for hourly step and block orders in Config 1; or with either PUN orders or CR in Config 2 and 3, respectively; or, with all in Config 4.

3.2.1. Global Criteria

The case study is run by both models and compared in

Table 1. The comparison of conventional and proposed model.

Model	Welfare	Volume	Time	NNS [M€]
	[M€]	[GWh]	[min]	Supply	Demand
Conventional	6821.42	6935.97	90	_	_ ${}^a$
Proposed	6822.36	6968.58	5	152.81	4070.02

^a The conventional model does not need NNS.

from three points of view: (i) SW, (ii) total AV (supply and demand), and (iii) computation time. Moreover, the last two columns show the total NNS of all in-the-money bids that can be transferred to mitigate the PABs and PAMICs. NNS is the difference between the received money and the required money for supply, vice-versa for demand. It should be noted that, at this step, the proposed model has been run without the NUPP, therefore, the pure market solution is shown.

It should be emphasized that conventional model requires many SWMP iterative launches [7] through PAB-SP, PRB-SP, PAMIC-SP, and PRMIC-SP in order to mitigate PABs, PRBs, PAMICs, and PRMICs. Thus it is very time consuming. In fact, the conventional model is not able to completely solve the non-convexity issues within the time limit [6], thus not many PABs, PRBs, PAMICs, and PRMICs can actually be mitigated. Meanwhile, the proposed model does not contain those SPs except the simplified PUN-SP and, therefore, it requires many less SWMP iterations. Moreover, the NUPP is LP and, thus, it will require an insignificant computation time. Finally, the proposed model increases the total social welfare and accepted volume because it does not eliminate any accepted bids as the conventional model does in the PAB-SP, PRB-SP, PAMIC-SP, and PRMIC-SP.

From Table 1, it is clear that the proposed model is more beneficial than the conventional model [7], because it takes significantly less time to converge (90 min. for the conventional model and only 5 min. for the proposed model), while the social welfare and accepted volume are increasing (see the second and third columns of Table 1). Hence, the next section only focuses on the impact of the NUPP and the new pricing scheme in the proposed model.

3.2.2. Economic Aspects

All of the loss-incurred by PABs and PAMICs in the pure market solution are mitigated by the minimum “uplift” approach in NUPP, as they are no longer conflicting with the acceptance rules. The proposed model needs to find the side-payment that is calculated by accepted volume from the pure market solution multiplied by the “uplift” payment from NUPP that helps to recover the incurred loss in order to find the Walsarian equilibrium while no missing money condition is guaranteed.

Table 2. The report of four configurations.

Config	TSO	PUN	$\mathbf{\Delta }{\mathbf{NNS}}_{\mathbf{s}}$	$\mathbf{\Delta }{\mathbf{NNS}}_{\mathbf{d}}$	Max_$\mathbf{\Delta }\mathbf{Pr}$	$\mathit{\xi }$
			[‰]	[‰]	[%]	[€/MWh]
1	No	No	+0.194	−0.0139	0.0	0.0224
2	No	Yes	−0.337	+0.0087	0.0	0.0198
3	Yes	No	−0.330	+0.0095	0.225	0.0223
4	Yes	Yes	−0.324	+0.0092	0.2	0.0197

presents the results for each configuration, based on three criteria: (i) the deviation of NNS for supply and demand with respect to the values of Table 1, denoted by $\Delta NN{S}_s$ and $\Delta NN{S}_d$ in permille, (ii) the maximum absolute deviation of electricity price, Max_$\Delta Pr$ (in percentage) between before MCP and after NUPP ($p{r}_a^t$), and (iii) the maximum “uplift” payment ($\xi$). Here, the positive value of $\Delta NN{S}_s$ shows that the payment is increasing, while the negative value indicates the decrease of the payment, and vice-versa for $\Delta NN{S}_d$.

Table 2 shows that Config 1 performs in the opposite way with respect to the last three configurations. Here, suppliers receive less payment to help demand buy more energy with lower price in comparison to the pure market solution. Thus, even if the model converges, the result seems to not be convincing, because players perceive the unfairness from the economic point of view. Thus, Config 1 is not discussed further.

Particularly, Config 2 has the smallest $\Delta NN{S}_s$ and $\Delta NN{S}_d$ in comparison to Config 3 and 4. This means that the demand pays the most and supplier receives the least among the last three configurations which are, again, not fair.

About the second criteria, the value of Max_$\Delta Pr$ is not null in Config 3 and 4, because the participation of TSO, $u_l^t$, leads to the change of the auxiliary variable $p{r}_a^t$, with respect to the $MC{P}_a^t$ of the pure market solution. Here, the maximum deviation of electricity price, and the maximum uplift of Config 4 are lower than Config 3, which means that the variation in payment before and after NUPP of Config 4 is the least.

Figure 5 shows VWAP for the conventional model and for Config 4. Here, the VWAP in the proposed model is lower than in the conventional model for all periods, because, in the conventional model, some in-the-money bids are not accepted as they are PAMICs and PABs. To balance the generation with the demand, expensive bids will be accepted instead. On the other hand, the PAMICs or PABs are accepted in the proposed model and the acceptance of the expensive bids is avoided. This leads not only to lowering of VWAP in the proposed model, but also to the modification of the trend of the VWAP during the day, especially in the hours when the number of PAMICs or PABs is high, like at hours 2 to 4. To sum up, according to the three criteria, Config 4 is the most efficient configuration, because it can mitigate the loss-incurred with the smallest deviation of payment with respect to the pure market solution.

3.2.3. Impact on Orders

In this sub-section, the main outcomes of the proposed model derived from the previous results are discussed for each type of order:

Block orders

The compensation of PABs allows for reducing the number of PRBs, thus the model enhances the fairness and social welfare [9]. In detail, the number of PRBs is 371 in the conventional model, while the proposed model only gives 34 PRBs, which hold a small volume and are deep-in-money (see Appendix C).

Complex orders

The impact of proposed model on OMIE is positive because PAMICs and PRMICs do no longer exist. The loss-incurred of PAMICs is recovered by constraint (52) in NUPP. This mechanism allows for suppliers to sell more, while the MIC (2) is satisfied. Here, there is only one MIC order, which is realized as PAMIC in the pure market solution. Consequently, this order is deactivated in the conventional model, but it is dispatched in the proposed model. Therefore, the total accepted volume of supply over 24 periods increases with 10 GWh in OMIE when using the proposed model.

PUN orders

The presence of PUN orders in the “uplift” payment scheme minimizes the maximum “uplift” payment, as it can be seen in Table 2 in Config. 2 and 4. Here, there are eight over 24 periods when PUN is decreased ($u_{pun}^t$ is negative), and marginally increase for the rest. It means that PUN orders not only contribute, but also take advantage of the new payment scheme.

3.3. Real Data Case Study

In this case study, the model is run with a simplified Western European topology that is shown in Figure 6, which only includes 30 bidding areas due to the limited availability of data. Here, the PUN areas are shown in grey color. The main aim of this test is to evaluate the ability of the proposed model when applied in a real life situation over a long period of time. Here, the data are from February to December of 2017, excepting the 29th of October, because this day contains 25 hours, which leads to a difficulty in harmonizing the data. Therefore, the real case contains 333 days in total. It should be noted that the block and piecewise orders exist in NL, BE, FR, and DE; PUN orders are in GME while OMIE contains only step orders, since the data are modified by the market operator before being published online; thus there are no complex orders in this test. It should be noted that all days are run with the two models: the conventional model and the proposed model with Config 4, as it previously proved to be the best configuration.

First of all, the percentage of PABs over all submitted block orders in a month from February to December 2017 obtained using the conventional model is shown in

Table 3. Percentage of PABs over all submitted blocks with conventional model from February to December of 2017.

Average Value	Min Value [Month]	Max Value [Month]
10.32%	7.63% [August]	13.66% [March]

. The first column shows the average percentage of PABs, meanwhile, the second and third column presents the minimum and the maximum percentage. From Table 3, the number of PABs in the market solution is high with about 10% on the average for each month, while the maximum is for March and the minimum is for August. However, with the proposed model, the number of PABs is zero for all months. Thus, the proposed model can thoroughly solve the unfairness of PABs.

Figure 7 shows the histogram of the difference obtained between the proposed model and the conventional model in terms of SW and AV. The smaller window in Figure 7 details the area around zero on the x-axis. The variation of SW and AV between the two models is small, since the distribution of SW in Figure 7a and AV in Figure 7b is mostly concentrated around zero. In detail, across the studied year, there are positive and negative variations in almost equal proportion, which can be interpreted as the proposed algorithm does not give major changes in term of SW and AV over the length of the studied year; however, a slight increase tendency in the SW can be observed in Figure 7a, which is positive.

Table 4. The report of computation time of both models.

Model	Average Value	Min Value [Day]	Max Value [Day]
	[s]	[s]	[s]
Conventional	453	48 [26th of April]	4532 [20th of July]
Proposed	337	88 [31th of July]	1448 [4th of November]

shows the computation time of both models, in which the first column is for the average value, the second column reports the minimum computation time and the related day, and the third column shows the maximum computation time and the related day. From Table 4, from the average value point of view, the proposed model takes less time to converge with 453 s for the conventional model and 337 s for the proposed model. This difference is of about two minutes, so it is significant. In the second column, the minimum computation time of the proposed model is higher than that of the conventional model, because, in this case, the PRB-SP does not activate, since there is no PRB found in the market solution after PUN-SP (see Figure 1). Meanwhile, the NUPP of the proposed model is activated in any case (see Figure 2). The significant reduction of the computation time introduced by the proposed model is confirmed also by the third column of Table 4, where the maximum recorded over the analyzed period decreased by about three times, from the unfeasible value of 75 min. to the more reasonable value of 25 min.

Table 5. The report of the daily maximum “uplift” payment in €/MWh.

Average Value	Min Value	Max Value
0.0036	1.5427 × 10−7	0.0888

shows the statistics regarding the daily maxmimum “uplift” payment over the analyzed period, where the maximum value is only 0.0888 €/MWh for 3rd of August and the minimum is 1.5427 × 10${}^{-7}$ €/MWh for 11th of May, while, in general, it is only 0.0036 /MWh on average. It is clear that the uplift is small, thus it will not significantly affect the payment of each participant.

In the following, the difference between the results obtained for the proposed model and the results obtained for the conventional model is shown for: revenue of supply and payment of demand (Figure 8), MCP (Figure 9), and CR (Figure 10). The smaller window in these figures details the area around zero on the x-axis. Each element of Figure 8 and Figure 10 is considered for each day (333 days), but each element in Figure 9 is for each hour (total 7991 h). Figure 8 and Figure 9 show an even distribution and concentration around zero of the variation of the analyzed financial indicators for most of the days: the supply revenues/demand payments present insignificant variations in approximately 80% of the 333 days (see Figure 8), while the MCP in about 90% of the 7991 h (see Figure 9). This, coupled with an even distribution between the positive and negative values, confirms that the proposed model has an insignificant impact on the market performance when it comes to these financial indicators.

Finally, in Figure 10, the CR of the proposed model is reduced, since the deviation is mostly distributed on the negative side. It is explained by the TSOs willingness to refuse a small amount of CR to support the new payment scheme. Here, the difference in percentage is quite high in comparison to the other figures, but it is still reasonable by itself: in most of the cases, the variation is in the range −5 ÷ 0%. This could be seen as the only downside of the proposed model.

4. Discussion

The mathematic model used to clear the Integrated European DAM is a very complex tool due to the huge size of the model (number of equations and variables representing the bids and the market rules) and due to the diversity of the market rules of the various countries it contains (PUN orders in Italy, MIC orders in Spain, FB network modelling in Central-West Europe, and ATC modelling in the rest). Additionally, many market rules introduce non-convexity issues in the model giving large computation times and introducing various economic problems, like paradoxically accepted or rejected bids.

Thus, the authors have first analyzed the approaches encountered in literature that solve these issues. Clearly, one of the best encountered was the “minimum-uplift approach”, which guarantees the Walrasian equilibrium. In this approach, the bids are accepted according to a pure maximization of the SW, while the loss-incurred of the paradoxical accepted or rejected orders are compensated through an additional uplift price for every market participant. The approach minimizes this uplift. However, according to the literature review, this approach has never been implemented at the full scale of the European DAM model, i.e., by considering all market rules and a realistic number of bids. This paper has developed a model that fully applies the concept. Here, the “uplift” payment is not only introduced for paradoxically accepted or rejected block orders, but also for all other types of orders like MIC or PUN orders. Second, the “uplift” is also introduced for the CR as it can be beneficial that also the TSO contributes in the “uplift” scheme. The model thus conceptualized was formulated into a “minimum-uplift” problem where not only the costliest uplift is minimized but also all market rules are fulfilled. Finally, this formulation has allowed great simplification of the conventional market clearing algorithm, since it was possible to eliminate most of the SPs mitigating the non-convexities from the conventional model.

With the new model formulated, three are the questions regarding its application to the real market: (i) will indeed the computation time reduce dramatically while, important, (ii) will the economic performance of the model be at least equal to the conventional model, i.e., the approach currently in use by the PXs, and, most important, and (iii) would the market participants perceive negatively the introduction of this model? To all three questions, the authors of the paper have answered by performing simulations on realistic test cases: an artificial one where almost the entire Integrated European DAM was modeled through artificially created bids that respect the characteristics of DAM in terms of type, size, and number, and a real data based case where a reduced, but still relevant, Integrated European DAM was modeled through the real data available online. The detailed analysis of the test cases presented in the previous section showed that numerical performance of the proposed algorithm is improved with respect to the conventional one: the computation time is dramatically reduced. However, does the proposed model change the economic performances of the market? For this, the two models were compared in terms of daily Social Welfare, daily accepted volume and MCPs. For all the analyzed cases, the absolute value of the variation of these indicators between these two models proved to be insignificant. Moreover, over a long period of time the distribution of these variations proved to be symmetric around zero, therefore, in long-term the new model shows the same behavior and economic performances with the conventional one.

Additionally, the final question, would the market participants perceive the results of the proposed model negatively? According to the results, the variation of the total daily payment of demand and total daily revenue of supply behaves statistically like the previous metrics, so there is no negative impact on them. Not least, in average, the maximum “uplift” payment for a market session is very small, well bellow 1 €cent/MWh, which is a very small value when considering that the MCPs are of the orders of tens of €. Moreover, also the maximum registered value was very small, i.e., about 9 €cent/MWh. All this show that market players, i.e., the sellers and the buyers are not affected in the proposed model with respect to the conventional one. The only actor that is slightly negatively affected seems to be the TSO, which tends to lose a small part of its congestion rent. But this can be fixed by imposing an upper limit to the CR “uplift” according to the TSO requirements. It can be easily predicted that this will not have a negative impact on the market players, since, in absolute terms, the volume of money involved in the CR is much smaller than the one involved in the transacted volumes of energy; therefore, there is a very high buffer on the market participants side to compensate a reduction of the CR “uplift”.

In conclusion, the results of the paper show that applying the “minimum-uplift” approach in the actual Integrated European DAM is a very good option: tests using real market data over an extended period of time showed that there are no disadvantages of this approach; there are only advantages, since the numerical performances of the proposed approach surpass the market model that is currently in use.

Clearly, future modifications in the market rules, e.g., the introduction of new bidding schemes, or adoption of different network models, e.g., all European Market passes to Flow Based approach or other, will require new developments in the proposed model. When doing this, it becomes critical to avoid a significant negative difference in terms of prices and payments between the conventional model (which, the market participants will always tend to take as reference) and the new model, difference that will certainly discontent the market players and make the proposed model inefficient. Thus, the extension of the proposed model should be carefully evaluated against this effect. The proposed model should be upgraded, such that the principles enunciated in this paper are respected. For example: (i) at the level of the market, the net effect needs to be null so that the sum of all side-payments is equal to zero and (ii) The new payment must satisfy all of the acceptance conditions. Moreover, exhaustive simulations should be performed for validation.