Optimal Coordinated Bidding of a Profit Maximizing, Risk-Averse EV Aggregator in Three-Settlement Markets Under Uncertainty

Abstract

This paper develops a two-stage stochastic and dynamically updated multi-period mixed integer linear program (SD-MILP) for optimal coordinated bidding of an electric vehicle (EV) aggregator to maximize its profit from participating in competitive day-ahead, intra-day and real-time markets. The hourly conditional value at risk (T-CVaR) is applied to model the risk of trading in different markets. The objective of two-stage SD-MILP is modeled as a convex combination of the expected profit and the T-CVaR hourly risk measure. When day-ahead, intra-day and real-time market prices and fleet mobility are uncertain, the proposed two-stage SD-MILP model yields optimal EV charging/discharging plans for day-ahead, intra-day and real-time markets at per device level. The degradation costs of EV batteries are precisely modeled. To reflect the continuous clearing nature of the intra-day and real-time markets, rolling planning is applied, which allows re-forecasting and re-dispatching. The proposed two-stage SD-MILP is used to derive a bidding curve of an aggregator managing 1000 EVs. Furthermore, the model statistics and computation time are recorded while simulating the developed algorithm with 5000 EVs.

1. Introduction

Due to the growing penetration of distributed energy resources (DERs), including photovoltaic panels (PVs), electric vehicles (EVs), and thermostatically controlled loads (TCLs), power systems are benefiting an increasing control flexibility, not only from the supply side, but also from the demand side. Worldwide EV integration level is assumed to grow up to 20 million by 2020 [1]. Hence, the usage of EV batteries to support a power system has a significant potential [2]. Several types of batteries are used as energy storage for EVs, for example lithium-ion, lead acid and nickel metal hydride batteries. Battery market analysis based on efficiency, cost, durability, performance, power and energy can be found in [3,4,5].

EV flexibilities are combined by aggregators to participate in corresponding markets and to provide bids. An aggregator aims to satisfy the driving needs of its fleet of EVs at the lowest possible cost [2,6]. In addition, the aggregator has an economic incentive to participate in different market places targeting higher revenue [7,8,9,10]. Because they can increase or decrease their power output quite fast, EV aggregators can maximize their profit while at the same time providing reserve power. In references [11,12,13,14,15,16], the authors study the potential of using EVs as a flexible power providers for real-time balancing market and system reserves.

The grid-to-vehicle (G2V) problem of bidding regulation and spinning reserves is investigated in [12]. The vehicle-to-grid (V2G) interaction with the grid as a flexibility source for real-time balancing market is modeled in [13,15,16]. However, using the batteries as storage devices for grid purposes reduces their lifetime [15,17], and therefore, EV owners must be compensated for the lost utility of their batteries.

Optimal bidding strategies are particularly relevant to EV aggregators, since they have the possibility to store energy and choose the best time to charge/discharge the electric power according to the market conditions. When prices are high, electric power is discharged and sold, whereas when prices are low, the electric power is charged to meet the driving needs of the aggregator’s fleet of EVs. Thus, modeling of market price uncertainties is an important feature to be considered when developing a bidding strategy. The optimal bidding problem of EVs is studied in [18], taking into consideration uncertain market price and nonlinear charging curve.

Two approaches can be emphasized for modeling uncertainty in a coordinated bidding problem to address the uncertain nature of market conditions and fleet characteristics: (a) risk management and (b) stochastic programming. The EV aggregator needs to prepare its bid function considering the volatility of the electricity prices. For that purpose, ref. [19] proposes two-stage stochastic optimization of EV participation in electricity markets incorporating VaR risk measure. To manage short-term volatility in market prices the conditional value at risk (CVaR) as a hedging instrument is incorporated in EV aggregation models [14,20]. The authors in [21] apply the concept of CVaR on an hourly basis for controlling the risk of hydropower producer’s hourly profit. In [21], it is illustrated that hourly bid functions derived from an hourly CVaR include more accurate information on the short-term price risk as compared to the bid functions derived from the daily CVaR.

The existing literature on bidding strategies for EV aggregators that uses stochastic programming can be categorized as (1) day-ahead market bidding and (2) day-ahead and balancing markets bidding. Optimal bidding strategies for participation of an EV aggregator in the day-ahead electricity market are modeled in [9,10].

Imbalances between volumes discharged/charged and volumes dispatched in the day-ahead market are settled in the balancing market, where market players submit their bids to increase or decrease the generation/consumption. Two-stage stochastic optimal bidding models of an EV aggregator for trading in day-ahead energy and regulation markets are presented in [11,13,14,15,17]. The reference [22] presents planning and operation models for an EV sharing community in day-ahead and regulating markets. A game theoretic approach is developed in [23] for analyzing the competition among the EV aggregators acting in day-ahead energy and ancillary services markets with variable wind energy.

In addition, adjustments in the actual discharge/charge can also be done by trading energy in the intra-day market, which allows for trading physical electricity between the participants. In previous studies [9,10,11,12,13,14,15,16,17,18,19,20,21,22,23], optimal bidding strategies for an EV aggregator have been developed. However these studies do not include participation of the aggregator in the intra-day market. Worldwide, significant increase of RES integration in the power system causes highly volatile day-ahead, intra-day and real-time market prices. Consequently, the intra-day market becomes a potentially promising market platform for a flexible market actor to participate in. Therefore, coordinated bidding is highly attractive for an EV aggregators who are able to engage in sequential day-ahead, intra-day and real-time markets and thereby increase their profit.

For a small market participant (a price-taker EV aggregator), whose decisions have no impact on the market outcomes, the bidding process is subject to uncertainty in day-ahead, intra-day, real-time market prices and the availability of the aggregator’s fleet of EVs.

This paper is the extension of the work in [16], and develops a stochastic, dynamically updated two-stage multi-period optimal bidding strategy for a risk-averse EV aggregator who participates in the day-ahead, intra-day and real-time markets. The model addresses the uncertain nature of market conditions and fleet characteristics. A Markov-based Holt winter (HW) model is used to predict electricity prices in all three market places. The uncertainty in day-ahead, intra-day and real-time market prices is represented by a set of scenarios, generated by a forecasting tool (based on the historical data) and reduced by the backward-reduction algorithm. Unlike previous two-stage stochastic formulations in [11,14,16,19], this model accounts dynamically for the clearing nature of the intra-day and real-time markets while deriving optimal bids for day-ahead, intra-day and real-time markets. In addition, the hourly version of CVaR (T-CVaR) is incorporated in the coordination to hedge against short-term market price volatility. The rolling planning is employed to update the scenario trees of intra-day and real-time prices within the day of planning. Finally, the developed model enables the aggregator to manage both stationary storages and EVs. The main contributions of the paper are:

• Development of a two-stage SD-MILP optimal coordinated bidding model for an aggregator who manages numerous storage units (stationary and EVs) and trades electric power in three-settlement markets, taking into account uncertainties in market price and fleet characteristics, as well as existing market rules. This model can be used for market exchange irrespective of (1) production or consumption technology and (2) mobile or stationary storage unit.
• Derivation of optimal coordinated charge (discharge) bids for day-ahead, intra-day and real-time markets with reasonable computation time using scenario-reduction techniques.
• Uncertainty modeling in all three market prices as well as of EV mobility parameters.
• Incorporation of the hourly CVaR (T-CVaR) to focus on the lower tails of the profits on hourly bases.

The paper is structured as follows. Section 2 introduces the general decision-making procedure of an EV aggregator participating in three-settlement markets. Section 3 explains the uncertainty modeling related to market prices and EV availability as well as the concept of rolling planning. Section 4 describes the two-stage dynamically updated stochastic mathematical model formulation of an aggregator, who manages a big number of energy storage units. Section 5 provides the case-study results and Section 6 concludes the paper.

2. Decision Process Framework

The typical decision-making procedure of an EV aggregator, as an electricity market business entity, can be considered similar to the decision-making framework of a conventional producer/consumer. Acting in highly uncertain market environments and with uncertain nature-of-fleet characteristics, the aggregator is willing to participate in intra-day and balancing markets closer to real-time to benefit from correcting actions on the day-ahead commitments. Thus, the EV aggregator’s decision process framework includes:

• The daily bidding in day-ahead markets under the volatile day-ahead market prices, meanwhile allocating capacity in adjustment and regulating markets.
• The hourly/quarter-hourly intra-day selling/buying and balancing up/down-regulating positions driven by intra-day and real-time volatile market prices, and the strict requirements to satisfy driving needs of its fleet of EVs. These adjustments are necessary to address the errors stemming from the availability and price forecast.

The decision process is depicted in Figure 1 for a small case with a set of day-ahead, intra-day and real-time price scenarios. The circles represent the decisions made by the EV aggregator, while the squares refer to the variable set calculated after the first stage decisions. Please note that the second stage scenarios contain information related to both intra-day and real-time market prices.

3. Scenario Generation

3.1. Market Places

Electricity markets are multi-settlement markets and are usually featured by the day-ahead, intra-day and real-time markets. In a day-ahead market, an auction is carried out for energy delivery for the following day with hourly increments. The day-ahead electricity market is operated in three steps: (1) bidding, (2) market clearing and (3) pricing [24]. The transmission system operator (TSO) runs an economic dispatch problem that considers received bids from the market participants, with the aim of meeting the system demand at the minimum possible cost. The market clearing price is calculated as the dual variable of the load balance constraint in the economic dispatch problem.

Intra-day markets are open directly after the clearing process of the day-ahead market, and close shortly before the actual delivery of that hour. This market floor aims to decrease the imbalances occurring between day-ahead and real-time markets. Intra-day markets are usually not liquid, and the trading of the physical electricity takes place between the participants. Thus there is no clear relation between day-ahead and intra-day market prices; for example within a specific hour the intra-day selling/buying price can be less/grater than cleared day-ahead market price for that specific hour.

Finally in the real-time market the TSO again runs economic dispatch based on the received bids from producers and consumers, aiming to meet the real-time market demand at the lowest possible cost. Similar to the day-ahead market, the real-time price is cleared and calculated as the dual variable of the load balance constraint in the real-time economic dispatch problem. Thus, each deviation occurring in the real-time market is reflected in real-time market prices. If the real-time market demand happens to be higher than the day-ahead market demand, the flexible generation units will up regulate and the market will experience up-regulation. The up-regulated price is often higher than the day-ahead market price. In a similar way, if the real-time market demand is less than the day-ahead market demand, the market experiences down-regulation. The down-regulation price will often be less than the day-ahead market price. These existing relations between day-ahead and real-time market prices are respected while performing forecasting, scenario generation and reduction.

3.2. Market Price Scenario Generation and Reduction

Studies in [21] show that the Markov-based HW model is one of the best performing predicting tools for electricity market prices. Accordingly, the day-ahead market price scenarios are predicted using the standard HW model [25] as in [21]. For that purpose historical data related to day-ahead market prices reported on Nord Pool website is used [26]. Then, the day-ahead market price scenarios are generated considering the expected values, the variances of day-ahead market prices and assuming normal distribution.

In contrast to day-ahead market prices, both price magnitude and direction have to be forecasted for intra-day and real-time markets. The combination of HW and Markov models is incorporated to forecast both intra-day and real-time market prices. The proposed Markov-based HW model for modeling and predicting the intra-day and real-time prices are presented in detail in [21]. To predict intra-day and real-time market price magnitude, the intra-day and real-time market historical prices are collected and processed. The intra-day market CSV excel files containing intra-day market trading activities are obtained from Nord Pool (these excel files are not available in the Nord Pool database [26]). Then, this data is preprocessed in two steps: first the data is grouped as buying and selling trading within each hour, and then buying and selling price series with hourly resolution are designed taking the average of all buying and all selling trading prices within each hour correspondingly. Similar to day-ahead market prices, the real-time market price series is available from the Nord Pool website [26].

To model the probabilistic information on random prices, a large number of scenarios is initially generated. The initial number of scenarios is then reduced such that the probability information is maintained as much as possible [27]. The problem of optimal scenario reduction can be stated as:

$$\begin{array}{c}\mathrm{Min}.D_J={\displaystyle \sum _{i\in J}}{\omega }_i{\mathrm{Min}.}_{j\notin J}\parallel {\lambda }^i-{\lambda }^j\parallel \hfill \end{array}$$

(1)

$$\begin{array}{c}\mathrm{Subject}\mathrm{to}:J\subset \{1,\dots ,N\},Card\{J\}=N-n.\hfill \end{array}$$

(2)

The optimization problem (1) and (2) can be approximately solved using the backward reduction technique. In this technique, price scenario ${\lambda }^k$ is deleted such that $D_{J^{k-1}\cup \{{\lambda }^k\}}$ $={\mathrm{Min}}_{\lambda \notin J^{k-1}}{D}_{J^{k-1}\cup \{{\lambda }^k\}}$ [27]. As a result, the backward reduction algorithm determines a subset from the initial scenario set and assigns new probabilities to the preserved ones.

3.3. Availability Simulation

In order to generate mobility scenarios for mimicking the uncertain driving patterns, a Monte Carlo simulation tool is incorporated. Then, discrete cumulative distribution functions (cdf) are employed. The cdfs are derived considering the following three type of probabilities: (1) the probability of travel on a specific day; (2) the probability that a trip starts in a specific hour; and (3) the probability that a trip covers a certain distance. It is assumed that these distributions are mutually independent as in [14].

3.4. Rolling Planning

As stated above, bids to day-ahead market for all hours of the delivery day are submitted simultaneously before noon the day prior to the delivery day. In contrast, both intra-day and real-time markets are continuous, hourly markets. This means the EV aggregator has new price information realized after the day-ahead market clearing and before the intra-day and real-time markets closure. Rolling horizon planning is employed to take advantage of the continuously released information over time (revealed intra-day and real-time market prices). Hence, the scenario tree of intra-day and real-time prices is updated within the planning day using the rolling planning. Let ${\Omega }_{[t,T]}$ be the scenario tree predicted for hours t to T using the historical prices up to hour t. Consequently, the ${\Omega }_{[t,T]}$ is dynamically updated by intra-day and real-time prices revealed until hour t. The ideal case would be to update the ${\Omega }_{[t,T]}$ on an hourly bases. However, solving the stochastic model dynamically increases the solution time exponentially. To solve the model with reasonable computational time, ${\Omega }_{[t,T]}$ is updated every few hours which is called ‘iteration’. For every iteration, a new scenario tree is used that updates the information related to intra-day and real-time market prices. The implementation of the rolling planning to update the intra-day and real-time market prices for every four-hour period is depicted in Figure 2. Note that each scenario contains information for both intra-day and real-time market prices.

4. Mathematical Problem Formulation

The current bidding problem formulation of an EV aggregator is an extension of the model presented in [16], where the trading possibilities in intra-day market are ignored and the optimization is risk-neutral.

A profit-maximizing risk-averse aggregator is assumed, who participates in day-ahead, intra-day and real-time markets. The aggregator maximizes its expected profit while minimizing its exposure to market price risk. The profits in different hours might have different statistical behavior due to the fact that the market prices can be functions of fundamentally different drivers for different trading hours within the planning day. The time-dependent conditional value at risk (T-CVaR) is proposed to manage the lower tail of profit distribution in each hour in Equations (3)–(5). Equation (3) aims at maximizing hourly CVaR subject to Equations (4) and (5). Here, constraints (4) and (5) set conditions on the hourly risk measure. According to Equation (4), $r_{s,t}=0$ if the profit for scenario s and hour t is greater than ${\xi }_t$. $r_{s,t}$ is defined by the difference between ${\xi }_t$ and the relative profit for all other scenarios and hours.

$$\begin{array}{cc}\hfill {T-CVaR}_t(\delta )=& {\mathrm{Max}.}_{{\xi }_t,r_{s,t},{\Pi }_{s,t}}{\xi }_t-\frac{1}{1-\delta }{\displaystyle \sum _s}{\omega }_s{r}_{s,t}\hfill \end{array}$$

(3)

$$\begin{array}{c}\mathrm{subject}\mathrm{to}:{\xi }_t-{\Pi }_{s,t}\le r_{s,t};\forall s,t\hfill \end{array}$$

(4)

$$\begin{array}{c}{r}_{s,t}\ge 0;\forall s,t,{\xi }_t,r_{s,t},{\Pi }_{s,t}\in \mathbb{R}.\hfill \end{array}$$

(5)

For a given confidence level $\delta$, the ${T-CVaR}_t(\delta )$ defines the expected value of profits less than $(1-\delta )$-quantile of distributed profit in hour t.

Two-Stage Stochastic Optimal Strategy of an EV Aggregator

The stochastic optimization problem stated in Equation (6) aims at the maximization of scenario-weighted expected profits from day-ahead energy trading, intra-day and real-time power exchange, while minimizing its exposure to market price risk. Hourly resolution is assumed for day-ahead, intra-day and real-time markets.

$$\begin{array}{c}{\mathrm{Max}.}_{\Phi }\mathbb{E}[{\Pi }^{Tot}]+\chi {\displaystyle \sum _t}{T-CVaR}_t={\displaystyle \sum _{t=1}^T}{\displaystyle \sum _{s=1}^{|{\Omega }_{[t,T]}|}}{\omega }_s(({\Pi }_{s,t}^{DA}\hfill \\ +{\Pi }_{s,t}^{ID}+{\Pi }_{s,t}^B)-C_s^{deg})+\chi {\displaystyle \sum _{t=1}^T}({\xi }_t-\frac{1}{1-\delta }{\displaystyle \sum _{s=1}^{|{\Omega }_{[t,T]}|}}{\omega }_s{r}_{s,t}),\hfill \end{array}$$

(6)

where ${\Pi }_{s,t}^{DA}$, ${\Pi }_{s,t}^{ID}$ and ${\Pi }_{s,t}^B$ are potential profits $\forall s,t$ from day-ahead, intra-day and real-time markets expressed as in (7), (8) and (9) correspondingly.

$$\begin{array}{c}{\Pi }_{s,t}^{DA}={\lambda }_{s,t}{\mathbb{P}}_{s,t}^{DAdch}-C{\mathbb{P}}_{s,t}^{DAdch}-{\lambda }_{s,t}{\mathbb{P}}_{s,t}^{DAch}+C{\mathbb{P}}_{s,t}^{DAch}\hfill \end{array}$$

(7)

$$\begin{array}{c}{\Pi }_{s,t}^{ID}={\lambda }_{s,t}^{sell}{\mathbb{P}}_{s,t}^{IDdch}-C{\mathbb{P}}_{s,t}^{IDdch}-{\lambda }_{s,t}^{buy}{\mathbb{P}}_{s,t}^{IDch}+C{\mathbb{P}}_{s,t}^{IDch}\hfill \end{array}$$

(8)

$$\begin{array}{c}{\Pi }_{s,t}^B={\lambda }_{s,t}^{up}{\mathbb{P}}_{s,t}^{Bdch}-C{\mathbb{P}}_{s,t}^{Bdch}-{\lambda }_{s,t}^{dn}{\mathbb{P}}_{s,t}^{Bch}+C{\mathbb{P}}_{s,t}^{Bch}.\hfill \end{array}$$

(9)

Equations (7)–(9) express the aggregator’s revenue minus cost while providing optimal discharge/charge bids in day-ahead, intra-day and real-time markets, respectively.

The equations in (10) provide the aggregated charge/discharge bids in three market places. The aggregator’s cost in all three markets while providing charging/discharging optimal bids includes the aggregator’s agreement with the EV owner set out in Equations (11)–(13). Here $C_s^{deg}$ is the degradation cost modeled as in [15] and set out in (14).

$$\begin{array}{c}{\mathbb{P}}_{s,t}^{DAdch/ch}={\displaystyle \sum _k}{p}_{s,k,t}^{DAdch/ch},{\mathbb{P}}_{s,t}^{IDdch/ch}={\displaystyle \sum _k}{p}_{s,k,t}^{IDdch/ch},\hfill \\ {\mathbb{P}}_{s,t}^{Bdch/ch}={\displaystyle \sum _k}{p}_{s,k,t}^{Bdch/ch},\hfill \end{array}$$

(10)

$$\begin{array}{c}C{\mathbb{P}}_{s,t}^{DAdch}={\displaystyle \sum _k}{c}_t\frac{p_{s,k,t}^{DAdch}}{{\eta }_k^{dch}},C{\mathbb{P}}_{s,t}^{DAch}={\displaystyle \sum _k}{c}_t{p}_{s,k,t}^{DAch}{\eta }_k^{dch},\hfill \end{array}$$

(11)

$$\begin{array}{c}C{\mathbb{P}}_{s,t}^{IDdch}={\displaystyle \sum _k}{c}_t\frac{p_{s,k,t}^{IDdch}}{{\eta }_k^{dch}},C{\mathbb{P}}_{s,t}^{IDch}={\displaystyle \sum _k}{c}_t{p}_{s,k,t}^{IDch}{\eta }_k^{dch},\hfill \end{array}$$

(12)

$$\begin{array}{c}C{\mathbb{P}}_{s,t}^{Bdch}={\displaystyle \sum _k}{c}_t\frac{p_{s,k,t}^{Bdch}}{{\eta }_k^{dch}},C{\mathbb{P}}_{s,t}^{Bch}={\displaystyle \sum _k}{c}_t{p}_{s,k,t}^{Bch}{\eta }_k^{dch},\hfill \end{array}$$

(13)

$$\begin{array}{c}\forall s;C_s^{deg}={\displaystyle \sum _k}{c}_k^{cap}\left|\frac{{\mu }_k}{100}\right|[\frac{{\sum }_t(p_{s,k,t}^{DAdch}+p_{s,k,t}^{DAch})-{\nu }_{s,k}}{{\overline{E}}_k}\hfill \\ +\frac{{\sum }_t(p_{s,k,t}^{IDdch}+p_{s,k,t}^{IDch})}{{\overline{E}}_k}+\frac{{\sum }_t(p_{s,k,t}^{Bdch}+p_{s,k,t}^{Bch})}{{\overline{E}}_k}].\hfill \end{array}$$

(14)

To derive the step-function bidding curve for hour t of the day-ahead market, the parameters ${\rho }_1$, ${\rho }_2$, ..., ${\rho }_I$ are first fixed at I arbitrary price levels. Then the unknown variables $x_1$, $x_2$, ..., $x_I$ of the step function are derived as follows:

$$\begin{array}{c}{\mathbb{P}}_{s,t}^{DAch/DAdch}={\displaystyle \sum _{l=0}^{i-1}}{x}_{i-l,t}^{DAch/DAdch}\mathrm{if}{\rho }_i\le {\lambda }_{s,t}\le {\rho }_{i+1};\forall s,t,i.\hfill \end{array}$$

(15)

Using binary variable ${\widehat{\alpha }}_{s,t,i}^{ch/dch}$ and a large enough constant ${\Gamma }_1$, (15) can be reformulated as constraints (16)–(18):

$$\begin{array}{c}{\rho }_i-{\Gamma }_1(1-{\widehat{\alpha }}_{s,t,i}^{ch/dch})\le {\lambda }_{s,t}\le {\rho }_{i+1}+{\Gamma }_1(1-{\widehat{\alpha }}_{s,t,i}^{ch/dch});\forall s,t,i\hfill \end{array}$$

(16)

$$\begin{array}{c}{\displaystyle \sum _{l=0}^{i-1}}{x}_{i-l,t}^{DAch/DAdch}-{\Gamma }_1(1-{\widehat{\alpha }}_{s,t,i}^{ch/dch})\le {\mathbb{P}}_{s,t}^{DAch/DAdch}\le {\displaystyle \sum _{l=0}^{i-1}}{x}_{i-l,t}^{DAch/DAdch}\hfill \\ +{\Gamma }_1(1-{\widehat{\alpha }}_{s,t,i}^{ch/dch});\forall s,t,i\hfill \end{array}$$

(17)

$$\begin{array}{c}{\displaystyle \sum _{i=1}^I}{\widehat{\alpha }}_{s,t,i}^{ch/dch}=1\forall s,t.\hfill \end{array}$$

(18)

The selling and buying bids to intra-day market are expressed in (19).

$$\begin{array}{c}{\mathbb{P}}_{s,t}^{IDch/IDdch}={\displaystyle \sum _{l=0}^{i-1}}{x}_{i-l,t}^{IDch/IDdch}\mathrm{if}{\rho }_i^{sell/buy}\le {\lambda }_{s,t}^{sell/buy}\le {\rho }_{i+1}^{sell/buy};\forall s,t,i.\hfill \end{array}$$

(19)

In the similar way, using binary variables ${\widehat{\alpha }}_{s,t,i}^{sell/buy}$ and a large enough constant ${\Gamma }_2$, we have:

$$\begin{array}{c}{\rho }_i^{sell/buy}-{\Gamma }_2(1-{\widehat{\alpha }}_{s,t,i}^{sell/buy})\le {\lambda }_{s,t}^{sell/buy}\le {\rho }_{i+1}^{sell/buy}+{\Gamma }_2(1-{\widehat{\alpha }}_{s,t,i}^{sell/buy});\forall s,t,i\hfill \end{array}$$

(20)

$$\begin{array}{c}{\displaystyle \sum _{l=0}^{i-1}}{x}_{i-l,t}^{IDch/IDdch}-{\Gamma }_2(1-{\widehat{\alpha }}_{s,t,i}^{sell/buy})\le {\mathbb{P}}_{s,t}^{IDch/IDdch}\le {\displaystyle \sum _{l=0}^{i-1}}{x}_{i-l,t}^{IDch/IDdch}\hfill \\ +{\Gamma }_2(1-{\widehat{\alpha }}_{s,t,i}^{sell/buy});\forall s,t,i\hfill \end{array}$$

(21)

$$\begin{array}{c}{\displaystyle \sum _{i=1}^I}{\widehat{\alpha }}_{s,t,i}^{sell/buy}=1;\forall s,t.\hfill \end{array}$$

(22)

Similarly, the up- and down-regulating bids to real-time market are expressed in (23).

$$\begin{array}{c}{\mathbb{P}}_{s,t}^{Bch/Bdch}={\displaystyle \sum _{l=0}^{i-1}}{x}_{i-l,t}^{Bch/Bdch}\mathrm{if}{\rho }_i^{up/down}\le {\lambda }_{s,t}^{up/dn}\le {\rho }_{i+1}^{up/down};\forall s,t,i.\hfill \end{array}$$

(23)

Again using binary variables ${\widehat{\alpha }}_{s,t,i}^{dn/up}$ and a large enough constant ${\Gamma }_3$ (23) can be reformulated as:

$$\begin{array}{c}{\rho }_i^{dn/up}-{\Gamma }_3(1-{\widehat{\alpha }}_{s,t,i}^{dn/up})\le {\lambda }_{s,t}^{dn/up}\le {\rho }_{i+1}^{dn/up}+{\Gamma }_3(1-{\widehat{\alpha }}_{s,t,i}^{dn/up});\forall s,t,i\hfill \end{array}$$

(24)

$$\begin{array}{c}{\displaystyle \sum _{l=0}^{i-1}}{x}_{i-l,t}^{Bch/Bdch}-{\Gamma }_3(1-{\widehat{\alpha }}_{s,t,i}^{dn/up})\le {\mathbb{P}}_{s,t}^{Bch/Bdch}\le {\displaystyle \sum _{l=0}^{i-1}}{x}_{i-l,t}^{Bch/Bdch}\hfill \\ +{\Gamma }_3(1-{\widehat{\alpha }}_{s,t,i}^{dn/up});\forall s,t,i\hfill \end{array}$$

(25)

$$\begin{array}{c}{\displaystyle \sum _{i=1}^I}{\widehat{\alpha }}_{s,t,i}^{dn/up}=1;\forall s,t.\hfill \end{array}$$

(26)

Constants ${\Gamma }_1$, ${\Gamma }_2$ and ${\Gamma }_3$ must be tuned carefully to avoid introducing extra bounds or ill-conditioning in the optimization problem.

The state of charge balance constraint $\forall s,k,t$ can be modeled in the following way [14]:

$$\begin{array}{c}So{C}_{s,k,t}=So{C}_{s,k,t-1}+[p_{k,t}^{DAch}{\eta }_k^{ch}-\frac{p_{k,t}^{DAdch}}{{\eta }_k^{dch}}+p_{s,k,t}^{IDch}{\eta }_k^{ch}-\frac{p_{s,k,t}^{IDdch}}{{\eta }_k^{dch}}+p_{s,k,t}^{Bch}{\eta }_k^{ch}\hfill \\ -\frac{p_{s,k,t}^{Bdch}}{{\eta }_k^{dch}}]A_{s,k,t}-D_{s,k,t}{\eta }_k^{dr}(1-A_{s,k,t}).\hfill \end{array}$$

(27)

Constraint (27) states that for each hour the new content of the storage is equal to its old content plus energy inflow minus energy outflow. Equation (27) allows modeling of both stationary and mobile (EV) storages. For stationary storages, $A_{s,k,t}$ availability matrix entities are always 1. For EVs the availability matrix element for hour t and scenario s is either 0 or 1 depending on whether the EV is available or on a journey. The storage level is bounded by its minimum and maximum levels (28).

$$\begin{array}{c}{{\displaystyle \underline{\gamma }}}_k^{min}{\overline{E}}_k\le So{C}_{s,k,t}\le {\overline{\gamma }}_k{\overline{E}}_k;\forall s,k,t.\hfill \end{array}$$

(28)

The constraint (29) prevents discharging/charging in the periods of unavailability, while the constraint (30) states the end SoC condition.

$$\begin{array}{c}{A}_{s,k,t}{{\displaystyle \underline{P}}}_k\le p_{s,k,t}^{DAdch}-p_{s,k,t}^{DAch}+p_{s,k,t}^{IDdch}-p_{s,k,t}^{IDch}+p_{s,k,t}^{Bdch}-p_{s,k,t}^{Bch}\le A_{s,k,t}{\overline{P}}_k;\forall s,k,t\hfill \end{array}$$

(29)

$$\begin{array}{c}So{C}_{s,k,T}\ge So{C}_k^{end};\forall s,k.\hfill \end{array}$$

(30)

Finally, constraints (31) and (32) impose the conditions of risk measure.

$$\begin{array}{c}\forall s,t;r_{s,t}\ge {\xi }_t-[({\lambda }_{s,t}{\mathbb{P}}_{s,t}^{DAdch}-C{\mathbb{P}}_{s,t}^{DAdch}-{\lambda }_{s,t}{\mathbb{P}}_{s,t}^{DAch}+C{\mathbb{P}}_{s,t}^{DAch})+({\lambda }_{s,t}^{sell}{\mathbb{P}}_{s,t}^{IDdch}\hfill \\ -C{\mathbb{P}}_{s,t}^{IDdch}-{\lambda }_{s,t}^{buy}{\mathbb{P}}_{s,t}^{IDch}+C{\mathbb{P}}_{s,t}^{IDch})+({\lambda }_{s,t}^{up}{\mathbb{P}}_{s,t}^{Bdch}-C{\mathbb{P}}_{s,t}^{Bdch}-{\lambda }_{s,t}^{dn}{\mathbb{P}}_{s,t}^{Bch}+C{\mathbb{P}}_{s,t}^{Bch})];\hfill \end{array}$$

(31)

$$\begin{array}{c}\forall s,t;r_{s,t}\ge 0.\hfill \end{array}$$

(32)

In Equation (31), $r_{s,t}$ is 0 if the profit in scenario s and time t is greater than ${\xi }_t$. Otherwise, $r_{s,t}$ is equal to the difference of ${\xi }_t$ and the corresponding profit.

With the concept of rolling planning dynamically updated, the stochastic two-stage optimal bidding strategy for deriving the coordinated bidding curves in day-ahead, intra-day and real-time markets is defined in Equations (33) and (34).

$$\begin{array}{c}{\mathrm{Max}.}_{\Phi }{\displaystyle \sum _{t=1}^T}{\displaystyle \sum _{s=1}^{|{\Omega }_{[t,T]}|}}{\omega }_s(({\Pi }_{s,t}^{DA}+{\Pi }_{s,t}^{ID}+{\Pi }_{s,t}^B)-C_s^{deg})\hfill \\ +\chi {\displaystyle \sum _{t=1}^T}({\xi }_t-\frac{1}{1-\delta }{\displaystyle \sum _{s=1}^{|{\Omega }_{[t,T]}|}}{\omega }_s{r}_{s,t})\hfill \end{array}$$

(33)

$$\begin{array}{c}\mathrm{subject}\mathrm{to}:(7)-(14),(10)-(13),(16)-(18),(20)-(22)\hfill \\ (24)-(26),(27)-(30),(31)-(32),\hfill \end{array}$$

(34)

where $\Phi$ = $\{p_{s,k,t}^{DAdch/DAch},p_{s,k,t}^{IDdch/IDch},p_{s,k,t}^{Bdch/Bch},{\mathbb{P}}_{s,t}^{DAdch/DAch},{\mathbb{P}}_{s,t}^{IDdch/IDch},{\mathbb{P}}_{s,t}^{Bdch/Bch},C{\mathbb{P}}_{s,t}^{DAdch/DAch},$ $C{\mathbb{P}}_{s,t}^{IDdch/IDch},C{\mathbb{P}}_{s,t}^{Bdch/Bch},x_{i,t}^{DAdch/DAch},x_{i,t}^{IDdch/IDch},x_{i,t}^{Bdch/Bch},So{C}_{s,k,t},$ ${\widehat{\alpha }}_{s,t,i},{\widehat{\alpha }}_{s,t,i}^{sell/buy}{\widehat{\alpha }}_{s,t,i}^{up/dn}\}$ is the set of decision variables.

The whole framework of deriving the optimal coordinated bids for three-settlement markets is presented in Figure 3.

5. Case Study

The developed two-stage SD-MILP optimal bidding strategy summarized in Equations (33) and (34) is applied to derive a bidding discharge/charge curve of an aggregator managing a fleet of 1000 EVs. Both charging and discharging modes are studied.

5.1. Market Price Series

The historical price data for both day-ahead and real-time markets are taken from the Nordic electricity market website [26]. The CSV excel files, which contain the information about intra-day market trading activities, are obtained from the Nordic electricity market.

5.2. General Parameters

The EV driving patterns are according to the reference [28]. The other general parameters are set out in

Table 1. General input parameters. Par.: parameter, Ref.: reference.

Par.	Value	Units	Ref.	Par.	Value	Units	Ref.
$\overline{E}$	50	KWh	[28]	${\eta }_k^{ch}$	90%		[29]
${\overline{\gamma }}_k$	20 %		[30]	${\eta }_k^{dch}$	93%		[29]
${{\displaystyle \underline{\gamma }}}_k$	100%		[30]	$So{C}_{k,}^{end}$	60%
${\overline{P}}_k$	6	KW	[28]	$c_k^{cap}$	200	€	[15]
${{\displaystyle \underline{P}}}_k$	6	KW	[28]	${\mu }_k$	−[0.0013]		[15]

.

5.3. Simulation Results

5.3.1. Tractability of the Solution

The proposed Markov-based HW model, scenario backward reduction-algorithm and the Monte Carlo simulation tool to produce mobility patterns are coded in Matlab.

To ensure computational tractability while solving stochastic models, it is necessary (1) to approximate the underlying probability distribution by a discrete probability measure and (2) to reduce the number of scenarios and thereby the computational complexity.

Scenario reduction should be done in a way that the statistical information in prices is maintained in the best possible way [27]. The backward-reduction algorithm is used to reduce the number of price scenarios. Using the forecasted prices, 1000 price scenarios with equal probabilities for three market floors are generated and they are reduced to 30 price scenarios.

In addition, 30 equally probable mobility scenarios are produced using the Monte Carlo simulation tool and are integrated with the 30 preserved day-ahead, intra-day and real-time price scenarios. Together with mobility scenarios these preserved price scenarios are used for calculating the optimal bidding curves of the EV aggregator.

5.3.2. Rolling Planning Horizon

A three-level step function is considered for bidding curves. The two-stage SD-MILP model is coded in GAMS platform and solved using CPLEX solver. All optimization problems are solved with an optimality gap of $0\%$. The whole simulation is run on a computer with 2.66 GHz processor and 4 GB RAM. The objective function values together with the computation time for a fleet of 1000 EVs and all iterations are stated in

Table 2. Model solution report for a fleet of 1000 EVs; $\chi$ = 0.01, It.: itertaion.

	It. 1	It. 2	It. 3	It. 4
$\mathbb{E}[{\Pi }^{Tot}]$ (€)	110.31	101.87	77.1	76.26
Comp. time (second)	68.34	58.3	59.56	63.76

. According to Table 2 the computation time is very short.

In addition, the model statistics are set out in

Table 3. Model statistics for different cases.

Number of EVs	10	100	500	1000	5000
Continuous variables	73,177	526,777	2,542,777	5,062,777	25,150,802
Discrete variables	12,960	12,960	12,960	12,960	12,960
Constraints	115,921	569,521	2,585,521	5,105,521	25,205,521
Comp. time (second)	0.78	5.3	40.25	73.76	387.84

when the suggested two-stage SD-MILP model is simulated with different number of EVs.

Figure 4 depicts the computation time as a function of the number of EVs involved in the simulation. The observations of Figure 4 show that the model solution time increases linearly when the number of involved EVs increases. Furthermore, the simulation of the developed SD-MILP model with 5000 EVs requires about 6.5 min (see Table 3), thus the model is applicable for simulating a larger case with a greater number of cars or a shorter dispatch interval.

The optimal coordinated bids of the aggregator in three sequential markets are set out in Figure 5, Figure 6 and Figure 7. Figure 5 illustrates that the charge/discharge bid volumes to the day-ahead market remain the same for all iterations. In contrast, Figure 6 and Figure 7 show that intra-day and real-time bid volumes (selling/buying, up/down regulation) change when time evolves and new price information is revealed over time. According to Figure 5, Figure 6 and Figure 7, the EV aggregator actively participates in day-ahead, intra-day and real-time market floors offering discharging bids and participates in real-time market providing down-regulation bids. Moreover, the two-stage SD-MILP offers discharging bids in all three markets when the realized market prices are in the second interval of the three-level price curve (at least €50) and offers charging bids when the realized market prices are cheap (when the prices belong to the first interval of the three-level price curve).

Furthermore the increasing level of intermittent wind and solar power production leads to higher volatility in day-ahead, intra-day, and real-time prices. The impact of increased volatility of market prices on trading of an EV aggregator in these market places can be analyzed using the methodology developed in this paper. To generate market price scenarios with higher volatility, the standard deviation of the price is doubled (the mean remaining the same). The energy tradings in different market places under base prices and prices with higher volatility are reported in

Table 4. Day-ahead, intra-day and real-time market trading with increased volatility of market prices; $\chi$ = 0.

	Base Prices	Volatile Prices	Difference in %
Day-ahead trading (€)	183	9	95% decrease
Intra-day trading (€)	64	413	85% increase
Real-time trading (€)	1924	1983	3% increase

. Table 4 indicates that continuous increases in the share of RES in the power system results in highly volatile market prices, which in turn makes the intra-day market place very attractive for energy trading for an EV aggregator with a flexible energy source.

5.3.3. Controlling the Risk Measure T-CVaR

The T-CVaR risk measure is introduced and modeled in the two-stage SD-MILP (33) and (34), which aims at optimizing the lower tail of profit distribution in each hour t.

To see the sensitivity of the model to risk, the two-stage SD-MILP is run with different risk levels. The lower bound of the $\chi$ constant is zero and describes a risk-neutral aggregator. The upper bound of the $\chi$ constant highly depends on the risk-averseness level of the aggregator. The trade-off between the expected profit and T-CVaR for different values of $\chi$ is shown in Figure 8. In this figure, by increasing the value of $\chi$, the expected profit decreases while the T-CVaR value increases.

Figure 8 shows that the expected profit for a risk neutral producer ($\chi =0$) is €118 with a T-CVaR of −€2027. However the expected profit for a risk-averse producer with $\chi =0.09$ is only €57 with a T-CVaR value of −€613.

Assuming no bi-lateral contracts and futures markets, the EV aggregator’s risk management applies only the volume traded in day-ahead, intra-day and real-time markets. For chosen $\chi$ levels, day-ahead, intra-day and real-time market bids for the 24 h of the planning period are depicted in Figure 9, Figure 10 and Figure 11. Figure 9 (upper part) shows that the increase in the risk-aversion level causes a volume shift in day-ahead discharging bids for hours 6, 15 and 21. When $\chi =0.01$ the bid volume for hour 6 and hour 15 is 6 MWh, while at the other end of the spectrum, for $\chi =1$, the bid is reduced to 0 MWh. For hour 21 the day-ahead discharging bid volume decreases from 6 MWh to 4.8 MWh. The day-ahead market charging bids are not affected while giving more weight of the hourly risk measure (see lower part of Figure 9). Accordingly, intra-day market discharging bids for hour 14 and hour 24 and the charging bid for hour 15 are sensitive towards hourly risk measure. Figure 10 illustrates that an increase in the risk aversion level decreases the discharge bid volume for hour 14 whereas it increases the discharge bid volume for hour 24 (upper figure) and the charge bid volume for hour 15 (lower figure). Finally, real-time market discharging bid volumes for hour 3 and hour 22 are the most affected ones. The increase in the risk-aversion level decreases the bid volume for both hours Figure 11 (upper figure). Again the real-time market charging bids are not affected by the inclusion of the risk factor.

Studies have been carried out by incorporating CVaR risk measure in the two-stage SD-MILP (33) and (34) optimization model. The simulated model calculates daily conditional values at risk as −€325. Consequently, an hourly conditional value at risk is $\frac{-325}{19}$ = −€17 (considering only available hours) for the EV aggregator.

Table 5. The hourly hourly conditional value at risk (T-CVaR) values; $\chi =0.1$.

Hours	T-CVaR (€)	Hours	T-CVaR (€)	Hours	T-CVaR (€)
1	$-56.6$	9	0	17	0
2	$-21.2$	10	$-37.86$	18	0
3	$-16.4$ *	11	$-52.4$	19	$-12.3$ *
4	$-21.6$	12	$-64.8$	20	$-29$
5	$-33.3$	13	$-26.3$	21	$-42$
6	$-34$	14	$-8.8$ *	22	$-23.6$
7	0	15	$-16$ *	23	$-18.5$
8	0	16	$-74.8$	24	$-23$

summarizes hourly T-CVaR values when T-CVaR risk measure is applied.

According to Table 5, only hours 3, 14, 15, and 19 have an hourly CVaR higher than −€17 and for the rest of the 15 h (when EVs are available), the hourly CVaR is less than the expected value of −€17 from the standard CVaR model. It is apparent that the proposed T-CVaR provides the flexibility to the risk-averse EV aggregator to optimize the lower tails of the aggregator’s hourly profits. Another advantage of the T-CVaR model as compared to the CVaR is that the aggregator can assign more weight to profit risk in a specific hour and thus improve the T-CVaR model, (33) and (34).

6. Conclusions

This paper proposes a two-stage SD-MILP to prepare optimal coordinated bids for day-ahead, intra-day, and real-time markets for a profit-maximizing and risk-averse EV aggregator. The main advantage of the developed SD-MILP model as compared to other bidding strategies is that the model includes intra-day market formulation and provides a potentially attractive market place for an EV aggregator for energy trading. The prices in different market places are forecasted with the proposed Markov-based HW model. The risk of having low profits is handled by incorporation of the hourly conditional value at risk (T-CVaR) into the SD-MILP model. The scenario tree is updated with the released information over time concerning intra-day and real-time market prices by including the rolling planning in the two-stage SD-MILP model. The developed procedure is tested using a fleet of 1000 EVs. Different studies have been carried out and the numerical results discussed. The model statistics and computation time are reported simulating the model with 5000 EVs.

For future work, network constraints can be incorporated into the stochastic model, to simulate and analyze the potential network conditions, study remedies of failures, and investigate the impact of physical and other factors on cost imbalance.