Scenario Selection for Iterative Stochastic Transmission Expansion Planning

Abstract

Reliable transmission expansion planning is critical to power systems’ development. To make reliable and sustainable transmission expansion plans, numerous sources of uncertainty including demand, generation capacity, and fuel cost must be taken into consideration in both spatial and temporal dimensions. This paper presents a new approach to selecting a small number of high-quality scenarios for transmission expansion. The Kantorovich distance of social welfare distributions was used to assess the quality of the selected scenarios. A case study was conducted on a power system model that represents the U.S. Eastern and Western Interconnections, and ten high-quality scenarios out of a total of one million were selected for two transmission plans. Results suggested that scenarios selected using the proposed algorithm were able to provide a much more accurate estimation of the value of transmission plans than other scenario selection algorithms in the literature.

1. Introduction

The integrated electric utility, owning both generation and transmission, traditionally develops generation plans first and then plans the transmission necessary to support those generation plans, an approach consistent with the fact that generation investments are normally far more costly than the needed transmission investments. Today, however, there are organizations that own transmission only. There are also service companies, called regional transmission organizations in the U.S., that own no infrastructure, but are responsible for coordinating the planning function. Both of these organizations must solve the transmission expansion planning (TEP) problem under uncertainty regarding technology, quantity, location, and timing of generation investments [1,2,3,4]. Robust and stochastic optimization approaches are two major modeling techniques for transmission planning under uncertainty, which have their own strengths and limitations.

Robust TEP maximizes the performance of the power system under the worst case scenario, which is dependent on the transmission plan [5,6,7,8,9,10]. Most solution techniques for robust TEP consist of two iterative steps: one step identifies the worst case scenario for a proposed transmission plan, whereas the other proposes the most robust transmission plan against a pool of worst case scenarios. A major advantage of robust TEP is its robustness against worst case scenarios, which is especially desirable for enhancing long-term resiliency of power system infrastructures. The iterative algorithm also allows the model to efficiently identify worst case scenarios from an enormous solution space with a large number of (or even infinitely many) possible scenarios. On the other hand, this approach often tends to be over-conservative by focusing on the possibility and ignoring the probability of worst case scenarios.

Stochastic TEP addresses uncertainty with a very different philosophy. It maximizes the average performance of the power system under all scenarios, weighted by their probabilities of occurrence. As such, the optimal stochastic transmission plan finds a good balance between the (positive or negative) impact and the likelihood of all scenarios [11,12,13,14,15,16]. However, this approach also faces a dilemma. On the one hand, due to the numerous sources of uncertainty and long planning horizon, a large number of scenarios is necessary to realistically represent the complexity and uncertainty of the TEP problem. On the other hand, however, the computation time of most solution techniques is very sensitive to the number of scenarios, and many algorithms become intractable for even a few dozen scenarios. To address this dilemma, scenario generation and reduction techniques have been proposed, which attempt to identify a small set of high-quality scenarios that represent the whole set of scenarios [17,18,19,20,21].

An alternative approach is to use an iterative stochastic TEP model, which attempts to combine the advantages of robust and stochastic TEP approaches and to overcome their limitations. A major limitation of the stochastic TEP approach with scenario generation and reduction is the assumption that the set of selected scenarios will be a good representation of the whole set of scenarios for all transmission plans. This is analogous to identifying one scenario as the “worst case scenario” for all transmission plans in the robust TEP approach. The iterative model in robust TEP acknowledges the need for re-identifying a worst case scenario for each new transmission plan, until the pool of worst case scenarios is sufficiently inclusive of all worst case scenarios. Similarly, the iterative stochastic TEP model re-identifies a new set of high-quality scenarios for each new transmission plan until the pool of high-quality scenarios is sufficiently representative of the whole set of scenarios. A formal definition of high-quality scenarios is given in Section 2.2. Figure 1 highlights the conceptual differences among robust TEP, stochastic TEP, and the iterative stochastic TEP. The scope of this paper is the highlighted component for selecting high-quality scenarios.

Scenario generation and reduction has been a topic of great interest in the power systems literature. Most existing methods use clustering [19,22,23,24] or sampling methods to reduce the number of scenarios from a randomly generated initial set. In a recent review article, Park et al. [25] compared four methods for scenario reduction using a two-stage stochastic transmission planning model, including random sampling, importance sampling [26], the distance-based method [17,27,28], iterative scenario reduction approaches, and stratified scenario sampling [25]. They used these methods to reduce a whole set of 20 scenarios to smaller subsets and compared their pros and cons. Other methods also include extended and improved initial-center-refined and weighted k-means [21], data-driven [29], forward selection [20], moment-based [18,30,31], and objective-based [32,33] approaches.

The proposed scenario selection approach in this paper differs from previous methods for scenario generation and reduction in three major ways. First, we generate scenarios with explicit consideration of temporal and spatial correlations, including generation investment and retirement decisions in response to demand, fuel cost, and transmission capacity. Second, we use the Kantorovich distance of social welfare distributions to assess the quality of the selected scenarios. Third, a different subset of high-quality scenarios is selected for each transmission plan candidate. In comparison, most existing methods in the literature ignore the correlations among the scenarios, select a subset of scenarios based on their similarity rather than their implications on social welfare, and use the same subset of scenarios for all transmission plans.

2. Model Formulation

In this section, we describe and explain the models that we propose to identify a small set of high-quality scenarios for a given transmission plan. In the following, the variables and parameters used in the model are described.

Sets and indices:

$\mathcal{Y}$	Set of planning years;
$\mathcal{T}$	Set of load blocks;
$\mathcal{B}$	Set of buses;
$\mathcal{L}$	Set of transmission lines;
$\mathcal{H}$	Set of generation technologies;
${\mathcal{G}}^{\mathrm{ER}}(b,y)$	Set of existing renewable generators at bus b in year y;
${\mathcal{G}}^{\mathrm{EN}}(b,y)$	Set of existing non-renewable generators at bus b in year y;
${\mathcal{G}}^{\mathrm{E}}(b,y)$	Set of existing generators at bus b in year y: ${\mathcal{G}}^{\mathrm{E}}(b,y)={\mathcal{G}}^{\mathrm{ER}}(b,y)\cup {\mathcal{G}}^{\mathrm{EN}}(b,y)$;
${\mathcal{G}}^{\mathrm{CR}}(b,y)$	Set of candidate renewable generators at bus b in year y;
${\mathcal{G}}^{\mathrm{CN}}(b,y)$	Set of candidate non-renewable generators at bus b in year y;
${\mathcal{G}}^{\mathrm{C}}(b,y)$	Set of candidate generators at bus b in year y: ${\mathcal{G}}^{\mathrm{C}}(b,y)={\mathcal{G}}^{\mathrm{CR}}(b,y)\cup {\mathcal{G}}^{\mathrm{CN}}(b,y)$;
${\mathcal{G}}^{\mathrm{R}}(b,y)$	Set of renewable generators at bus b in year y: ${\mathcal{G}}^{\mathrm{R}}(b,y)$ $={\mathcal{G}}^{\mathrm{ER}}(b,y)\cup {\mathcal{G}}^{\mathrm{CR}}(b,y)$;
$\mathcal{G}(b,y)$	Set of existing and candidate generators at bus b in year y: $\mathcal{G}(b,y)={\mathcal{G}}^{\mathrm{E}}(b,y)\cup {\mathcal{G}}^{\mathrm{C}}(b,y)$;
${\mathcal{G}}_h^{\mathrm{E}}(b,y)$	Set of existing generators of technology h at bus b in year y.

Parameters:

r	Discount rate, in %;
$C^{\mathrm{LS}}$	Cost of load-shedding, in $/MWh;
$T_t$	Operating duration of load block t, in hours;
$H_g$	Heat rate for generator g, in Mbtu/MWh;
$F_{g,y}^{\mathrm{C}}$	Fuel cost of generator g’s technology in year y, in $/Mbtu;
$C_g^{\mathrm{OM}}$	Fixed Operation and Maintenance (O&M) cost for generator g’s technology, in $/MW;
$C_g^{\mathrm{I}}$	Amortized annual investment cost for generator g’s technology, in $/MW;
$C_g^{\mathrm{V}}$	Variable O&M cost for generator g’s technology, in $/MWh;
$C_{g,y}^{\mathrm{V}}$	Variable generation cost for generator g in year y: $C_{g,y}^{\mathrm{V}}=H_g{F}_{g,y}^{\mathrm{C}}+C_g^{\mathrm{V}}$, in $/MWh;
$D_{b,t,y}$	Demand at bus b for load block t in year y, in MW;
$V_{b,t,y}$	Monetary value of energy consumption at bus b for load block t in year y, in $/MWh;
$B_{i,j}$	Susceptance of transmission line $(i,j)$, in Siemens;
$F_{i,j,y}^{\max }$	Capacity of transmission line $(i,j)$ in year y, in MW;
$C_{g,t}$	Capacity factor of generator g for load block t, in %;
$P_g^{\max }$	Production capacity of generator g, in MW;
${\alpha }_y$	Renewable portfolio requirement in year y, in %.

Decision variables:

$p_{g,b,t,y}$	Power production from generator g at bus b for load block t in year y, in MW;
$f_{i,j,t,y}$	Power flow through transmission line $(i,j)$ for load block t in year y, in MW;
$d_{b,t,y}^{\mathrm{LS}}$	Load shedding at bus b for load block t in year y, in MW;
${\theta }_{b,t,y}$	Voltage angle at bus b for load block t in year y, in radians;
$p_{b,t,y}^{\mathrm{LMP}}$	Locational marginal price at bus b for load block t in year y, in $/MWh;
$\mathsf{\Psi }$	Social welfare of energy producers and consumers in the power system, in $;
$x_{g,b,y}$	Binary variable indicating whether the generator g at bus b exists in year y ($x_{g,b,y}=1$) or not ($x_{g,b,y}=0$). For an existing generator $g\in {\mathcal{G}}^{\mathrm{E}}(b,y)$, $x_{g,b,y}=0$ means the generator has retired; and for a candidate generator $g\in {\mathcal{G}}^{\mathrm{C}}(b,y)$, $x_{g,b,y}=1$ means the generator has been added to the existing generation capacity.

2.1. Motivating Example

Suppose there are 10 scenarios $\{s_1,\dots ,s_{10}\}$ and 10 transmission plans $\{p_1,\dots ,p_{10}\}$, and we try to identify the optimal transmission plan. The number of scenarios and plans is limited to 10 for illustration purposes, but the methods are applicable to situations where there are millions or even infinitely many scenarios and plans. With the values in the table representing social welfare of the transmission plans under different scenarios,

Table 1. Illustrative example of robust TEP. Numerical values represent social welfare of 10 transmission plans under 10 scenarios. For each transmission plan, the social welfare under the worst case scenario is highlighted.

	$s_1$	$s_2$	$s_3$	$s_4$	$s_5$	$s_6$	$s_7$	$s_8$	$s_9$	$s_{10}$
$p_1$	12	46	50	27	33	46	44	28	41	29
$p_2$	21	10	14	37	46	25	42	30	22	27
$p_3$	46	36	29	24	23	30	45	33	37	11
$p_4$	34	49	20	25	21	36	23	14	30	27
$p_5$	39	48	18	33	15	42	22	28	28	36
$p_6$	36	13	36	27	30	33	48	40	43	29
$p_7$	14	45	16	37	28	30	36	24	47	49
$p_8$	24	11	32	36	26	21	15	40	43	33
$p_9$	18	22	27	19	40	35	29	23	50	41
$p_{10}$	38	16	20	46	30	19	31	43	24	40

illustrates the solution process of the robust TEP approach.

Table 2. Illustrative example of stochastic TEP. Numerical values represent social welfare of 10 transmission plans under 10 scenarios. For each transmission plan, we used the scenario selection algorithm in Section 3.3 to select two high-quality scenarios, whose social welfare values are highlighted in green.

	$s_1$	$s_2$	$s_3$	$s_4$	$s_5$	$s_6$	$s_7$	$s_8$	$s_9$	$s_{10}$
$p_1$	12	46	50	27	33	46	44	28	41	29
$p_2$	21	10	14	37	46	25	42	30	22	27
$p_3$	46	36	29	24	23	30	45	33	37	11
$p_4$	34	49	20	25	21	36	23	14	30	27
$p_5$	39	48	18	33	15	42	22	28	28	36
$p_6$	36	13	36	27	30	33	48	40	43	29
$p_7$	14	45	16	37	28	30	36	24	47	49
$p_8$	24	11	32	36	26	21	15	40	43	33
$p_9$	18	22	27	19	40	35	29	23	50	41
$p_{10}$	38	16	20	46	30	19	31	43	24	40

illustrates the solution process of the iterative stochastic TEP approach for the same example, in which two high-quality scenarios were selected to represent the whole set of 10 scenarios for each transmission plan. The algorithm proposed in Section 3.3 was used to select these scenarios and assign their probabilities.

The optimal transmission plan for robust TEP approach is determined in

Table 3. Optimal solutions using robust TEP under the worst case scenario, stochastic TEP with all 10 scenarios, and stochastic TEP under 2 high-quality scenarios. Assuming equal likelihood for the 10 scenarios, we calculate the probabilities of the two high-quality ones as the percentages of scenarios that are better represented by the scenarios. Take plan $p_2$ for example; $s_7$ represents three higher welfare scenarios ($s_4$, $s_5$, and $s_7$ itself), and $s_9$ represents the other seven lower welfare scenarios, so 0.3 and 0.7 are used for their respective probabilities for the forth column in the table.

Transmission Plan	Robust (Worst Scenario)	Stochastic (All Scenarios)	Stochastic (High-Quality Scenarios)
$p_1$	12	35.6	46 × 0.5 + 28 × 0.5 = 37.0
$p_2$	10	27.4	42 × 0.3 + 22 × 0.7 = 28.0
$p_3$	11	31.4	24 × 0.5 + 37 × 0.5 = 30.5
$p_4$	14	27.9	34 × 0.4 + 21 × 0.6 = 26.2
$p_5$	15	30.9	39 × 0.5 + 22 × 0.5 = 30.5
$p_6$	13	33.5	30 × 0.8 + 48 × 0.2 = 33.6
$p_7$	14	32.6	45 × 0.5 + 24 × 0.5 = 34.5
$p_8$	11	28.1	36 × 0.5 + 21 × 0.5 = 28.5
$p_9$	18	30.4	22 × 0.6 + 40×0.4 = 29.2
$p_{10}$	16	30.7	38 × 0.6 + 20 × 0.4 = 30.8

as $p_9$, which resulted in a higher social welfare than all other transmission plans under their respective worst case scenarios. For iterative stochastic TEP approach, the optimal transmission plan is determined in Table 3 as $p_1$, which maximized the weighted average social welfare. The third column of Table 3 calculates the average performance of all transmission plans under all scenarios, which shows that the true optimal solution to the stochastic TEP problem was indeed $p_1$. This example illustrated how the iterative stochastic TEP approach was able to identify the optimal (or close to optimal) solution by selecting a subset of high-quality scenarios chosen specific to each transmission plan.

2.2. Definition of High-Quality Scenarios

For a given transmission plan, a scenario includes three elements: demand for all buses, fuel cost for all generation technologies, and generation capacity for all generators, throughout the entire planning horizon. Such a definition of a scenario captures the temporal and spatial trajectory of the power system, which we view as evolving as a function of a given transmission plan.

High-quality scenarios selected using our proposed approach satisfy two requirements. First, the correlations between generation capacity and other elements (i.e., demand and fuel cost) are reflected. This is because generation investments and retirement decisions are made by decentralized and for-profit generation companies, which have been found to be sensitive to demand and fuel cost. Second, the probabilistic distance between the distribution of social welfare resulting from high-quality scenarios and that from all scenarios is small (the smaller the distance, the higher the quality).

2.3. Estimation of Generation Investment and Retirement

For given demand and fuel cost elements of a scenario, we propose to estimate the generation capacity element of the scenario by solving a simplified generation expansion planning (GEP) problem, which has the following features.

• We cast the GEP problem as a bilevel optimization model, in which the upper level maximizes the net present value of the investment by determining the investment of new generation capacity and the retirement of existing generators, whereas the lower level computes the optimal power flow (used by the upper level to calculate the revenue of power generation) as a result of the GEP decisions from the upper level. Similar models have been used in other studies such as [7,34].
• It uses a single upper level decision maker for all generation investment and retirement decisions in all buses. This is a simplifying approximation of the power systems, in which investment and retirement decisions are made by multiple generation companies in a decentralized manner. Similar approximations have been used in other studies such as [18].
• The GEP model identifies generation investments based on profit maximization accounting for fixed and variable operational costs, investment costs, and load shedding costs, subject to renewable portfolio standards and generation adequacy requirements, attributes common to other studies [18,35,36,37,38].

The bilevel optimization model of the GEP problem is formulated as the following optimization model in Equations (1)–(4), where the upper level decision variable x represents generation investment and retirement decisions $x_{g,b,y},\forall g,b,y$, and the lower level decision variable p represents both dispatch decisions $p_{g,b,t,y},\forall g,b,t,y$ and locational marginal prices $p_{b,t,y}^{\mathrm{LMP}},\forall b,t,y$.

$$\begin{array}{ccc}\underset{x,p}{max}\hfill & \hfill f^{\mathrm{GEP}}(x,p)& \hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& A_{1}x\le b_1\end{array}$$

(2)

$$\begin{array}{cc} x\mathrm{binary}.& \hfill \end{array}$$

(3)

$$\begin{array}{ccc}\hfill & \hfill p\in argmin\{d_2^{\top }p:A_{2}x+B_{2}p\le b_2,p\ge 0\}.& \hfill \end{array}$$

(4)

The upper level objective function $f^{\mathrm{GEP}}(x,p)$ is to maximize the profit of generation companies, which can be estimated as:

$$\begin{array}{c}\hfill f^{\mathrm{GEP}}(x,p)=\sum _{\begin{array}{c}g\in \mathcal{G}(b,y),b\in \mathcal{B},\\ t\in \mathcal{T},y\in \mathcal{Y}\end{array}}\frac{p_{g,b,t,y}max\{p_{b,t,y}^{\mathrm{LMP}}-C_{g,y}^{\mathrm{V}},0\}{T}_t}{{\left(1+r\right)}^y}{x}_{g,b,y}\\ \hfill -\sum _{\begin{array}{c}g\in \mathcal{G}(b,y),\\ b\in \mathcal{B},y\in \mathcal{Y}\end{array}}\frac{C_h^{\mathrm{OM}}{P}_g^{\max }}{{\left(1+r\right)}^y}{x}_{g,b,y}-\sum _{\begin{array}{c}g\in {\mathcal{G}}^{\mathrm{C}}(b,y),\\ b\in \mathcal{B},y\in \mathcal{Y}\end{array}}\frac{C_h^{\mathrm{I}}{P}_g^{\max }}{{\left(1+r\right)}^y}{x}_{g,b,y}.\end{array}$$

(5)

The first term in the objective function is the estimated revenue from selling power generation at the locational marginal prices. The second term in the objective function is the O&M cost for existing and new generators, and the third term is the investment cost for new generators. All future cost terms are discounted to the current year to calculate the net present value. All fixed and variable investment costs are amortized so that investments towards the end of planning horizon would not be disincentivized, since only part of the investment cost appropriated for the planning horizon is calculated to offset the benefit of the investment.

The upper level constraints in Equations (2) and (3) are defined as follows:

$$\begin{array}{cc}\hfill & x_{g,b,y}\ge x_{g,b,y+1},\forall g\in {\mathcal{G}}^{\mathrm{E}}(b,y),b\in \mathcal{B},y\in \mathcal{Y}\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill & x_{g,b,y}\le x_{g,b,y+1},\forall g\in {\mathcal{G}}^{\mathrm{C}}(b,y),b\in \mathcal{B},y\in \mathcal{Y}\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill & \sum _{\begin{array}{c}g\in {\mathcal{G}}^{\mathrm{R}}(b,y),\\ b\in \mathcal{B}\end{array}}{P}_g^{\max }{x}_{g,b,y}\ge {\alpha }_y\sum _{\begin{array}{c}g\in \mathcal{G}(b,y),\\ b\in \mathcal{B}\end{array}}{P}_g^{\max }{x}_{g,b,y},\forall y\in \mathcal{Y}\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill & \sum _{\begin{array}{c}g\in \mathcal{G}(b,y),\\ b\in \mathcal{B}\end{array}}{P}_g^{\max }{x}_{g,b,y}\ge 1.2\sum _{b\in \mathcal{B}}\underset{t\in \mathcal{T}}{max}{D}_{b,t,y},\forall y\in \mathcal{Y}\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill & x_{g,b,y}\in \{0,1\},\forall g\in \mathcal{G}(b,y),b\in \mathcal{B},y\in \mathcal{Y}.\hfill \end{array}$$

(10)

Here, Constraints in Equations (6) and (7) allow the retirement of existing generators and investment of new generators; Constraint in Equation (8) imposes a renewable portfolio standard type of requirement on the minimal percentage of generation capacity being renewable; Constraint in Equation (9) imposes a generation adequacy requirement, where the total generation capacity for each year must exceed 120% of predicted peak demand; and Constraint in Equation (10) is the definition of binary decision variables.

Constraint in Equation (4) defines the lower level problem, which takes generation investment and retirement decisions from the upper level as input and solves the optimal power flow problem to determine power dispatches and locational marginal prices throughout the planning horizon. This lower level problem can be defined as follows, which needs to be solved for all $t\in \mathcal{T}$ and $y\in \mathcal{Y}$.

$$\begin{array}{cc}\hfill min& \sum _{\begin{array}{c}g\in {\mathcal{G}}^{\mathrm{E}}(b,y)\\ b\in \mathcal{B}\end{array}}{C}_{g,y}^{\mathrm{V}}{p}_{g,b,t,y}+\sum _{b\in \mathcal{B}}{C}^{\mathrm{LS}}{d}_{b,t,y}^{\mathrm{LS}}\\ \hfill \mathrm{s}.\mathrm{t}.& {\sum }_{g\in {\mathcal{G}}^{\mathrm{E}}(b,y)}{p}_{g,b,t,y}+{\sum }_{i\in \mathcal{B}}{f}_{i,b,t,y}& \hfill \end{array}$$

(11)

$$\begin{array}{cc} -{\sum }_{j\in \mathcal{B}}{f}_{b,j,t,y}=D_{b,t,y}-d_{b,t,y}^{\mathrm{LS}},& \forall b\in \mathcal{B}\hfill \end{array}$$

(12)

$$\begin{array}{ccc}\hfill & 0\le d_{b,t,y}^{\mathrm{LS}}\le D_{b,t,y},& \forall b\in \mathcal{B}\hfill \end{array}$$

(13)

$$\begin{array}{ccc}\hfill & f_{i,j,t,y}=B_{i,j}({\theta }_{i,t,y}-{\theta }_{j,t,y}),& \forall (i,j)\in \mathcal{L}\hfill \end{array}$$

(14)

$$\begin{array}{ccc}\hfill & -\pi \le {\theta }_{b,t,y}\le \pi ,& \forall b\in \mathcal{B}\hfill \end{array}$$

(15)

$$\begin{array}{ccc}\hfill & -F_{i,j,y}^{\max }\le f_{i,j,t,y}\le F_{i,j,y}^{\max },& \forall (i,j)\in \mathcal{L}\hfill \end{array}$$

(16)

$$\begin{array}{ccc}\hfill & 0\le p_{g,b,t,y}\le C_{g,t}{P}_g^{\max }{x}_{g,b,y},& \forall g\in {\mathcal{G}}^{\mathrm{E}}(b,y),b\in \mathcal{B}.\hfill \end{array}$$

(17)

The lower level objective function in Equation (11) is to minimize the production cost and load shedding cost. Constraint in Equation (12) enforces Kirchhoff’s current law, which requires nodal balance of power generation, in-flow, out-flow, demand, and load shedding; Constraint in Equation (13) defines the lower and upper bounds of load shedding; Constraint in Equation (14) enforces Kirchhoff’s voltage law, which computes power flows through transmission lines based on voltage angles and susceptance of the power network; Constraint in Equation (15) defines the bounds of voltage angles; Constraint in Equation (16) limits the power flow within transmission capacity; and Constraint in Equation (17) limits the power production within existing generators’ capacity, which depends on generation investment and retirement decisions.

2.4. Definition of Social Welfare

Social welfare is a commonly used objective for transmission planning [39,40,41,42,43,44,45]. We measure the quality of selected scenarios based on the similarity between social welfare distributions resulting from the selected and whole set of scenarios, rather than the similarity of the elements of the scenarios. For a given transmission plan under a given scenario, the social welfare ($\mathsf{\Psi }$) is defined as the summation of producers’ surplus and consumers’ surplus:

$$\begin{array}{c}\hfill \mathsf{\Psi }=f^{\mathrm{GEP}}(x,p)+\sum _{\begin{array}{c}b\in \mathcal{B},t\in \mathcal{T},\\ y\in \mathcal{Y}\end{array}}\frac{(V_{b,t,y}-p_{b,t,y}^{\mathrm{LMP}})D_{b,t,y}{T}_t}{{\left(1+r\right)}^y}-\sum _{\begin{array}{c}b\in \mathcal{B},t\in \mathcal{T},\\ y\in \mathcal{Y}\end{array}}\frac{(C^{\mathrm{LS}}+V_{b,t,y}-p_{b,t,y}^{\mathrm{LMP}})d_{b,t,y}^{\mathrm{LS}}{T}_t}{{\left(1+r\right)}^y}.\end{array}$$

(18)

Here, the first term is the objective function of the GEP model, which represents the producers’ surplus; the second and third terms are consumers’ surplus, which includes monetary valuation of energy consumption less prices of energy and economic cost of load shedding. All cost and benefit terms in the social welfare definition are discounted to Year 0, so that it reflects the net present value of all transactions throughout the planning horizon.

2.5. The Kantorovich Distance

We use the Kantorovich distance [46] to measure the probabilistic distance between the two distributions of social welfare resulting from the selected high-quality scenarios and those from the whole set of scenarios:

$${\mathcal{D}}^{\mathrm{Kan}}({\mathcal{S}}^{\mathrm{H}},{\mathcal{S}}^{\mathrm{W}}):=\sum _{s\in {\mathcal{S}}^{\mathrm{W}}}{p}_s\underset{s^{\prime }\in {\mathcal{S}}^{\mathrm{H}}}{min}|{\mathsf{\Psi }}_s-{\mathsf{\Psi }}_{s^{\prime }}|.$$

(19)

Here, the notations are defined as follows:

• ${\mathcal{S}}^{\mathrm{W}}$: the whole set of scenarios;
• ${\mathcal{S}}^{\mathrm{H}}$: the high-quality set of scenarios;
• $p_s$: probability of scenario s;
• ${\mathsf{\Psi }}_s$: social welfare under scenario s.

3. Solution Methods

3.1. Algorithm for Generation Expansion Planning Calculation

For a given transmission plan and two elements (i.e., demand and fuel cost) of a given scenario, we propose the following Algorithm 1 to calculate the generation capacity element by solving the GEP problem in Equations (1)–(4). In this algorithm, we use ${\mathcal{L}}^{\mathrm{OPF}}\left(\widehat{x}\right)$ to denote the parametric lower level optimal dispatch problem with a given generation investment decision $\widehat{x}$:

$$p\in argmin\{d_2^{\top }p:A_2\widehat{x}+B_{2}p\le b_2,p\ge 0\},$$

and we use ${\mathcal{U}}^{\mathrm{GEP}}\left(\widehat{p}\right)$ to denote the upper level GEP problem with a given dispatch decision $\widehat{p}$:

$$\underset{x,\widehat{p}}{max}\{f^{\mathrm{GEP}}(x,\widehat{p}):A_{1}x\le b_1,x\mathrm{binary}\}.$$

Algorithm 1 Algorithm for the GEP problem in Equations (1)–(4).

1: Choose an arbitrary $x^1$. Define $k=1$ and $m=1$. Set the hyper-parameter M as a positive integer. 2: while$m\le M$do 3:     solve ${\mathcal{L}}^{\mathrm{OPF}}\left(x^k\right)$, and let $p^k$ denote an optimal solution. 4:     solve ${\mathcal{U}}^{\mathrm{GEP}}\left(p^k\right)$, and let $x^{k+1}$ denote an optimal solution. 5:     if $|f^{\mathrm{GEP}}(x^k,p^k)-f^{\mathrm{GEP}}(x^{k-1},p^{k-1})|<ϵ$ then 6:       update $m\leftarrow m+1$; 7:     else 8:       update $m\leftarrow 0$. 9:     end if 10:    update $k\leftarrow k+1$. 11: end while 12: $(x^{*},p^{*})\in argmax\{f^{\mathrm{GEP}}(x^k,p^k):\forall k\}$.

The termination criterion of this algorithm is that the improvement in the GEP objective function is lower than the threshold $ϵ$ for M consecutive iterations. To accelerate the convergence of the algorithm, the dispatch of existing generators from the previous iteration is used for the next iteration, and the dispatch rate (ratio of dispatch over full generation capacity) of new generators is set to be the average of existing ones at the same bus in the same year:

$$\begin{array}{c}\hfill {\widehat{p}}_{g,b,t,y}=\frac{{\sum }_{\widehat{g}\in \{\mathcal{G}(b,y):x_{g,b,y}=1\}}{p}_{\widehat{g},b,t,y}}{{\sum }_{\widehat{g}\in \{\mathcal{G}(b,y):x_{g,b,y}=1\}}{P}_{\widehat{g}}^{\max }}{P}_g^{\max },\forall g\in \{\mathcal{G}(b,y):x_{g,b,y}=0\},b\in \mathcal{B},t\in \mathcal{T},y\in \mathcal{Y}.\end{array}$$

There exist numerous algorithms for solving bilevel optimization models such as in Equations (1)–(4), e.g., branch-and-bound [47] and the Karush–Kuhn–Tucker (KKT) reformulation with big-M parameters [48]. After experimenting with multiple algorithms, we found the proposed heuristic to yield a good balance between computation time and solution quality. The termination criterion was based on the maximum number of iterations.

3.2. Algorithm for Social Welfare Estimation

We use a linear regression model to provide a computationally efficient estimation of social welfare. Conceptually, high-quality scenarios could be selected by first calculating the social welfare for all scenarios and then selecting a subset to minimize the probabilistic distance between the distributions of social welfare resulting from the high-quality and whole set of scenarios. However, this approach requires solving the GEP problem millions of times using the time consuming Algorithm 1. Alternatively, our strategy is to train a regression model to estimate social welfare and select the high-quality scenarios based on the estimated rather than actual social welfare values. If trained efficiently, the regression model requires only a small set of training data to provide reasonably accurate estimation; thus, we only need to use Algorithm 1 to calculate the actual social welfare for a small number of scenarios to produce the training data.

The multiple linear regression model uses the average load for each year, load block, bus, and the average fuel cost for each technology and each year as explanatory variables and social welfare as the response variable:

$$\begin{array}{c}\hfill \widehat{\mathsf{\Psi }}={\beta }_0+\sum _{b\in \mathcal{B}}{\beta }_b^D{\overline{D}}_b+\sum _{y\in \mathcal{Y}}{\beta }_y^D{\overline{D}}_y+\sum _{t\in \mathcal{T}}{\beta }_t^D{\overline{D}}_t+\sum _{h\in \mathcal{H}}{\beta }_h^F{\overline{F}}_h^C+\sum _{y\in \mathcal{Y}}{\beta }_y^F{\overline{F}}_y^C,\end{array}$$

(20)

where:

• ${\overline{D}}_b=\frac{1}{\left|\mathcal{T}\right|·\left|\mathcal{Y}\right|}{\displaystyle \sum _{t\in \mathcal{T}}\sum _{y\in \mathcal{Y}}}{D}_{b,t,y}$,
• ${\overline{D}}_y=\frac{1}{\left|\mathcal{T}\right|·\left|\mathcal{B}\right|}{\displaystyle \sum _{t\in \mathcal{T}}\sum _{b\in \mathcal{B}}}{D}_{b,t,y}$,
• ${\overline{D}}_t=\frac{1}{\left|\mathcal{B}\right|·\left|\mathcal{Y}\right|}{\displaystyle \sum _{b\in \mathcal{B}}\sum _{y\in \mathcal{Y}}}{D}_{b,t,y}$,
• ${\overline{F}}_h^C=\frac{1}{{\displaystyle \sum _{b\in \mathcal{B}}\sum _{y\in \mathcal{Y}}}\left|{\mathcal{G}}_h^{\mathrm{E}}(b,y)\right|}{\displaystyle \sum _{b\in \mathcal{B}}\sum _{y\in \mathcal{Y}}\sum _{g\in {\mathcal{G}}_h^{\mathrm{E}}(b,y)}}{F}_{g,y}^{\mathrm{C}}$,
• ${\overline{F}}_y^C=\frac{1}{{\displaystyle \sum _{b\in \mathcal{B}}}\left|{\mathcal{G}}^{\mathrm{E}}(b,y)\right|}{\displaystyle \sum _{b\in \mathcal{B}}\sum _{g\in {\mathcal{G}}^{\mathrm{E}}(b,y)}}{F}_{g,y}^{\mathrm{C}}$, and
• ${\beta }_0,{\beta }_b^{\mathrm{D}},{\beta }_y^{\mathrm{D}},{\beta }_t^{\mathrm{D}},{\beta }_h^{\mathrm{F}}$, and ${\beta }_y^{\mathrm{F}}$ are regression coefficients that need to be estimated from training data.

3.3. Algorithm for High-Quality Scenario Selection

We present the following algorithm shown in Figure 2 for selecting a small number of high-quality scenarios.

• Step 0: initialization. Randomly select ${\mathcal{S}}^{\mathrm{H}}$, and set the estimated social welfare as ${\widehat{\mathsf{\Psi }}}_s=0,\forall s\in {\mathcal{S}}^{\mathrm{H}}$. Initialize the training dataset as empty.
• Step 1: social welfare calculation. Solve the GEP problem using Algorithm 1, and use its optimal solution $(x^{*},p^{*})$ and Equation (18) to calculate the actual social welfare for the set of high-quality scenarios ${\mathcal{S}}^{\mathrm{H}}$.
  Checkpoint: The algorithm finishes if the error between actual and estimated social welfare is small enough and proceeds to Step 2 otherwise.
• Step 2: social welfare estimation. Add new results from Step 1 to the training dataset; obtain updated regression parameters for Model in Equation (20), then use the updated model to estimate social welfare values for the whole set of scenarios ${\mathcal{S}}^{\mathrm{W}}$.
• Step 3: scenario selection. Update the set of high-quality scenarios ${\mathcal{S}}^{\mathrm{H}}$ by minimizing the Kantorovich distance from ${\mathcal{S}}^{\mathrm{W}}$, ${\mathcal{D}}^{\mathrm{Kan}}({\mathcal{S}}^{\mathrm{H}},{\mathcal{S}}^{\mathrm{W}})$, which was defined in Equation (19). Various heuristic algorithms, such as the golden section search method [49], can be used in this step. As proven by Dupačová et al. [17], the probabilities of high-quality scenarios are given by $p_s={\displaystyle \sum _{i\in I\left(s\right)}}{p}_i,\forall s\in {\mathcal{S}}^{\mathrm{H}}$, where $I\left(s\right)=\{i\in {\mathcal{S}}^{\mathrm{W}}:|{\widehat{\mathsf{\Psi }}}_s-{\widehat{\mathsf{\Psi }}}_i|\le |{\widehat{\mathsf{\Psi }}}_s-{\widehat{\mathsf{\Psi }}}_j|,\forall j\in {\mathcal{S}}^{\mathrm{W}}\}$.

4. Case Study

We demonstrate the effectiveness of the proposed method in a case study, in which two transmission expansion plans are being evaluated for the U.S. Eastern and Western Interconnections. A total of one million scenarios were used to represent the uncertainty of the power system over the next 15 years, and the proposed algorithm was deployed to select ten high-quality scenarios for the two transmission plans.

4.1. Data and Computational Settings

We used the same dataset for the U.S. Eastern and Western Interconnections as used in [50] with some modifications. The dataset contained 169 buses, 730 transmission lines, 1640 existing generators, and 1568 candidate generators, representing the transmission infrastructures of the North American power grid. The locations of the 169 buses are shown in Figure 3. Demand for each year was divided into 19 load blocks. There were 60 generation technologies and fuel types, including coal, gas, oil, nuclear, hydro, geothermal, biomass, wind, and solar. Approximately 30% of existing generation capacity was renewable, and this ratio was required to increase by 1% each year, so that it would reach 45% by the end of year 15. One million demand and fuel cost elements of the scenarios were randomly generated with an average 1% annual growth rate for both. All algorithms were implemented in MATLAB [51], and to solve the models, we used MATLAB and the TOMLAB interface [52] to call CPLEX V.12 [53], used as the mixed integer linear programming solver.

4.2. Visualization of Power System Status

We designed a circular figure to visualize the status of the power system, as shown in Figure 4, which shows the status of the system in Year 0. All buses are represented by black dots and arranged in a circle according to their bus numbers from # 1 at the three o’clock position to # 169 in a clockwise direction. The blue lines inside this circle represent transmission lines that connect buses. Due to the circular arrangement of the dots, the lengths of the line segments in the figure are not proportional to the actual lengths of the transmission lines. The first layer outside the buses represents demand, with the lengths of the purple bars being proportional to the loads. Dark purple was used to indicate load shedding. The second layer represents generation capacity, with the lengths of the green bars being proportional to the generation capacity. Renewable and nonrenewable generation capacities are differentiated with light green and dark green colors, respectively. The third layer represents fuel costs, with the lengths of the orange bars being proportional to the average fuel cost for the generators at the associated buses. At the price of distorted geographical locations of the buses, this figure presents all three elements (demand, generation, and fuel cost) of scenarios considered in this study, allowing the decision maker to have an overview of the system status at once.

4.3. Validation of the Algorithm for Social Welfare Estimation

The algorithm for social welfare estimation from Section 3.2 was used in Step 2 of the algorithm for high-quality scenario selection. It took approximately 10 min to calculate the actual social welfare for one scenario. Using a termination criterion as ${max}_s\frac{|{\mathsf{\Psi }}_s-\widehat{{\mathsf{\Psi }}_s}|}{{\mathsf{\Psi }}_s}<1\%$ for the training scenarios, the algorithm finished after calculating the actual social welfare for 50 scenarios (in approximately 8 h), which would have taken 19 years for the whole set of one million scenarios. To validate the quality of social welfare estimation, we calculated the actual social welfare for 100 random scenarios that were not used in the training set using Algorithm 1. Figure 5 compares the estimated and actual social welfare of these 100 scenarios, and the minimum, mean, and maximum of $\frac{|{\mathsf{\Psi }}_s-\widehat{{\mathsf{\Psi }}_s}|}{{\mathsf{\Psi }}_s}$ were 0.00%, 0.35%, and 0.95%, respectively, for these 100 scenarios.

4.4. Scenario Selection for Two Transmission Plans

We demonstrate the effectiveness of the scenario selection algorithm in Section 3.3 using two transmission plans.

• Transmission Plan 1: existing transmission capacity will stay constant for the next 15 years without any expansion.
• Transmission Plan 2: the capacities of 26 congested lines will be doubled in Year 5, and another 27 congested lines will be doubled in Year 10.

Ten high-quality scenarios were selected for each transmission plan, and two of them are shown in Figure 6. Notice that the ten high-quality scenarios may be different for different plans, because power network topologies change under different plans. These figures illustrate how the power system would evolve in both the temporal and spatial dimensions with different transmission capacities under different scenarios. Although the selected scenarios demonstrated temporal and spatial correlations, it was not straightforward to interpret their social welfare implications, which demonstrated the need for the proposed algorithms for selecting high-quality scenarios.

As a comparison with the proposed algorithm, we also used three other approaches to select ten scenarios for the two transmission plans:

• Random selection: ten scenarios were randomly selected from the whole set of one million scenarios.
• K-means: ten scenarios were selected using the K-means method [21] based on demand and fuel cost information of the whole set of one million scenarios.
• K-medoids: ten scenarios were selected using the K-medoids method [54] based on demand and fuel cost information of the whole set of one million scenarios.

Results from these methods are summarized in

Table 4. Social welfare ($\mathsf{\Psi }$, in $${10}^{15}$) and probability ($p^{\mathrm{H}}$) of 10 selected scenarios and the Kantorovich distance ($D^{\mathrm{Kan}}$ in $${10}^{19}$) for Transmission Plan 1. The 10 selected scenarios for different approaches may be different.

Scenario	Our Approach		Random		K-Means		K-Medoids
	$\mathsf{\Psi }$	${\mathit{p}}^{\mathbf{H}}$	$\mathsf{\Psi }$	${\mathit{p}}^{\mathbf{H}}$	$\mathsf{\Psi }$	${\mathit{p}}^{\mathbf{H}}$	$\mathsf{\Psi }$	${\mathit{p}}^{\mathbf{H}}$
1	3.35	0.04	3.68	0.16	3.44	0.06	3.56	0.09
2	3.68	0.08	4.03	0.21	3.72	0.13	3.82	0.13
3	3.88	0.10	4.31	0.15	4.02	0.14	4.09	0.15
4	4.07	0.13	4.37	0.04	4.23	0.16	4.26	0.14
5	4.23	0.13	4.38	0.01	4.35	0.03	4.26	0.04
6	4.36	0.12	4.38	0.11	4.39	0.18	4.44	0.18
7	4.47	0.12	4.58	0.14	4.55	0.11	4.56	0.09
8	4.58	0.11	4.66	0.05	4.58	0.06	4.68	0.10
9	4.72	0.11	4.71	0.04	4.68	0.11	4.81	0.06
10	4.93	0.05	4.79	0.09	4.92	0.04	4.93	0.04
$D^{\mathrm{Kan}}$	4.29		7.11		5.22		5.31

and

Table 5. Social welfare ($\mathsf{\Psi }$, in $${10}^{15}$) and probability ($p^{\mathrm{H}}$) of 10 selected scenarios and the Kantorovich distance ($D^{\mathrm{Kan}}$ in $${10}^{19}$) for Transmission Plan 2. The 10 selected scenarios for different approaches may be different.

Scenario	Our Approach		Random		K-Means		K-Medoids
	$\mathsf{\Psi }$	${\mathit{p}}^{\mathbf{H}}$	$\mathsf{\Psi }$	${\mathit{p}}^{\mathbf{H}}$	$\mathsf{\Psi }$	${\mathit{p}}^{\mathbf{H}}$	$\mathsf{\Psi }$	${\mathit{p}}^{\mathbf{H}}$
1	3.35	0.04	3.68	0.16	3.44	0.06	3.57	0.09
2	3.68	0.08	4.03	0.21	3.73	0.13	3.82	0.13
3	3.88	0.10	4.31	0.14	3.99	0.14	4.06	0.15
4	4.07	0.13	4.37	0.04	4.23	0.16	4.27	0.14
5	4.24	0.13	4.38	0.01	4.36	0.03	4.27	0.04
6	4.37	0.12	4.38	0.11	4.40	0.18	4.45	0.18
7	4.47	0.12	4.58	0.14	4.58	0.06	4.58	0.09
8	4.59	0.11	4.66	0.05	4.59	0.11	4.66	0.10
9	4.72	0.11	4.71	0.05	4.69	0.11	4.82	0.06
10	4.93	0.05	4.79	0.09	4.94	0.04	4.92	0.04
$D^{\mathrm{Kan}}$	4.22		7.09		5.40		5.32

, in which the ten selected scenarios from each method are sorted in ascending order of social welfare and listed in the same ten rows (e.g., the row for Scenario 1 shows four different worst case scenarios selected using the four methods). Computationally, the proposed approach took approximately 8 h, whereas the other methods only took seconds.

4.5. Cost Benefit Analysis of Transmission Expansion

The analysis from Section 4.4 enables cost benefit analysis of Transmission Plan 2. As shown in

Table 6. Average social welfare (in $${10}^{15}$) for the two transmission plans estimated using different methods.

Method	Plan 1	Plan 2	Difference
All ${10}^6$ scenarios	4.26234	4.26757	0.00523
10 scenarios, our approach	4.22598	4.23097	0.00499
10 random scenarios	4.38840	4.38840	0.00000
10 scenarios, K-means	4.29801	4.30522	0.00721
10 scenarios, K-medoids	4.33987	4.34203	0.00216

, the difference between average social welfare across the whole set of one million scenarios for the two transmission plans was $5.23 trillion, which revealed the social benefit of Transmission Plan 2 throughout the 15-year horizon. The proposed approach estimated this difference as $4.99 trillion using the 10 high-quality scenarios. In comparison, the K-means approach over-estimated this value as $7.21 trillion; K-medoids under-estimated it as $2.16 trillion; and the random selection approach estimated this value to be negligible. These results suggested that the proposed approach was able to estimate the social benefit of transmission plans reasonably accurately, which also allowed it to be potentially integrated into the iterative stochastic TEP model proposed in Figure 1.

5. Conclusions

The main contributions of this paper included a scenario selection method for iterative transmission planning, which had three salient features: (1) correlations between generation capacity and demand, fuel cost, and transmission capacity were explicitly captured; (2) high-quality scenarios were selected to minimize the Kantorovich distance of social welfare distributions between the selected and the whole set of scenarios; and (3) the set of high-quality scenarios was specific for each transmission plan, which enabled this approach to interact iteratively with a stochastic TEP model.

A case study was conducted to demonstrate the proposed approach, in which a 169-bus network was used to represent the U.S. Eastern and Western Interconnections. When compared with three other methods for scenario selection, our algorithm was found to provide a more accurate estimation of the economic value of transmission plans.

The proposed approach was not without its limitations and caveats. For example, the GEP model was a simplified estimation of the actual decision-making process, which involved far more realistic and complex constraints such as multiple decision makers and risk hedging constraints. We also acknowledge that there are numerous other sources of uncertainty in realistic transmission planning projects, which were not explicitly taken into consideration in the proposed approach, such as investment cost, growth rate of distributed energy resources, renewable energy production, energy policies, etc. Besides Kantorovich distance, other probabilistic distances could also be used as the selection criterion for high-quality scenarios. Moreover, the social welfare evaluation could be extended to include environmental and reliability benefits of the power system. Co-optimization of GEP and TEP is another potential topic for future research.