Cross-Validation of the MEDEAS Energy-Economy-Environment Model with the Integrated MARKAL-EFOM System (TIMES) and the Long-Range Energy Alternatives Planning System (LEAP)

Abstract

In the present study, we compare energy transition scenarios from a new set of integrated assessment models, the suite of MEDEAS models, based on a systems dynamic modeling approach, with scenarios from two already well know structurally and conceptually different integrated assessment models, the Integrated MARKAL-EFOM System (TIMES) and the Long-Range Energy Alternatives Planning system (LEAP). The investigation was carried out to cross-compare and benchmark the response of MEDEAS models with TIMES and LEAP in depicting the energy transition in two different countries, Austria and Bulgaria. The preliminary results show a good agreement across all the models in representing scenarios projecting historical trends, while a major discrepancy is detectable when the rate of implementation of renewable energy is forced to increase to achieve energy system decarbonization. The discrepancy is mainly traceable to the differences in the models’ conception and structures rather than in a real mismatch in representing the same scenarios. The present study is put forward as a guideline for validating new modeling approaches that link energy policy decision tools to the global biophysical and socioeconomic constraints.

1. Introduction

The dynamics of complex systems, like socio-economic systems, energy systems, and environmental systems, are all interconnected by a dense network of feedbacks. Through to the development of integrated assessment models (IAMs), which link socioeconomic, energy, and environmental models, it is possible to simulate simultaneously human and environmental systems, implementing dynamics related to society and economy with dynamics in the biosphere and atmosphere into a single modeling framework [1,2]. Since the precursor World3 model [3], several IAMs have been created: However, especially those with greater influence over policy, share a core set of assumptions whose validity is still disputed in the scientific community, underpinning the development of new models [4].

The MEDEAS suite of models [5] is a set of mathematical models that aims to improve the state-of-the-art of IAMs: System dynamics modeling approach makes MEDEAS able to overcome the traditional IAMs sequential structure that allows only for a restricted number of feedbacks among the represented subsystems. Indeed, the highly aggregated nature of the dynamics simulated with IAMs usually translates in an oversimplification of the dynamics related to the economic dimension, e.g., by using aggregated production functions and using prices as indicators of scarcity [6]. Additional limitations of IAMs are the absence of physical limits to the installation of renewable energy sources (RES), and, most relevant, IAMs often omit climate change assessment and impacts [7,8,9]. The approach used in MEDEAS proposes an innovative modeling framework (Figure 1), integrating global biophysical and socioeconomic constraints to support the design of policy. Despite their short existence, the MEDEAS models have already been used to study different aspects of the energy–economy–environment interrelations in several publications; for more detail, the reader is referred to references [10,11,12,13] and the project website [14].

Considering new limits for RES and the impacts of climate change, MEDEAS configures as a more advanced IAM, providing new perspectives on the actions required for the transition, identifying new topics that need attention and further study, for instance, scarcity of the rare earth metals needed to develop energy storage [15,16]. MEDEAS also configures as powerful tools to give insight on ‘decoupling’ strategies [17]: In MEDEAS, economic demand is estimated from exogenous pathways of expected per capita gross domestic product and population evolution [5,13].

MEDEAS models, initially developed in Vensim^®, have an open-source version in the Python programming language, the pymedeas models, which provide three geographical levels: World, EU 28, and Austria (pymedeas_w, pymedeas_eu, and pymedeas_at) [10]. The structure of MEDEAS set of models is nested, meaning that some outputs from the simulations of the world model become inputs to the European model, whose output are in turn used as inputs to the country models.

In this work, we benchmark the outputs of the MEDEAS country models (Austria and Bulgaria) against The Integrated MARKAL-EFOM System (TIMES) model of the Austrian energy system [18,19] for Austria (TIMES_at) and the Long-Range Energy Alternatives Planning system (LEAP) for Bulgaria [20] (LEAP_bg).

TIMES and LEAP are two model generators. TIMES is an energy system model generator that strictly uses linear optimization to produce a least-cost energy system, optimized according to several constraints. The main outputs from TIMES are energy system configurations, which meet the end-use energy service demands, at least cost, while also adhering to various constraints set by the user. TIMES supports the simulation of dynamics related to all technologies available for thermal, renewable, storage/conversion, and transport [21].

LEAP is an integrated, scenario-based, linear modeling tool that accounts for both the energy sector and non-energy sector greenhouse gas (GHG) emission sources and sinks [20]. The LEAP modeling procedure asks the user to specify key non-energy assumptions (demographic, macroeconomic, etc.), energy demand, energy losses, own needs, exogenous and endogenous production capacities, import/export so that LEAP calculates the necessary energy production, additional capacities needed, primary energy requirements, emissions, and costs.

Unlike TIMES and LEAP linear modeling, and as mentioned before, the MEDEAS set of models were developed using the system dynamics approach, which facilitates the integration of interdisciplinary subsystems and dynamics, as well as interactions between them, therefore allowing to capture of non-linearities in the energy system. The general differences (qualitative comparison) between the MEDEAS, TIMES, and LEAP approaches (types of models [21], purposes [22]) are reported at the Medeas project website [23].

In the present study, TIMES_at and LEAP_bg are no longer model generators, but the actual specific models adapted for the two countries from the respective national energy agencies: The Austrian Energy Agency (AEA) for TIMES_at and the Black Sea Energy Research Centre (BSERC) for Bulgaria. TIMES_at and LEAP_bg have been used and continuously developed by the energy agencies for years, giving a reliable representation of all the sectors of the respective national energy systems. For this reason, TIMES_at and LEAP_bg are used as benchmarks to test the results of the respective MEDEAS models for Austria and Bulgaria.

The comparison aims to verify if the MEDEAS country models’ projections are in agreement, within acceptable ranges, with those produced by LEAP_bg and TIMES_at models.

While the World and EU MEDEAS models have already been presented and discussed in the literature [5,10,13,16,24,25], until the current work, the MEDEAS country models for Austria and Bulgaria had not been the subject of any publication. Hence, the current work aims to present the adaptation of the MEDEAS models to a smaller regional scale as well as to test their validity by benchmarking their results against the well-known TIMES and LEAP models.

2. Materials and Methods

In this section, we describe the methodologies used for the cross-validation. Three main aspects are presented: identification of appropriate scenarios to be compared across the models; identification of the variables to be compared within each scenario across the models; method to compare the selected variables.

The models’ outputs to be compared are those generated simulating two long term scenarios: The business as usual (BAU) and optimal level transition (OLT) scenarios, each one adapted to Austria and Bulgaria respective energy–economy–environment structures.

The BAU scenario allows the variables of the model to evolve in the future following historical trends and current and currently planned policies, while, OLT represents a scenario based mainly on improvements in energy efficiency and implementing renewable energy, to accomplish national decarbonization according to the goals of the Paris Agreement [26]. In this regard, the constraint of the Paris agreement has been evaluated in terms of national carbon budgets between 2012 and 2050, calculated for Austria (1.85 GtCO₂ eq) and Bulgaria (1.45 GtCO₂ eq) with the methodology developed by Perissi et al. [27] and summarized in Appendix A. The scenario based on decarbonization hypothesis is the starting point to represent a reference estimation for the end goal of a decarbonizing transition that is compatible with the achievement of the Paris goal. A discussion on the possible achievement or failure of this national budget goal is outside of the aim of the present paper (focused on comparing modeling tools) and will be addressed in future works.

To compare the models’ behaviors, several methodologies can be used, including the root means squared differences (RMSDs), percent variance, and maximum likelihood [28]. RMSDs and maximum likelihood are similar methodologies, however, the RMSDs approach is more intuitive [29,30], especially when independent variables are not random, as in the present case, in which the investigated variables are functions of time series. Moreover, RMSDs is easy to compute and applies to many types of models and types of data, including projections: the lower the RMSD, the smaller the discrepancy between the models under comparison. The RMSDs have the same units of the variables involved in their computing, which is a useful feature to rapidly evaluate the differences of the trajectories generated by MEDEAS with the ones generated by the benchmarking models.

The RMSDs were calculated to compare the outputs of the MEDEAS country models for Austria and Bulgaria with those of the respective Austrian and Bulgarian national models, TIMES_at and LEAP_bg. However, in the present case, the cross-validations for modeling tools assume a slightly different meaning. For each of the pairs of outputs (for instance, one output series from MEDEAS_at and one output series from TIMES_at), we calculated the RMSD in absolute values (listed in

Table 1. List of common outputs of the MEDEAS_at and TIMES-at analyzed in cross-validation. The nomenclature used here for the output variables names is the Vensim system for MEDEAS countries and The Integrated MARKAL-EFOM System TIMES model nomenclature for TIMES Austria.

	SECTORS	MEDEAS_at	TIMES_at
1	Electricity sector	Real final energy by sector and fuel AUT [scenarios, final sources! Electricity Gas and Water Supply]	Export-Ver brauch Sektor Energie
2	Transport sector	Real final energy by sector and fuel AUT [scenarios, final sources! TRANSPORT SECTORS!]	Export-EEV by Sector-en [transport]
3	Industry	Real final energy by sector and fuel AUT [scenarios, final sources! INDUSTRY SECTORS!]	Export-EEV by Sector-en [industry]
4	Agriculture	Real final energy by sector and fuel AUT [scenarios, final sources! agriculture hunting forestry and fishing]	Export-EEV by Sector-en [agriculture]
5	Natural gas	PES fossil fuel extraction [scenarios, gases]	Export-BIV-en [gas]
6	Coal	PES fossil fuel extraction [scenarios, solids]	Export-BIV-en [coal]
7	Oil	PES fossil fuel extraction [scenarios, liquids]	Export-BIV-en [oil]
8	Solar PV	PE solar PV for Elec generation EJ [scenarios]	Export-BIV-en [solar PV]
9	Hydroelectric	PE hydro for Elec generation EJ [scenarios]	Export-BIV-en [hydro]
10	Biomass	PE bioE for Elec generation EJ [scenarios]	Export-BIV-en [biomass]
11	Heat	real FE consumption by fuel [scenarios, heat]	Export-EEV-total-agg-en [heat]
12	Electricity	real FE consumption by fuel [scenarios, electricity]	Export-EEV-total-agg-en [Elec]
13	Solar PV	installed capacity RES elec TW [solar PV, scenarios]	Export-ELC-Cap-Ty [solar PV]
14	Wind (inshore)	installed capacity RES elec TW [wind onshore, scenarios]	Export-ELC-Cap-Ty [windOn]
15	Hydroelectric excl. Pumped storage	installed capacity RES elec TW [hydro, scenarios]	Export-ELC-Cap-Ty [hydro]
16	Biomass	installed capacity RES elec TW [“solid bioE-elec”, scenarios]	Export-ELC-Cap-Ty [solid Bio-e]
17	Electricity sector	GHG emissions by FE Electricity sector [scenarios, final sources]	Export-GHG-perTH GSektor-en [elecricity]
18	Transport sector	GHG emissions by FE Transport sector [scenarios, final sources]	Export-GHG-perTH GSektor-en [transport]
19	Industry sector	GHG emissions by FE Industry sector [scenarios, final sources]	Export-GHG-perTH GSektor-en [industry]
20	Agriculture sector	GHG emissions by FE Agriculture sector [scenarios, final sources]	Export-GHG-perTH GSektor-en [agriculture]
21	Services sector	GHG emissions by FE Services sector [scenarios, final sources]	Export-GHG-perTH GSektor-en [services]
22	Household sector	GHG emissions by FE Households sector [scenarios, final sources]	Export-GHG-perTHG-en [households]
23	Share of electrification of transport sector	Share demand electricity in transport [scenarios]	Export-Transport-EEV-total-en [transport]

and

Table 2. List of common outputs of the MEDEAS_bg and LEAP_bg analyzed in cross-validation. The nomenclature used here for the output variables names is the Vensim models ones for MEDEAS countries and the Long-Range Energy Alternatives Planning System (LEAP) nomenclature for LEAP Bulgaria.

	SECTORS	MEDEAS_bg	LEAP_bg
1	Transport sector	Real final energy by sector and fuel BGR [BAU, transport]	Energy demand final units: Transport
2	Industry sector	Real final energy by sector and fuel BGR [BAU, industry]	Energy demand final units: Industry
3	Agriculture sector	Real final energy by sector and fuel BGR [BAU, agriculture]	Energy demand final units: Agriculture
4	Services sector	Real final energy by sector and fuel BGR [BAU, services]	Energy demand final units: Services
5	Coal	PES fossil fuel extraction [BAU, solids]	Primary requirements/Coal Antracite + Coal Lignite + Coke
6	Oil	PES fossil fuel extraction [BAU, liquids]	Primary requirements/Gasoline + Diesel + Jet Kerosine + Residual Fuel oil + LPG
7		PE solar PV for Elec generation EJ [BAU]	Primary requirements/Solar (PV share)
8	Wind (onshore and offshore)	PE onshore wind for Elec generation EJ [BAU]	Primary requirements/wind
9	Hydroelectric	PE hydro for Elec generation EJ [BAU]	Primary requirements/Hydro
10	Biomass	PE bioE for Elec generation EJ [BAU]	Primary requirements/biomass + biogas + biodiesel + ethanol + MSW + charcoal
11	Electricity	real FE consumption by fuel [BAU, electricity]	FEC/electricity
12	District heat	real FE consumption by fuel [BAU, heat]	FEC/heat
13	Solar PV	installed capacity RES elec TW [solar PV]	Capacity: PV
14	Wind (onshore and offshore)	installed capacity RES elec TW [wind onshore]	Capacity: Wind
15	Hydroelectric excl. Pumped storage	installed capacity RES elec TW [hydro, scenarios]	Capacity: Hydro (excl. PSP)
16	Biomass (electric)	installed capacity RES elec TW [“solid bioE-elec”, scenarios]	Capacity: Biomass
17	Electricity sector	GHG Electricity sector [scenarios]	total = direct + indirect (heat and electricity emissions allocated to demand).
18	Transport sector	GHG Transport sector [scenarios]
19	Industry sector	GHG Industry sector [scenarios]
20	Agriculture sector	GHG Agriculture sector [scenarios]
21	Services sector	GHG Services sector [scenarios]
22	Household sector	GHG Households sector [scenarios]
23	Cumulative GHG emissions (million metric ton CO2 eq.)	Cumulative CO2e GHG emissions [scenarios]	Direct plus Indirect emissions (CO2e).
24	Cumulative GHG Transport sector	GHG emissions by FE Transport sector [scenarios, final sources]	Emissions allocated to demands: Transport
25	Cumulative GHG Industry sector	GHG emissions by FE Industry sector [scenarios, final sources]	Emissions allocated to demands: Industry
26	Cumulative GHG Agriculture sector	GHG emissions by FE Agriculture sector [scenarios, final sources]	Emissions allocated to demands: Agriculture
27	Cumulative GHG Services sector	GHG emissions by FE Services sector [scenarios, final sources]	Emissions allocated to demands: Services
28	Cumulative GHG Household sector	GHG emissions by FE Households sector [scenarios, final sources]	Emissions allocated to demands: Residential
29	Share of electrification of transport sector	Share demand electricity in transport [scenarios]	Share of electricity vs. Total
30	Share of RES in transportation	share RES elect in transport [scenarios] + share RES liquids in transport [scenarios]	Includes RES share in transport and RES share of the electricity in transport

) and with normalized values (listed in

Table 3. Standard Root Mean Square Differences (RMSDs) between MEDEAS_at and TIMES_at within the BAU_at scenario. RMSDs are calculated starting from the year 2010.

	Austria SECTORS BAU_at Abs	2030	2040	2050
1	Electricity sector	5.87 × 10−2	5.67 × 10−2	5.52 × 10−2
2	Transport sector	2.36 × 10−1	2.26 × 10−1	2.16 × 10−1
3	Industry	7.92 × 10−2	7.85 × 10−2	7.15 × 10−2
4	Agriculture	1.27 × 10−3	2.46 × 10−3	3.66 × 10−3
5	Natural gas	8.86 × 10−2	1.32 × 10−1	1.74 × 10−1
6	Coal	1.46 × 10−1	2.03 × 10−1	2.51 × 10−1
7	Oil	6.24 × 10−2	1.06 × 10−1	1.50 × 10−1
8	Solar PV	4.07 × 10−3	8.51 × 10−3	1.47 × 10−2
9	Hydroelectric	9.91 × 10−3	8.33 × 10−3	8.51 × 10−3
10	Biomass	1.93 × 10−1	1.93 × 10−1	1.90 × 10−1
11	Heat	3.10 × 10−1	3.35 × 10−1	3.64 × 10−1
12	Electricity	8.43 × 10−2	1.17 × 10−1	1.51 × 10−1
13	Solar PV	8.37 × 10−4	1.90 × 10−3	3.39 × 10−3
14	Wind (inshore)	8.47 × 10−4	1.95 × 10−3	2.96 × 10−3
15	Hydroelectric excl. Pumped storage	1.18 × 10−3	1.01 × 10−3	9.47 × 10−4
16	Biomass	1.21 × 10−3	1.16 × 10−3	1.16 × 10−3
17	Electricity sector	3.36 × 10−3	3.13 × 10−3	2.99 × 10−3
18	Transport sector	1.36 × 10−2	1.23 × 10−2	1.11 × 10−2
19	Industry sector	8.44 × 10−3	1.26 × 10−2	1.71 × 10−2
20	Agriculture sector	2.08 × 10−3	2.38 × 10−3	2.67 × 10−3
21	Services sector	6.41 × 10−3	7.65 × 10−3	8.80 × 10−3
22	Household sector	2.63 × 10−2	3.07 × 10−2	3.50 × 10−2
23	Share of electrification of transport sector	2.57 × 10−3	7.43 × 10−3	1.65 × 10−2

and

Table 4. Standard RMSDs between MEDEAS_at and TIMES_at within the OLT_at scenario. RMSDs are calculated starting from the year 2010.

	Austria SECTORS OLT_at Abs	2030	2040	2050
1	Electricity sector	5.31 × 10−2	4.55 × 10−2	4.12 × 10−2
2	Transport sector	2.02 × 10−1	1.67 × 10−1	1.45 × 10−1
3	Industry	8.06 × 10−2	1.17 × 10−1	1.49 × 10−1
4	Agriculture	4.71 × 10−3	7.04 × 10−3	8.09 × 10−3
5	Natural gas	4.02 × 10−2	3.57 × 10−2	3.29 × 10−2
6	Coal	4.12 × 10−2	4.06 × 10−2	3.78 × 10−2
7	Oil	1.38 × 10−1	2.27 × 10−1	2.68 × 10−1
8	Solar PV	9.17 × 10−3	2.43 × 10−2	4.24 × 10−2
9	Hydroelectric	1.14 × 10−2	1.14 × 10−2	1.21 × 10−2
10	Biomass	1.85 × 10−1	2.01 × 10−1	1.88 × 10−1
11	Heat	2.70 × 10−1	2.51 × 10−1	2.44 × 10−1
12	Electricity	4.56 × 10−2	6.20 × 10−2	1.00 × 10−1
13	Solar PV	2.21 × 10−3	6.28 × 10−3	1.12 × 10−2
14	Wind (inshore)	8.58 × 10−4	2.03 × 10−3	3.24 × 10−3
15	Hydroelectric excl. Pumped storage	1.17 × 10−3	9.88 × 10−4	8.84 × 10−4
16	Biomass	1.13 × 10−3	9.77 × 10−4	9.34 × 10−4
17	Electricity sector	3.14 × 10−3	2.58 × 10−3	2.25 × 10−3
18	Transport sector	1.18 × 10−2	1.00 × 10−2	9.16 × 10−3
19	Industry sector	2.87 × 10−3	3.68 × 10−3	3.45 × 10−3
20	Agriculture sector	1.25 × 10−3	1.03 × 10−3	9.10 × 10−4
21	Services sector	4.96 × 10−3	4.64 × 10−3	4.36 × 10−3
22	Household sector	2.24 × 10−2	2.22 × 10−2	2.24 × 10−2
23	Share of electrification of transport sector	4.14 × 10−2	1.37 × 10−1	1.94 × 10−1

). RMSDs in absolute values have the same unit as the variable they refer to, thus the principle that the smaller is RMSD, the better is the accordance between the outputs, is not valid because the magnitude of RMSD depends on the unit used to measure the variable. However, RMSDs in absolute values are useful to assess the deviation of the same variable in the two different scenarios, BAU and OLT.

Normalized RMSDs are instead suitable to compare the differences among all considered variables within the models of the same country (considering the principle of the smaller RMSDs). Though there are no consistent methods of normalization in the literature, common choices are the mean [31] or the range (defined as the maximum value minus the minimum value) of the reference data. In the present study, we opted for the mean, to avoid indetermination in case of the difference between the maximum and the minimum of the series was small.

As reference series, we take the means of MEDEAS variables so that it is also possible to assess which of the two benchmarking models (TIMES_at and to LEAP_bg) has more or fewer similarities to the respective MEDEAS country.

The calculation of RMSDs has been automated using a MATLAB^® script. The algorithm calculated RMSD for each pair of data as follows:

• Standard (or absolute) RMSDs:

$$RMS{D}_j = \sqrt{\frac{{\sum }_{t = 1995}^{2050}{\left(M{E}_{t,j} -T{L}_{t, j}\right)}^2}{2050-2010}}.$$

• Normalized RMSDs:

$$RMS{D}_j=\frac{1}{{\sum }_t^{\mathrm{T}}M{E}_{t,j}/\left(T-t+1\right)}\sqrt{\frac{{{\displaystyle \sum }}_t^{\mathrm{T}}{\left(M{E}_{t,j}-T{L}_{t,j}\right)}^2}{\left(T-t+1\right)}},$$

where $M{E}_{t,j}$ and $T{L}_{t,j}$ represent the data of any output variables j at time t, (for instance, considering $M{E}_{t,j}$ the data of MEDEAS_model and $T{L}_{t,j}$ the data of TIMES or LEAP, within the same scenario).

Note that the code also adjusts the units of measurement and provides graphs depicting the RMSDs as a function of the respective pairs of variables. It also provides RMSDs outputs every five years in the interval between 2025 and 2050 starting from 2010, to assess how the variances evolve across the time series.

The absolute and normalized RMSDs have been calculated for Austrian and Bulgarian country models within the two different scenarios under study: BAU and OLT.

For Bulgaria, the algorithm is slightly different in comparison to the one used for Austria as the input files of the two MEDEAS models are not the same. This reflects peculiarities in the energy system of the two countries. However, conceptually, the code executes the same calculations. The dimension adjustment on output units has been taken into account, they are reported and labeled in the MATLAB codes in Appendix B.

In addition, TIMES_at and LEAP_bg are based on an integrated approach of modeling methodologies and they are energy-system models thus, mainly the energy market variables (Table 1 and Table 2) of the four models will allow for the cross-comparison within a set of designed transition scenarios. On the other hand, MEDEAS models take into account not only energy markets but also energy and resource availability. The main feedbacks to the economy module from the rest of the model are delivered by climate change impacts and energy supply availability. Thus, to perform the comparison, only energy-related feedbacks are activated, to optimize the MEDEAS assumptions to fit the TIMES and LEAP assumptions (e.g., renewable capacities).

3. Results

The images below show the energy consumption for the industry sector in Austria and Bulgaria for BAU and OLT scenarios.

The previous Figure 2 is an example of the kinds of comparisons that are made between the three models. The decrease of industry energy consumption in both Austria and Bulgaria OLT scenarios is due to a mix of increased technical efficiency and scenario assumptions. Indeed, the characteristics of the OLT scenario, which is focused on Greenhouse Gases (GHG) emissions reduction (carbon budget limitations), leads to reduced realizable economic growth, much lower economic demand, lower energy consumption in all sectors, including industry (details on scenarios assumptions are available at Project Medeas website [32]).

All the RMSDs are reported, in correspondence with each pair of considered variables, in Table 3 (Standard BAU) and Table 4 (Standard OLT),

Table 5. Normalized RMSDs between the MEDEAS_at and TIMES_at analyzed in BAU_at scenarios cross-validation.

	Austria SECTORS BAU_at Norm	2030	2040	2050
1	Electricity sector	1.44	1.36	1.29
2	Transport sector	1.84	1.73	1.65
3	Industry	0.19	0.17	0.15
4	Agriculture	0.05	0.10	0.14
5	Natural gas	0.24	0.32	0.38
6	Coal	0.61	0.70	0.75
7	Oil	0.12	0.20	0.27
8	Solar PV	1.15	2.12	3.33
9	Hydroelectric	0.07	0.06	0.06
10	Biomass	4.19	4.30	4.32
11	Heat	0.80	0.81	0.83
12	Electricity	0.26	0.32	0.37
13	Solar PV	0.75	1.50	2.45
14	Wind (inshore)	0.26	0.45	0.54
15	Hydroelectric excl. Pumped storage	0.09	0.07	0.07
16	Biomass	0.71	0.69	0.70
17	Electricity sector	1.15	0.98	0.86
18	Transport sector	1.36	1.16	0.99
19	Industry sector	0.25	0.33	0.40
20	Agriculture sector	0.71	0.74	0.77
21	Services sector	0.81	0.88	0.92
22	Household sector	0.84	0.89	0.93
23	Share of electrification of transport sector	0.05	0.12	0.20

(Normalized BAU), and

Table 6. Normalized RMSDs between the MEDEAS_at and TIMES_at analyzed in OLT_at scenarios cross-validation.

	Austria SECTORS OLT_at Norm	2030	2040	2050
1	Electricity sector	1.44	1.35	1.30
2	Transport sector	1.72	1.41	1.21
3	Industry	0.24	0.39	0.54
4	Agriculture	0.23	0.39	0.49
5	Natural gas	0.13	0.13	0.13
6	Coal	0.33	0.41	0.46
7	Oil	0.30	0.52	0.62
8	Solar PV	2.60	6.02	9.53
9	Hydroelectric	0.08	0.08	0.08
10	Biomass	4.05	4.59	4.46
11	Heat	0.80	0.80	0.80
12	Electricity	0.16	0.23	0.38
13	Solar PV	1.99	4.96	8.04
14	Wind (inshore)	0.26	0.45	0.55
15	Hydroelectric excl. Pumped storage	0.09	0.07	0.06
16	Biomass	0.66	0.58	0.56
17	Electricity sector	1.52	1.58	1.68
18	Transport sector	1.34	1.15	1.06
19	Industry sector	0.12	0.20	0.23
20	Agriculture sector	0.60	0.63	0.68
21	Services sector	0.78	0.82	0.84
22	Household sector	0.84	0.87	0.90
23	Share of electrification of transport sector	0.77	1.89	2.04

(Normalized OLT) for Austria; in

Table 7. Standard RMSDs between MEDEAS_bg and LEAP_bg within the BAU_bg scenario. RMSDs are calculated starting from the year 2010.

	Bulgaria SECTORS BAU_bg Abs	2030	2040	2050
1	Transport sector	5.86 × 10−2	4.86 × 10−2	4.23 × 10−2
2	Industry sector	3.37 × 10−2	4.18 × 10−2	4.58 × 10−2
3	Agriculture sector	3.18 × 10−3	2.65 × 10−3	2.37 × 10−3
4	Services sector	3.82 × 10−2	3.82 × 10−2	3.76 × 10−2
5	Coal	3.42 × 10−2	3.87 × 10−2	3.38 × 10−2
6	Oil	2.99 × 10−2	4.08 × 10−2	4.04 × 10−2
7	Solar PV requirements	1.55 × 10−1	1.69 × 10−1	1.76 × 10−1
8	Wind (onshore and offshore)	4.34 × 10−3	7.57 × 10−3	9.64 × 10−3
9	Hydroelectric	2.05 × 10−3	5.37 × 10−3	6.94 × 10−3
10	Biomass	6.35 × 10−3	8.41 × 10−3	7.87 × 10−3
11	Electricity	2.42 × 10−3	4.06 × 10−3	5.16 × 10−3
12	District heat	2.06 × 10−2	3.09 × 10−2	4.27 × 10−2
13	Solar PV	5.46 × 10−2	4.75 × 10−2	4.16 × 10−2
14	Wind (onshore and offshore)	4.08 × 10−4	9.75 × 10−4	1.49 × 10−3
15	Hydroelectric excl. Pumped storage	6.97 × 10−4	8.14 × 10−4	8.03 × 10−4
16	Biomass (electric)	1.82 × 10−4	3.33 × 10−4	4.41 × 10−4
17	Electricity sector	1.38 × 10−2	1.14 × 10−2	9.95 × 10−3
18	Transport sector	2.64 × 10−3	2.25 × 10−3	2.07 × 10−3
19	Industry sector	6.27 × 10−3	6.38 × 10−3	6.15 × 10−3
20	Agriculture sector	2.43 × 10−4	2.11 × 10−4	1.87 × 10−4
21	Services sector	1.13 × 10−3	1.04 × 10−3	1.12 × 10−3
22	Household sector	8.28 × 10−4	7.61 × 10−4	8.96 × 10−4
23	Cumulative GHG emissions (million metric ton CO2 eq.)	1.44 × 10−1	2.24 × 10−1	2.81 × 10−1
24	Cumulative GHG Transport sector	2.21 × 10−2	1.86 × 10−2	1.63 × 10−2
25	Cumulative GHG Industry sector	1.27 × 10−2	1.06 × 10−2	9.48 × 10−3
26	Cumulative GHG Agriculture sector	1.16 × 10−3	9.61 × 10−4	8.54 × 10−4
27	Cumulative GHG Services sector	5.17 × 10−3	4.34 × 10−3	3.77 × 10−3
28	Cumulative GHG Household sector	5.78 × 10−2	5.57 × 10−2	5.12 × 10−2
29	Share of electrification of transport sector	4.75 × 10−2	1.12 × 10−1	1.78 × 10−1
30	Share of RES in transportation	6.73 × 10−2	8.22 × 10−2	8.43 × 10−2

(Standard BAU) and

Table 8. Standard RMSDs between MEDEAS_bg and LEAP_bg within the OLT_bg scenario. RMSDs are calculated starting from the year 2010.

	Bulgaria SECTORS OLT_bg Abs	2030	2040	2050
1	Transport sector	5.86 × 10−2	4.86 × 10−2	4.23 × 10−2
2	Industry sector	3.37 × 10−2	4.18 × 10−2	4.58 × 10−2
3	Agriculture sector	3.18 × 10−3	2.65 × 10−3	2.37 × 10−3
4	Services sector	3.82 × 10−2	3.82 × 10−2	3.76 × 10−2
5	Coal	3.42 × 10−2	3.87 × 10−2	3.38 × 10−2
6	Oil	2.99 × 10−2	4.08 × 10−2	4.04 × 10−2
7	Solar PV requirements	1.55 × 10−1	1.69 × 10−1	1.76 × 10−1
8	Wind (onshore and offshore)	4.34 × 10−3	7.57 × 10−3	9.64 × 10−3
9	Hydroelectric	2.05 × 10−3	5.37 × 10−3	6.94 × 10−3
10	Biomass	6.35 × 10−3	8.41 × 10−3	7.87 × 10−3
11	Electricity	2.42 × 10−3	4.06 × 10−3	5.16 × 10−3
12	District heat	2.06 × 10−2	3.09 × 10−2	4.27 × 10−2
13	Solar PV	5.46 × 10−2	4.75 × 10−2	4.16 × 10−2
14	Wind (onshore and offshore)	4.08 × 10−4	9.75 × 10−4	1.49 × 10−3
15	Hydroelectric excl. Pumped storage	6.97 × 10−4	8.14 × 10−4	8.03 × 10−4
16	Biomass (electric)	1.82 × 10−4	3.33 × 10−4	4.41 × 10−4
17	Electricity sector	1.38 × 10−2	1.14 × 10−2	9.95 × 10−3
18	Transport sector	2.64 × 10−3	2.25 × 10−3	2.07 × 10−3
19	Industry sector	6.27 × 10−3	6.38 × 10−3	6.15 × 10−3
20	Agriculture sector	2.43 × 10−4	2.11 × 10−4	1.87 × 10−4
21	Services sector	1.13 × 10−3	1.04 × 10−3	1.12 × 10−3
22	Household sector	8.28 × 10−4	7.61 × 10−4	8.96 × 10−4
23	Cumulative GHG emissions (million metric ton CO2 eq.)	1.44 × 10−1	2.24 × 10−1	2.81 × 10−1
24	Cumulative GHG Transport sector	2.21 × 10−2	1.86 × 10−2	1.63 × 10−2
25	Cumulative GHG Industry sector	1.27 × 10−2	1.06 × 10−2	9.48 × 10−3
26	Cumulative GHG Agriculture sector	1.16 × 10−3	9.61 × 10−4	8.54 × 10−4
27	Cumulative GHG Services sector	5.17 × 10−3	4.34 × 10−3	3.77 × 10−3
28	Cumulative GHG Household sector	5.78 × 10−2	5.57 × 10−2	5.12 × 10−2
29	Share of electrification of transport sector	4.75 × 10−2	1.12 × 10−1	1.78 × 10−1
30	Share of RES in transportation	6.73 × 10−2	8.22 × 10−2	8.43 × 10−2

(Standard OLT),

Table 9. Normalized RMSDs between the MEDEAS_bg and LEAP_bg analyzed in BAU scenarios cross-validation.

	Bulgaria SECTORS BAU_bg Norm	2030	2040	2050
1	Transport sector	1.09	1.05	1.04
2	Industry sector	0.08	0.10	0.10
3	Agriculture sector	0.37	0.40	0.43
4	Services sector	7.49	7.85	8.11
5	Coal	1.37	1.34	1.29
6	Oil	0.31	0.31	0.30
7	Solar PV requirements	0.11	0.12	0.13
8	Wind (onshore and offshore)	0.77	0.64	0.49
9	Hydroelectric	0.09	0.13	0.28
10	Biomass	0.30	0.24	0.21
11	Electricity	0.82	0.81	0.81
12	District heat	0.15	0.13	0.12
13	Solar PV	0.66	0.67	0.68
14	Wind (onshore and offshore)	0.30	0.23	0.22
15	Hydroelectric excl. Pumped storage	0.50	0.42	0.32
16	Biomass (electric)	0.27	0.24	0.21
17	Electricity sector	4.32	4.15	4.06
18	Transport sector	0.58	0.53	0.50
19	Industry sector	0.21	0.26	0.29
20	Agriculture sector	0.36	0.41	0.45
21	Services sector	0.15	0.15	0.14
22	Household sector	0.13	0.13	0.12
23	Cumulative GHG emissions (million metric ton CO2 eq.)	0.44	0.59	0.69
24	Cumulative GHG Transport sector	5.08	4.89	4.78
25	Cumulative GHG Industry sector	0.65	0.62	0.61
26	Cumulative GHG Agriculture sector	0.55	0.57	0.58
27	Cumulative GHG Services sector	0.85	0.85	0.85
28	Cumulative GHG Household sector	0.94	0.94	0.94
29	Share of electrification of transport sector	0.40	0.35	0.33
30	Share of RES in transportation	35.94	20.59	13.10

(Normalized BAU), and

Table 10. Normalized RMSDs between the MEDEAS_bg and LEAP_bg analyzed in OLT scenarios cross-validation.

	Bulgaria SECTORS OLT_bg Norm	2030	2040	2050
1	Transport sector	0.91	0.74	0.67
2	Industry sector	0.41	0.62	0.83
3	Agriculture sector	0.31	0.31	0.34
4	Services sector	9.06	11.06	13.04
5	Coal	1.66	1.73	1.81
6	Oil	0.17	0.25	0.28
7	Solar PV requirements	0.21	0.31	0.35
8	Wind (onshore and offshore)	1.21	1.97	2.28
9	Hydroelectric	0.46	1.13	1.29
10	Biomass	0.45	0.58	0.53
11	Electricity	0.94	1.62	2.12
12	District heat	0.20	0.34	0.54
13	Solar PV	0.62	0.61	0.62
14	Wind (onshore and offshore)	0.44	0.98	1.36
15	Hydroelectric excl. Pumped storage	0.31	0.35	0.33
16	Biomass (electric)	3.39	6.34	8.49
17	Electricity sector	6.58	7.14	7.99
18	Transport sector	0.51	0.42	0.41
19	Industry sector	0.69	0.92	1.14
20	Agriculture sector	0.29	0.30	0.33
21	Services sector	0.19	0.21	0.29
22	Household sector	0.08	0.09	0.13
23	Cumulative GHG emissions (million metric ton CO2 eq.)	0.44	0.60	0.70
24	Cumulative GHG Transport sector	4.27	3.51	3.22
25	Cumulative GHG Industry sector	0.68	0.78	0.90
26	Cumulative GHG Agriculture sector	0.55	0.60	0.68
27	Cumulative GHG Services sector	0.85	0.89	0.98
28	Cumulative GHG Household sector	0.95	0.96	0.98
29	Share of electrification of transport sector	3.05	5.67	7.12
30	Share of RES in transportation	35.61	14.26	6.40

(Normalized OLT) for Bulgaria.

The RMSDs give information on the level of agreement between the curves, allowing also for qualitative comparisons of the time-series across all the investigated models.

As previously introduced, to assess the overall similarity between the benchmarking models (LEAP_Bg and TIMES_at) and MEDEAS country models across all the variables and the scenarios, we calculated the distribution of the normalized RMSDs with respect to the mean of the MEDEAS series of data.

The normalized RMSDs have been calculated every ten years since 2010 to 2050 (2010–2030, 2010–2040, 2010–2050) to analyze the variances among the variables to evaluate the response of models over time. We included the historical data (2010–2020) in the RMSDs calculation to account also for the difference in the historical sources between the benchmarking models and MEDEAS countries.

With the normalized RMSDs distribution we highlight how many and which variables series shows lower normalized RMSDs, representing a good agreement between MEDEAS and national models, or instead how many show higher RMSDs, representing the low resemblance between MEDEAS and both TIMES and LEAP outputs, investigating, at the same time, the models’ temporal variances’ response.

3.1. Results for Austria

The RMSDs distribution in absolute values for Austria is reported in Table 3 and Table 4. Variables energy units are in Exajoule (EJ) if not differently specified (i.e., for “share” variables).

Comparing the series for Austria in absolute values, most of the variables that TIMES and MEDEAS have in common show similar RMSDs for the BAU and the OLT scenarios.

The most important differences are visible for variables 2 (transport), 5, 6, 7 (fossil extraction), 10 (biomass requirement), 11 (heat), 23 (share of electricity in transport). These results suggest that these variables are probably the most affected by the initial condition and evolution of the scenarios from BAU to OLT.

A key difference in RMSD values is seen in variables that are characterized by an exponential growth in MEDEAS compared to stable growth (or no growth) in TIMES. These behaviors in BAU scenarios have been observed for variable 11 (heat-real FE consumption by fuel in MEDEAS_at and district heat in TIMES_at), variable 2 (real final energy by sector and fuel AUT [transport] in MEDEAS_at and Export-EEVbySector-en-Transport for TIMES_at), variable 5 (PES fossil fuel extraction [scenarios, gases] in MEDEAS_at and Export-BIV-en-Fossil fuels gaseous in TIMES_at) and variable 6 (PES fossil fuel extraction [scenarios, solids] in MEDEAS_at and Export-BIV-en-Fossil fuels solids in TIMES_at). This is mainly due to the unrestricted availability of resources in MEDEAS models, and to the fact that the projections in TIMES_at are instead almost stable for the whole period.

For the same variables (2, 5, and 6), RMSDs decrease in the respective OLT scenarios, where both MEDEAS_at and TIMES_at show declining trends.

Variables related to liquids show the opposite trends. Variable number 7 (PES fossil fuel extraction [scenarios, liquids] in MEDEAS_at and Export-BIV-en-Fossil fuels liquids in TIMES_at) shows higher RMSD for OLT scenarios, mainly due to the deep phasing out of liquids in TIMES_at projections between 2030 and 2050. Variables 17–23, which describe the projections of RES capacity installation and the projections of GHG emissions by sectors, show a very good agreement in being represented by TIMES_at and MEDEAS_at models for both BAU and OLT scenarios.

Regarding the achievement of the national budget within the OLT scenario, TIMES_at show simulated budget quite close to the national objective, around 2.5 GtCO₂ eq. MEDEAS_at results in a cumulative emissions simulation double than TIMES_at. The order of magnitude is the same, the major deviation of MEDEAS_at is again due to setting unrestricted availability of resources in MEDEAS models.

This suggests that, for Austria, the increase in renewable implementation rates is the main action that would lead to national decarbonization in the frame of the Paris goal. Further studies will point out a more exhaustive decarbonization strategy.

Comparing the series in normalized values for the Austrian BAU scenario (Table 5), it is then possible to explore the difference between variables across the whole data series behaviors (Figure 3).

Decreases in RMSDs over time can be observed, in variables 1 (electricity), 2 (energy in transport), 9 (hydroelectric requirement), 15 (hydroelectric pumped storage), 17 (electricity sector), and 18 (transport). Thus, variable 1 and 9 results are more affected by the difference in the initial conditions but the discrepancy tends to diminish gradually as both TIMES and MEDEAS approach the 2050 decarbonization goal. This is also evident in variables 17 and 18, which describe the related GHG emissions.

The deviation for PV variables 8 and 13 increase three times in 2050, since TIMES_at considers a nearly linear increase on PV development while in MEDEAS_at, the variable (all variables related to renewables) grow exponentially according to a set growth rate.

The change in RMSDs with time for the other variables are minor, reflecting a good agreement with the simulations between the two models.

In the OLT scenario (Table 6, Figure 4), the variances for the PV variables 8 and 13 increase are even higher, highlighting again that TIMES_at considers an almost linear increase on PV development while in MEDEAS_at, the variable (all variables related to renewables) grows exponentially.

The change in RMSDs with time for the other variables is minor, reflecting a good agreement with the simulations between the two models.

3.2. Results for Bulgaria

The compared variables and the RMSDs in absolute values for MEDEAS_bg and LEAP_bg are reported in Table 7 (BAU) and Table 8 (OLT).

Variable 7 represents the increase in solar PV generation that in LEAP_bg is considered to evolve very rapidly since 2015–2016 and then the growth stabilized up to 2050. MEDEAS_bg instead reports slower growth rates along with all the time-lapse considered, spreading from 2010 to 2040, in which it reaches the same values of LEAP projections. The difference of the trajectories is attributed to the different type of inputs in the two models. While LEAP_bg allows specification of concrete annual values, the input in MEDEAS_bg is much more generalized and the particular annual values are calculated endogenously, considering different the limitations.

Variable 23 represents the total GHG emissions, including the ones not from the energy sectors. The discrepancy is due to the different modeling approaches. LEAP can achieve any emission reduction target by the introduction of fuel switch (e.g., switch to RES) and energy efficiency. MEDEAS models, however, take into account more factors, such as the additional energy needed to “fuel” the transition (e.g., the additional RES and storage capacities), the energy returned on energy invested (EROI), and resources necessary for climate change adaptation. All these increase the GHG emissions in MEDEAS compared to LEAP. For example, in BAU, in LEAP_bg, the GHG emissions are stable throughout the whole period, while in MEDEAS_bg, this variable increases about four times.

Comparing the series in normalized RMSDs values (Table 9 and Table 10), it is possible to explore the difference across the whole series of variables in BAU (Figure 5) and OLT (Figure 6) scenarios.

The variable 30 “Share of RES in transport” in the BAU scenario is influenced by the previous explained different approach between MEDEAS and LEAP in achieving decarbonization switching to RES; this provokes an increase in the share of electricity for transportation in 2020 even in a BAU perspective; this also affects the variable 24, the carbon budget accounts for transport and the GHG electricity sector (variable 17).

The variance in variable 4, energy for the service sector, is also due to the different approach of the two models: In MEDEAS, the final energy demands by final fuel necessary to produce a certain amount of goods and services is constrained by the available final energy that accounts for limitations of nonrenewable fuels—peak oil phenomena—and the sustainable potential of renewables, while this demand is not constrained in LEAP_bg.

The variance due to this limitation is emphasized in RMSDs (Table 10) evaluation for OLT scenarios (Figure 6). In OLT, biomass production faces the increase necessary to provide biofuels for transport: the LEAP unconstrained demand for biomass is much higher than the MEDEAS one.

Nevertheless, the simulated carbon budgets with MEDEAS_bg and LEAP_bg in OLT scenario are of the same order of magnitude, even this is higher than the estimated national objective of 1.45 GtCO₂ eq, suggesting that further measures than RE implementations are necessary to meet the Paris target.

Similarly, to the case of Austria and TIMES, there are more similarities between MEDEAS_bg and LEAP_bg in designing the BAU scenarios than in designing OLT.

4. Discussion

Today, several IAMs coexist to deal with the complex interactions, high uncertainties, and knowledge gaps between interconnections within the environment and human societies [32,33,34]. For these reasons, it is usually quite challenging to compare IAMs, with very different characteristics [5]. Moreover, these simulation tools do not consider the possibility to jointly analyze the resource limitations, the impact of climate, and the feedbacks with the environment; besides, they are mainly world models with no geographical disaggregation of the economic evolution at lower geographical levels. The MEDEAS suite of models, not only fills those gaps for the world level but, due to its nested structure, also allows for exploring the global effect at regional and national levels. These are unprecedent features among the existing IAMs and the comparison of MEDEAS models with existing models [22] opens a new field of investigation to assess the efficiency of models in representing socio/ecological/economic dynamics of a country. For instance, some of the feedback loops of the MEDEAS models have to be turned off to allow for a realistic comparison with the traditional national models including TIMES and LEAP: this helps in making the two approaches comparable, but on the other side, the geographical upper-level variables and assumptions that may affect regional dynamics have not been included in the analysis, meaning that only partial conclusions on cross-validating MEDEAS country with a benchmark can be achieved at the present stage of investigation.

Nevertheless, extremely interesting findings can be highlighted from this study. One clear distinction from the cross-comparison and qualitative evaluation between MEDEAS country and TIMES-Austria and LEAP-Bulgaria highlight the approach used to simulate limited availability of natural resources: In MEDEAS, energy is not generated within the energy system, but it is converted into a usable form by technologies and is provided by a limited amount of available fossil resources. TIMES_at and LEAP_bg do not account for the limited availability of fossil fuels.

Moreover, MEDEAS can provide more comprehensive simulations by allowing disequilibrium dynamics in the economy by not assuming clearing markets (i.e., not imposing general equilibrium) [35], by including the evaluation of the effects of climate change on the economy [36] and the effects of (economic) development on depletion of natural resources [37], all in the context of simulations for the transition to a decarbonized energy system. Although other general equilibrium models like TIMES-Austria and LEAP-Bulgaria do not account for these dynamics, these models still provide invaluable information for infrastructure, investment, and capacity requirements for the transition, especially when specified at the national level.

TIMES_at was developed to create medium to long-term scenarios for the Austrian energy system. The optimization is conducted according to several constraints that are built into the model and can be parameterized. Crucially, in the TIMES model, energy demand development is to a large extent the result of exogenous scenario parameters including, most importantly, GDP growth and population growth. These parameters need to be assumed or modeled within a different framework. In MEDEAS, growth is exogenous only until energy limits are reached (if reached). Notably, there is no feedback from the energy system to GDP within TIMES, which has implications for scenario transition rate comparisons.

The system boundaries of the TIMES model include the Austrian energy system, structured into several sectors (industry, households, agriculture, electricity, and district heat supply etc.). To allow for direct and easy comparison with historical data, the structure of the model and model results is largely consistent with Austrian energy statistics. TIMES includes several assumptions that have implications for the results. First, TIMES assumes perfect foresight, which means that all investment decisions are made in each period with full knowledge of future events (fuel price developments, technologies available in the future, etc.).

LEAP_bg is an integrated, scenario-based modeling tool that accounts for both energy sector and non-energy sector greenhouse gas (GHG) emission sources and sinks [20]. The LEAP modeling procedure asks for specifying of key non-energy assumptions (demographic, macroeconomic, etc.), energy demand and energy supply, energy losses, own needs, exogenous and endogenous production capacities, import/export, so that LEAP_bg calculates the necessary energy production (to cover the demand, losses, etc.), additional capacities needed, primary energy requirements, emissions, and costs.

5. Conclusions

In the present study, we compared the results of the MEDEAS country models for Austria and Bulgaria with those of existing country-level models, TIMES and LEAP, used by Austrian Energy Agency and the Black Sea Energy Research Center of Bulgaria, with the aim to draft preliminary guidelines in validating a completely new modeling approach, the MEDEAS IAM at the regional level.

The main difference between MEDEAS and the other models is the innovative approach implemented in the development of MEDEAS, especially as regarding the modeling of the economy (non-clearing markets) and the environmental and biophysical constraints introduced and, most importantly, the connection between the two. TIMES-Austria and LEAP-Bulgaria being more focused on the components of the energy system and in optimizing, minimizing, or maximizing energy functions undoubtedly put more emphasis on detailed simulations for specific geographical areas. MEDEAS is instead focused on simulating dynamics and interactions within the energy system and its dependencies and interactions with other social-environmental systems.

Regarding quantitative evaluations, the models’ outputs compared are those generated simulating two long term scenarios: the business as usual (BAU) and optimal level transition (OLT) scenarios, each one adapted to Austria and Bulgaria respective energy–economy–environment structures. The BAU scenario assumes the future evolves following historical trends, while, OLT represents a scenario of economic growth and reduction of energy use through improvements in energy efficiency and implementing renewable energy, to accomplish national decarbonization according to the goals of the Paris Agreement.

As for the results from the normalizations of the TIMES and LEAP data series in respect to the MEDEAS series mean values (within each dataset), the RMSDs show that correlations between the models and across all the variables are higher for BAU scenarios in comparing MEDEAS_at with TIMES_at and also for MEDEAS_bg versus LEAP_bg. However, the highers RMSDs in OLT scenarios are mainly traceable to the models’ conception and structure differences rather than in a real mismatch in representing the same scenarios.