A Review of Criticisms of Integrated Assessment Models and Proposed Approaches to Address These, through the Lens of BECCS

Abstract

This paper reviews the many criticisms that Integrated Assessment Models (IAMs)—the bedrock of mitigation analysis—have received in recent years. Critics have asserted that there is a lack of transparency around model structures and input assumptions, a lack of credibility in those input assumptions that are made visible, an over-reliance on particular technologies and an inadequate representation of real-world policies and processes such as innovation and behaviour change. The paper then reviews the proposals and actions that follow from these criticisms, which fall into three broad categories: scrap the models and use other techniques to set out low-carbon futures; transform them by improving their representation of real-world processes and their transparency; and supplement them with other models and approaches. The article considers the implications of each proposal, through the particular lens of how it would explore the role of a key low-carbon technology—bioenergy with carbon capture and storage (BECCS), to produce net negative emissions. The paper concludes that IAMs remain critically important in mitigation pathways analysis, because they can encompass a large number of technologies and policies in a consistent framework, but that they should increasingly be supplemented with other models and analytical approaches.

1. Introduction

Much of the global-level analysis on climate change mitigation has to date been heavily based on the results of integrated assessment models (IAMs); from the second Intergovernmental Panel on Climate Change (IPCC) assessment report in 1996 [1], a key feature of the IPCC mitigation working group (Working Group III—WGIII) has been the underpinning IAM analysis. The IPCC reports have been built on a wider collaborative effort in using and comparing IAMs, organised via the Energy Modelling Forum which has run since 1976 [2], and other international collaborative initiatives (e.g., [3]). Until the Paris Agreement [4], the majority of such analysis focused on limiting temperature change to 2 °C, the internationally agreed climate target in the run-up to the 21st Conference of the Parties (COP21), and exploring the impacts of higher global temperatures. As expected in the wake of the Paris Agreement, new analysis is emerging on the mitigation implications of the 1.5 °C goal, much of it summarised in the IPCC’s Special Report on 1.5 °C [5], published in October 2018. Building on the analysis presented in this report, there is now a need to better understand the implications of the additional mitigation required to achieve 1.5 °C compared to the 2 °C goal, in terms of the costs, sectors that need to further decarbonise and the real-world feasibility of achieving these different goals. In parallel, there has been more focus on the national scale implementation of policies that are required to implement mitigation goals (e.g., [6]) rather than global models focused on target setting. As such, it is an appropriate juncture to reflect on the utility and credibility of IAMs in defining potential low-carbon pathways, with a view to understanding whether they should be improved, and if so, then how best to do this.

This paper reviews the major themes that have arisen in IAM analysis in the period since the IPCC fifth assessment report’s (AR5) WGIII mitigation report was published in 2014 [7], focusing in particular on the criticisms that have been asserted about IAMs, and the actions that have been proposed to address these criticisms. In order to bring to life the implications of the actions proposed to address the criticisms, we delve deeply into the representation and role of a particularly prominent and important low-carbon technology group which is represented in IAMs. This is bioenergy with carbon capture and storage (BECCS), to achieve net negative emissions, thereby making some of the most stringent mitigation targets achievable in the framework of the IAMs. BECCS is therefore a useful lens through which to assess the three major suggested actions that have followed from the many criticisms of IAMs—to scrap them, to transform them, or to supplement them with other analytical models and methods.

The review is not strictly systematic, which means it does not necessarily encompass every critique of IAMs that has been published in recent years. However, we believe it to be very comprehensive, in terms of its coverage of the major themes that have arisen when reviewing and critiquing the use of IAMs in low-carbon pathways analysis. In addition, the review is focused primarily on the use of IAMs in global mitigation pathways analysis, rather than on any specific country-level modelling. Nevertheless, as discussed in Section 3.6, we refer to the treatment of different regions in IAM analysis when discussing the relative challenge of achieving mitigation in different countries and the extent to which IAMs have reflected this.

In reviewing and reflecting upon the different critiques of IAMs, it should be noted that integrated assessment modelling is subject to the benefits and limitations of modelling more generally: namely, that by definition, modelling involves a simplification of reality, and the sheer number of detailed quantitative outputs can distract from the core insights—chiefly amongst them a greater user appreciation of policy problems—that they can provide [2]. In spite of this important defence of models, it remains a legitimate and indeed critical activity to continue questioning whether the models used to approach particular policy problems are sufficiently fit for purpose to provide genuinely useful insights, and if not, then how they might be replaced, improved or supplemented.

The rest of this paper is set out as follows: Section 2 provides an overview of IAMs in the context of this paper, where our focus falls on the energy system (and in some cases land use) component of these models, given that this has been the major recipient of recent critiques and its centrality in producing the low-carbon pathways reported in the IPCC reports. Section 3 then reviews the critiques of IAMs, grouping them into specific topics and limitations. Section 4 sets out the different suggestions that have been made as a result of these criticisms, which fall into three categories of “scrap”, “transform” or “supplement”. Section 5 focuses on BECCS to assess what the implications of each of these three categories of suggestions would be. Section 6 presents our conclusions of next steps for the IAM community as well as for the broader analytical and policy communities working on low-carbon pathways.

2. What Are Integrated Assessment Models and How Are They Used for Mitigation Analysis?

IAMs are computer models that describe the potential evolution of the global energy system, as well as other GHG-emitting systems such as agriculture and land use, over the 21st century. Alternative pathways are determined using assumptions around economic and population growth, as well as growth of other relevant drivers of demand for GHG-emitting activities such as transport, heating, lighting, industrial products and agricultural goods. The models have traditionally considered scenarios which either continue energy/industrial/agricultural/other GHG-emitting practices on a “business-as-usual” pathway, or alternatively substitute low-carbon technologies and lifestyles for GHG-intensive (hereafter “carbon-intensive”) technologies and lifestyles, to reach sufficiently low GHG emissions levels to limit long-term temperature change to specified goals.

The setting of alternative pathways has become more sophisticated, first through the development of emissions scenarios based on possible global futures [8], then more recently through the development of five different shared socio-economic pathways (SSPs) and their related storylines [9,10], which represent a variety of potential future development pathways for the global economy, including its energy intensity and mix of energy sources. The six IAMs involved in the initial development of the SSPs (and the recent analysis of 1.5C pathways across the different SSPs [11]) are amongst the most widely used in global mitigation scenario analysis:

• IMAGE [12], hosted by the PBL Netherlands Environmental Assessment Agency
• MESSAGE-GLOBIOM [13], hosted by the International Institute for Applied Systems Analysis, Austria
• AIM/CGE [14], hosted by the National Institute for Environmental Studies, Japan
• GCAM [15], hosted by the Pacific Northwest National Laboratory, USA
• REMIND-MAgPIE [16], hosted by the Potsdam Institute, Germany
• WITCH-GLOBIOM [17], hosted by Fondazione Eni Enrico Mattei, Italy.

These IAMs differ in a number of ways, including in the degree of technological detail to which they describe the global and regional energy systems, the degree of sectoral detail to which they describe the macro-economy of these regions, the availability of mitigation technologies and options, and the method through which they reach a solution for each time period represented. In addition, there are a considerable number of other IAMs which are commonly used in mitigation analysis and which feature in IPCC reports (see [18] for an overview of which models are included).

IAMs representing energy systems can be described as having three principal building blocks:

• An energy demand block, which specifies the energy demand in each sector of the global economy, split into world regions. These sectors are commonly specified as transport, buildings, industrial manufacturing and agriculture. Some IAMs do not represent all of these sectors explicitly, whilst others have even greater sectoral detail—for example separate commercial and residential buildings sectors. The energy demand in each sector in each region in each year can be calculated with respect to underlying socio-economic drivers. For example, transport demand in a given region might be driven by growth in GDP per capita, based on underlying population and GDP projections.
• An energy system block, which allows the model to choose from a wide range of energy technologies and energy vectors (i.e., fuels) in order that the energy demands in the different sectors and regions at different points in time are met. This block represents the costs of the technologies and fuels available in each region’s energy system, as well as the performance (efficiency, availability, lifetime etc.) of the energy technologies. These technologies include energy supply side technologies (e.g., coal, gas, and nuclear power stations, solar PV and wind farms) and in the more technologically detailed models can also include energy demand technologies (e.g., gas boilers, electric heat pumps, petrol and diesel vehicles, hydrogen and electric vehicles, blast furnaces and electric arc furnaces). The energy technologies and fuels used to meet energy demand in each time period represented by the model are associated with greenhouse gas emissions using appropriate emissions factors for each fuel combusted.
• A climate block, which takes the emissions from the energy system in each period, and calculates the resulting temperature change profile over the projection time horizon of the model.

The above description ignores a number of complexities, not least by focusing on the energy system in particular. However, many IAMs also represent the non-energy sectors such as the land use and agricultural sectors, responsible for emitting non-CO₂ gases as well as CO₂ from non-energy sources. IAMs which don’t represent non-energy system emissions can be linked to specific non-CO₂ and CO₂ land-use models. In addition, in many instances IAMs are not run in their full integrated form, but rather as standalone energy system models, which can be given the objective of meeting future energy needs without breaching specified CO₂/greenhouse gas emissions levels from the energy system.

In almost all cases, IAMs are designed to meet specified climate or emissions constraints in the lowest “cost” manner, but the way in which this optimisation is achieved varies between models. Different IAMs have different representations of mitigation costs. Some models measure mitigation costs by comparing the total present value cost of a low-carbon energy system (consistent with meeting given climate targets) over a specified period to the present value cost of a “business-as-usual” energy system over the same period, with no account taken of any knock-on effects to the wider economy. Others are linked to, or embedded in, more detailed macroeconomic models, which more fully account for the changes in prices, and outputs, of different economic sectors, as a result of changing energy costs. These full-scale IAMs—which include the six models that underpinned the IPCC WGIII SSP process as above—are designed to explore the cost-effectiveness of achieving mitigation goals [19].

It should be noted that the term integrated assessment model may also be used to describe a different type of model to those described above. This second type of model has simplified relationships between economic growth and emissions, mitigation levels and costs of mitigation, emissions and temperature change and finally temperature change and monetised costs of climate impacts. Examples of such models are RICE [20], DICE [21], FUND [22] and PAGE [23]. These models allow an examination of the costs of reducing emissions as well as the (often monetised) changes in impacts from reducing temperature change, thereby allowing a cost-benefit, rather than cost-effectiveness, analysis. These reduced-form IAMs—whose results are most commonly reported in IPCC WGII reports—focus on the impacts of climate change [19].

The divide between the two model types is not always clear-cut, with full-scale IAMs such as WITCH, for example, encompassing the ability to undertake cost-benefit analysis through its incorporation of a “damage” function which represents the damages from increased temperature changes, and thus the benefits of reducing temperature changes [17]. A recent comprehensive analysis of the different climate-economy models [24] provides a further set of classifications of models, this time focusing on the different ways in which models represent economic growth and the degree to which they reach some form of economic equilibrium (i.e., where all prices and quantities of different economic goods and services are reached).

Whilst a variety of different classifications is possible, the focus of this study is on the IAMs that have proven most prominent in IPCC assessment reports, as represented by the six models used in SSP and RCP analysis described above, which have relatively detailed representation of the energy and other greenhouse gas emitting systems, and the technologies and measures available to mitigate these emissions.

3. Results

Full-scale IAMs used for evaluation of mitigation pathways have attracted much comment, reflection and criticism in recent years, probably because they are the dominant group of tools for setting out possible long-term mitigation pathways. Much of this criticism has been quite adversarial, with commentators from outside the IAM modelling community querying both the underlying intellectual foundations of IAMs (e.g., [25,26]), that IAMs are not responsive enough to insights from broader research fields such as political or social science (e.g., [27]), and that the transparency and process for building and documenting the models is inadequate (e.g., [28]). The major criticisms are described in the following sub-sections, based on an extensive literature review. Our goal in this section is to add structure and categorisation to the different types of criticisms that the models have received. This allows us to use this categorisation in the remainder of the paper, particularly Section 5, to understand how the proposed actions to address the criticisms would impact on each of these categories in turn. It should be noted that all of the criticisms presented here stem from the existing critical literature.

3.1. Lack of Transparency Around What Drives Model Results

Applications of IAMs for policy-relevant recommendations on long-term mitigation pathways have been accused of lacking transparency in key underlying assumptions such as energy resource costs, constraints on technology take-up, and demand responses to carbon pricing. One critique, from Rosen [29], focuses on one of the major model inter-comparison studies (the AMPERE project) that fed into the IPCC’s Fifth Assessment Report. A key study from AMPERE [30], which forms the particular focus of this critique, assesses the implications (on energy technologies and costs) of a staged accession to a global climate policy regime. Rosen [29] argues that the study does not make clear how different IAM outputs depend on their technology input assumptions and indeed what these assumptions are. The IAM teams’ response [31] addresses this criticism through pointing out that IAMs have been compared and differences classified according to a variety of key diagnostics [32]. In addition, the response points to the considerable documentation of the models used, as well as details of input assumptions [31]. Furthermore, a recent paper includes specific details of some of the power sector technology costs included in the models, along with comparisons to cost projections from organisations such as the International Energy Agency [33].

One way to further aid transparency of IAM mitigation exercises could be to undertake in-depth sensitivity analysis of IAM results to changes in key technology costs, technology performance parameters, fuel costs or other key input assumptions. Although there are studies which look at the implications of uncertainty in technology costs on overall mitigation costs (e.g., [34,35]), it is still, in general, not common practice in IAM exercises to delve deeply into the key drivers of—and differences between—model results in terms of model structure and technology cost, performance, availability or other inputs [36]. This is especially important to avoid “group-think” on reasonable inputs and outputs between major IAM teams [37].

It has been acknowledged that using a variety of IAMs is an important way of addressing the limitations of individual IAMs, with some representing technological detail but not macro-economic dynamics, and others having a complementary set of strengths and weaknesses [36]. Nevertheless, some commentaries (e.g., [38]) note that it is intractable for peer-reviewers to be able to fully test and critique model structures, given the huge complexity of this task. As such, IAMs risk being seen as “black box” models, the inner workings of which are inaccessible, and the results of which are therefore not treated as definitive to policy and other decision makers, given their lack of transparency.

It should be acknowledged that a recent multi-modelling exercise exploring the role of bioenergy in stringent mitigation scenarios makes important inroads into better explaining mitigation scenario drivers and between-model differences [39]. This includes commenting on how the drivers of bioenergy uptake patterns are related to biomass supply, the availability of different bioenergy technology routes (such as bio-electricity and bio-liquids with CCS) and the availability of other low-carbon electricity options in different models. It also includes the provision of key input assumptions such as bioenergy and fossil fuel feedstock prices in its supplementary material.

Nevertheless, given the sometimes considerable range of results that emerge from the models (e.g., the IPCC’s fifth assessment report [7] shows an economic cost of GHG mitigation to 450 ppm of between 3% and 11% of GDP by 2100) the above criticisms suggest that it would be advantageous for a better understanding of between-model differences and within-model sensitivities to be explored. This would help to shed light on which model features and input assumptions are most influential in driving the model outputs and differences.

3.2. Out of Date, Inappropriate or “Unknowable” Input Assumptions Into IAMs

The IPCC Fifth Assessment Report (AR5) [7] received criticism (e.g., [40]) regarding its use of scenarios which begin rapid mitigation from 2010, well into the past given the launch date (2014) of the AR5 report, although given that the IPCC reports review studies already in existence, it is to be expected that at least some mitigation scenarios may use assumptions which have become inappropriate or out of date. In addition, the rapidly changing costs of some technologies, like solar PV in recent years has rendered the input assumptions into some IAMs as somewhat out of date. For example, one paper resulting from the AMPERE programme [41] includes in its supplementary material details of “example” solar PV and storage capital costs, based on a 2008 analysis [42]. This has central PV (i.e., utility-scale PV) costs of $3500/kW in 2020, compared to more recent typical values of closer to $1000/kW [43]. Indeed, a more recent study acknowledges this rather out-of-date treatment of solar PV costs in particular, using updated estimates to demonstrate a far greater role for this technology than previous IAM exercises have tended to show [44].

Rosen and Guenther [25] go further than criticising the use of out-of-date cost projections and argue that, because of the significant transformations in the energy system involved in mitigating dangerous climate change, as well as the long timescales over which mitigation analysis is undertaken, it is essentially unknowable how much it will cost (or benefit) to tackle climate change. They contend that analysis should rather focus on short and medium term actions which minimise costs and maximise benefits.

In addition to this critique, the IPCC fifth assessment report scenario database [45] reveals that there is even a significant variation in base year (often around 2010) data between models. Such variation may reflect genuine uncertainty in knowing base year energy and emissions data [46] which most often do not belong to a single, unambiguous data set. Nevertheless, without a more explicit explanation of the origins of such differences, such variations may serve to harm the credibility of results. If, for example, the starting year for mitigation scenarios has energy demand levels which are significantly lower than outturn data for that year, then future mitigation in such scenarios may look easier/less costly than reality. Whilst it is understandable that different modelling groups use different data sources for calibrating their base year data on energy demand, the wide range still suggests that more work might be needed to understand the implications of differing base year energy demand values, and in particular how this impacts on mitigation effort to achieve desired temperature goals.

3.3. Lack of Clarity on What Constitutes Model Outputs As Opposed to Modeller Inputs

Over recent years several studies have asserted the benefits of enhanced energy efficiency to achieving stringent mitigation goals (e.g., [47,48]) but it is often not clear whether enhanced energy efficiency results from models choosing greater energy efficiency over, say, greater deployment of low-carbon energy supply technologies as part of their solution algorithms. This is because several IAMs have not tended to explicitly represent energy demand technologies (such as lighting, appliances, vehicles, and manufacturing plants), but rather have a greater level of detailed representation of technologies on the energy supply side (refining, heat generation, electricity generation) [49]. Such models are used with highly detailed and well-documented storylines around changes in energy efficiency and fuel use patterns in the energy demand sectors [50]. Nevertheless, it can sometimes be unclear whether a more stringent mitigation scenario of (for example) greater electrification or greater energy efficiency in transport occurred because the model selected these options, or because the model user decided this as part of a pathway storyline.

3.4. Reliance of Mitigation Costs On Baseline Assumptions

There is a broad range of possible futures around which the economy and energy system could develop even without climate change as a concern, which leads to a variety of potential business-as-usual or “no-mitigation” baseline scenarios. Rosen and Guenther [25] assert that measuring mitigation costs against these different scenarios leads to a wide variety of results which makes mitigation cost estimates less useful. The IAM community has explicitly addressed this through its recent publication of the shared socio-economic pathways (SSPs), whose base cases range from very fossil fuel-intensive and relatively energy inefficient (resulting in strong continued emissions growth, as in SSP5) to relatively low-carbon and energy efficient (as in SSP1) [9]. Costs of mitigation are highly dependent on the additional mitigation from the baseline to reach given levels of emissions in line with international targets. However, such a variety of scenarios of the future base cases, whilst a genuine reflection of uncertainty, could in principle also lead to a large array of results in terms of the costs and measures needed to tackle climate change.

Furthermore, Stern [51] asserts that IAMs’ baseline cases do not account for the potential for unchecked climate change to derail economic growth assumptions, a criticism also levelled by Rosen and Guenther [25]. In addition, Stern [51] notes that other damages from business-as-usual base cases, principally from local air pollution, are not costed into these cases. The IAM community and other researchers are currently making efforts, however, to incorporate a wider array of co-benefits and other consequences of mitigation scenarios (as discussed in Section 3.8).

3.5. Inadequate Representation of Innovation

Farmer et al. [19] stress that simplifications of the innovation and technology diffusion process are a major weakness of aggregated tools such as IAMs. Stern [51] further notes that the economic benefits of innovation spillovers from clean innovation (i.e., the increased productivity that can spread to the wider economy as a result of green innovation) are not captured, and also that there could well be net benefits of mitigation, rather than costs, even if avoided climate change-related damages are not accounted for.

Rosen and Guenther [25] note that potentially very significant levels of innovation from low-carbon technology deployment, which are in general not captured by many of the IAMs, is another factor that makes the calculation of mitigation costs intractable. In addition, IAMs have been described as having particular “patterns of model behaviour” [32], including models displaying a tendency to deploy a variety of low-carbon technologies simultaneously rather than see one technology dominate in any given time period [52].

Certain IAMs have a sophisticated representation of innovation deriving from both R&D and deployment-related learning, and some IAM teams are working with innovation scholars to think through the scaling-up dynamics of technology deployment [53] as well as the limiting institutional and financial constraints on new technologies [54]. It is perhaps unfair to expect IAMs to represent such complex processes in full, but where a fuller picture of innovation can be characterised, it would be useful and insightful to see how this affects the technology and economic implications of low-carbon pathways.

3.6. Lack of Representation of Reality of Behavioural and Economic Systems

Many IAMs rely on a least-cost objective, whereby the mitigation actions taken to meet a global target, or (where specified) regional targets, are applied on a least cost basis. This means that—for those models that represent explicitly the take-up of energy efficiency measures on the demand side (such as more efficient cars, heating, lighting and appliances)—these technologies tend to be deployed even in no-climate policy cases, given that they have lower lifecycle costs than less efficient technologies. Specific analyses have been undertaken to test the implications of alternative behaviours which include real-world consumer preference criteria (e.g., in the transport sector [55]), which suggest that the rates of transition in least-cost models may be too rapid. Furthermore, some studies have started to explore specific modal shifts in an IAM framework (e.g., [56]).

Farmer et al. [19] identify the need for the development of new economic modelling methodologies to better represent the real-world complexities of simulating low-carbon transition pathways. Their suggested approaches include the use of Dynamic Stochastic General Equilibrium (DSGE) models, which are used by central bank forecasters, and which introduce a number of real-world features such as uncertainty, economic rigidities and imperfect competition into the modelling framework. They also suggest the use of agent-based models (ABMs) which represent interactions between decision-makers and institutions under set rules. At this stage, however, such agent and complexity approaches are limited to models of energy sub-sectors [57] and there is no established application of these techniques to global IAMs.

An additional feature of most IAMs is that they do not represent some of the significant differences that are likely to drive, or impede, mitigation in different countries and regions, particularly developing regions. For example, less developed countries may face low levels of energy access, a reliance on traditional fuels and informal economies, and poorly functioning markets such as power markets [58,59,60]. These challenges could mean developing regions experience higher costs of mitigation (owing to lower availability of mitigation technologies, or higher implementation costs of those technologies) than specified in some IAMs, where these countries’ energy technology costs can often be assumed to be lower than for developed countries (see e.g., [33]). This can also mean they may be simulated to mitigate much more in “cost-optimal” scenarios than they realistically can do, or than any equity or burden-sharing approach might imply (see e.g., [61]).

3.7. Challenges in Assessing the Real-World Feasibility of Modelled Low-Carbon Pathways

The IPCC AR5 WGIII report chapter on assessing transformation pathways [62] states that IAMs can inform the question of feasibility of mitigation scenarios by providing a range of relevant outputs including technology deployment rates, mitigation costs and energy prices, but that they cannot provide an absolute sense, since different models can achieve some stringent mitigation scenarios whereas others cannot. Riahi et al. [41] discuss such feasibility limits as being reached when a particular model cannot find a solution to a mitigation constraint, as a result of: lack of mitigation options; binding constraints for the diffusion of technologies; extremely high price signals (such as rapid increases in carbon prices). Riahi et al. [41] go on to caution that these feasibility limits concern technical and economic issues, and must be strictly differentiated from the feasibility of a low-carbon transformation in the real world, which also depends on a number of other factors such as political and social concerns.

There have been a number of efforts to assess the degree of feasibility, or challenge, associated with modelled pathways, including the systematic use of key metrics such as costs, carbon prices and required rates of decarbonisation [63]. Other studies compare modelled decarbonisation rates to historical energy transition rates [64,65,66,67], and testing of the feasibility of scenarios with expert energy modellers and transition experts [64]. However, at this stage it remains elusive as to how best to determine and communicate the achievability of modelled pathways. What is clear is that some pathways included in the IPCC’s AR5 report are clearly not feasible without extremely courageous assumptions, since they reach marginal carbon prices of tens or even hundreds of thousands of dollars by the second half of the century [18].

3.8. Interactions of Energy Transition Pathways with Other Factors and Policy Goals

To date there has been only limited integrated analysis on the broader implications of long-term low-carbon pathways in IAMs. However, this is now changing, with some studies (e.g., [68]) reporting on air pollution, water use and other important impacts. In addition, recent analyses (e.g., [69]) have been produced which explicitly examine the potential synergies and conflicts between different low-carbon development pathways and the recently agreed sustainable development goals. Specifically, low carbon pathways have a set of synergies and trade-offs with welfare and well-being, physical and social infrastructures, and sustainable management of the natural environment [70].

Furthermore, there has been criticism of IAMs for failing to account for the links between energy technologies and other industrial and manufacturing parts of the economy and ecosystems services, including full lifecycle assessments of energy technologies [71]. However there have been recent attempts at incorporating lifecycle assessments into IAMs [72].

3.9. Lack of Technology and Energy System Detail at Fine (Geographic and Temporal) Scales

As IAMs operate at a global scale over many decades, lack of computational power has meant that they do not represent energy systems at very fine geographic or temporal scales [73]. A pervasive theme in the assessment of low-carbon pathways is the degree to which high penetrations of intermittent renewables (primarily wind and solar PV) can be compatible with achieving the same level of reliability of electricity supply as is achieved using load-following sources such as coal, oil and gas-fired power generation, which dominate in many regions currently. IAMs tend to lack the ability to represent the electricity (and heat) systems at hourly or half-hourly time slices, so cannot themselves easily calculate the required levels of demand flexibility, storage, interconnection or other flexibility required to accommodate such increased renewables penetration. As such, compromises are often made, such as limiting the maximum penetration of intermittent renewables in specific scenarios (e.g., [41]), or assuming a simple uplift for energy storage costs per unit of wind or solar deployed (e.g., [74]).

Such inadequacies are addressed in specific electricity sector models and the IAM teams themselves are—to varying degrees of sophistication—working towards better representing electricity systems and to incorporate the lessons from the more temporally and geographically detailed models [75].

3.10. Reliance on Negative Emissions Technologies to Reach Stringent Climate Targets

The use of negative emissions technologies (NETs), particularly BECCS, in mitigation scenarios has proven controversial, with some commentaries (e.g., [40]) critical of the reliance of long-term pathways on net negative emissions, and others (e.g., [76]) at least cautious on future projections, given that BECCS is immature or untested amongst the range of technologies deployed in stringent mitigation scenarios. Rogelj et al.’s [11] analysis of 1.5 °C-consistent pathways, using newly-developed scenarios based on a multi-model, multi-SSP comparison, still relies purely on BECCS or afforestation/reduced deforestation to achieve negative emissions.

There are some IAM studies (e.g., [77]) that focus on expanding the set of NETs beyond BECCS—for example direct air capture—but these are still relatively uncommon. Smith et al. [78] review a range of potential negative emissions technologies (NETs), in terms of their costs, availability and wider implications. Such evidence is likely to prove important in expanding the range of negative emissions options available to IAMs when evaluating pathways to very stringent mitigation goals.

This will not deal with the central controversial issue which is the reliance on negative emissions technologies. But the advent of the Paris Agreement, whose 1.5 °C temperature goal requires very little future emissions of CO₂, makes some degree of negative emissions virtually unavoidable [5].

3.11. Summary of IAM Criticisms and Response of the IAM Community to These

Table 1. Summary of criticisms of IAMs and responses to these.

Criticism/Limitation Category	Example of Specific Criticism	IAM Community Response
Lack of transparency	Difficult to see what drives results, both within and between models, owing to lack of availability of underlying assumptions and model structure details.	Much greater availability of data and details on models and inter-model diagnostic tests. [31]
Inappropriate input assumptions	Low share of solar PV in relatively recent mitigation modelling exercises	Implementation of lower PV costs has been presented, with much larger share. [44]
Lack of clarity on model inputs versus outputs	Degree to which radical energy demand reduction in 1.5 °C scenarios is a result of model choice or modeler input	Different future scenarios of socio-economic development (SSPs) have explicit analysis of degree of energy efficiency in baselines [79]
Reliance of mitigation costs on baseline assumptions	Significant differences in baselines can result in significant differences in costs of achieving mitigation against these baselines.	Different future scenarios of socio-economic development (SSPs) have explicit analysis of emissions and fuels in baselines [79]
Inadequate representation of innovation processes	Inadequate representation of innovation in low-carbon technologies, in particular spillovers from one technology’s innovation to others.	Some specific analyses of whole sector innovation rates (e.g., multi-cluster technology learning in transport [80]
Lack of representation of behavioural and economic systems	Significant mitigation through behavior changes such as transport modal shifts is in general not represented	Some studies have explored specific modal shifts in an IAM framework (e.g., [56])
Lack of assessment of real-world feasibility	Limited discussion of feasibility of pathways given full consideration of social, political, economic, technical barriers and drivers.	Explicit acknowledgement of the focus on technical and economic feasibility [11]
Lack of interaction with other policy goals	Lack of consideration of mitigation pathways in light of other policy goals such as energy security, SDGs	Growing number of studies specifically exploring these interactions and trade-offs (e.g., [69])
Lack of representation of fine temporal and geographical scale	Lack of representation of operation of electricity systems considering geographical dispersion of wind, solar resources, and systems balancing with high penetrations of renewables, as un more detailed national electricity sector models.	Incorporation of finer time-slicing (e.g., at hourly level) to represent operation of storage and intermittent renewables in electricity systems [75]
Over-reliance on negative emissions technologies	Unrealistically high levels of (largely untested) negative emissions technologies such as BECCS to reach climate targets	Studies limiting BECCS to explore its importance in low-carbon pathways [39]

summarises the criticisms made of IAMs as highlighted above, as well as detailing examples of where IAMs have been developed in such a way that they directly address the criticism made.

4. Suggestions to Address These Criticisms

As is clear from Table 1, the IAM modelling community has undertaken—and continues to undertake—several efforts to improve their models, so as to address many of the criticisms targeted at them. But the models’ critics have called for two further courses of action: the first, to do away with the models entirely; and the second, to supplement them with a range of other models and analytical techniques. In this section we briefly describe what these different strategies would imply, before more deeply investigating these implications in the following Section, using the specific example of BECCS to achieve negative emissions in the IAMs.

4.1. Scrap the Models

The suggestion that IAMs should be discarded because they are unfit for purpose is not one that has been explicitly made, but one which is implicit in some of the recent criticisms. For example, Rosen [29] and Rosen and Guenther [25] suggest that IAMs can tell us nothing useful about mitigation costs or pathways because of the fundamental impossibility of forecasting technology costs on the multi-decadal timescales with which they are concerned. A clear implication of this is that there is no use for IAMs employed in this analysis, with Rosen [29] instead prescribing a strategy of simply mitigating as rapidly as possible given the need to address climate change.

A somewhat analogous criticism has been made by Pindyck [26], who asserts that lack of agreement over the appropriate rate at which to discount future climate-related damages to the economy, the lack of any theoretical underpinning for the climate-related damage “functions” in the models, a wide range of climate sensitivity and the lack of account of extreme outcomes all make IAMs which compare mitigation costs with climate damage costs to be nothing more than frameworks for highly subjective input assumptions. This argument, therefore, finds that this class of IAMs are of “little or no value” in assessing key metrics such as the damage cost of carbon dioxide emissions (deemed the social cost of carbon, SCC). Pindyck’s [26] suggestion as to what to do instead of using these models is to elicit “rough, subjective” estimates of the probability of climate change to cause a catastrophic impact, as well as a distribution of the size of that impact on the economy. This approach would eschew any use of IAMs, and avoid what Pindyck [26] argues is their false sense of precision.

Whilst Rosen’s and Pindyck’s proposed alternatives are themseleves open to criticism (for example around how to quantify the costs and benefits of different mitigation strategies, and explore sensitivities around these) they have nevertheless been clearly stated as alternatives to IAMs.

4.2. Transform the Models

A further set of commentators have argued for a step change in the design and application of a new generation of IAMs. Stern [51] focuses on key real-world processes such as behavioural changes, the benefits of new infrastructure networks and technology innovation spill-overs. Farmer et al. [19] focus on the scope and complexity of the decision making process in global mitigation pathways and call for the incorporation of agent-based modelling (ABM), which simulates the economy through interactions between a large number of agents on the basis of specified rules. Another fundamental transformation would be in transparency of model code, software and data [73], and in replicability [81] to make IAMs truly accessible and testable, a process far beyond the current IAM community standard practices of inter-model comparisons and diagnostic runs [32].

As implied by Table 1, to some extent this transformation is already underway, but is being undertaken in an evolutionary and incremental approach to existing IAM formulations. The practical challenges in moving from an incremental to a step-change transformation in IAMs are very significant, and include the potential inability of existing IAM structure to be adapted to a new formulation, the relative paucity of IAM modellers with alternate disciplinary backgrounds (especially in the social sciences), and in the available funding and incentives for allocating scarce modeller time in rebuilding and re-documenting complex models [38].

4.3. Supplement the Models with Other Models and Analytical Approaches

A different approach—although related to transforming the models through multiple improvements as discussed above—is to leave the models largely as they are, but to supplement them with other models and approaches.

This could entail bringing insights on low carbon mitigation pathways from very different perspectives, for example social-technical transitions [82]. Efforts to link IAM analysis to the social-technical multi-level perspective have advanced beyond external calls, to co-authored position papers by leading proponents from the respective research communities [83]. Another supplemental approach could include downscaling IAMs to national level energy system modelling to enable the policy, infrastructure and regulatory detail of any low carbon pathways to be fully explored (e.g., [6]). Alternatively, non-model approaches to futures analysis could be better employed including new perspectives on the scenarios [84] used to drive IAM model runs. As argued by Doukas et al. [36], a key element of enabling such supplementary approaches is stakeholder engagement, which includes an understanding of what policy makers actually want from the models, and informing them of what the models can and cannot do, and what other analytical approaches have to offer in this context.

To some extent these supplementary approaches are already used. In the UK for example, the Committee on Climate Change has used a variety of modelling approaches to set national-level carbon budgets. These approaches have included using global IAM low-carbon pathways modelling to understand the level of 2050 global emissions consistent with a 2 °C temperature target, national-level energy system modelling to set out cost-effective pathways to achieve the UK’s share of this 2050 target, supplemented by spatially and temporally detailed electricity dispatch modelling and end-use sectoral marginal abatement cost curve analysis to assess the feasibility of implementation as well as impacts on the macro-economy, fuel poverty, industrial competitiveness and energy security [85].

5. What Would the Implications of Each Suggestion Be? The Case of BECCS

To work through the implications of each of these proposed approaches to addressing the multiple criticisms of IAMs, we examine in-depth what these strategies would imply for the analysis of the potential role of bioenergy with carbon capture and storage (BECCS) in future low-carbon pathways. We choose BECCS because it has become arguably the most critical and controversial element of low-carbon pathways which meet very stringent temperature goals. In combination with increased energy efficiency, it constitutes a critical difference between 1.5 °C and 2 °C scenarios [11,47]. Some critiques of BECCS and NETs have focused on the fact that BECCS is so far uncommercialised, and on the possibility that focusing on BECCS will weaken resolve for near-term emissions reductions [27]. Others have taken a more robust analytical approach to question the real-world scalability and achievability of net negative emissions from BECCS [86]. At the same time, recent commentaries have called for “urgent” discussion and analysis of negative emissions more generally [87].

In order to understand the multiple issues and criticisms around BECCS and how its use in low-carbon pathways has been represented in IAMs, this section summarises each major criticism of IAMs as discussed in Section 3, and hypothesizes whether, and if so how, each of the suggested approaches introduced in Section 4 would lead to a more complete representation of BECCS.

Table 2. Implications of suggested approaches on the transparency of BECCS in low-carbon pathways.

Limitation (as Applied to BECCS)	Implications of “Scrap”	Implications of “Transform”	Implications of “Supplement”
Lack of transparency around what drives model results	Scrapping IAMs would not in itself make assumptions more transparent—indeed, reliance on expert views alone, one suggested alternative, would if anything make the drivers of BECCS take-up more opaque.	Analysis of the drivers of global bioenergy production (e.g., [88]), analysis of the effects of assumptions on model results (e.g., [39]), clarification of cost and potential assumptions (e.g., [89]) would all help clarify the role of BECCS.	Examine biophysical limits to BECCS deployment (e.g., [78]) to constrain IAM models to only realistic and feasible scenarios, as determined off-model.
Out of date, inappropriate or “unknowable” input assumptions into IAMs, e.g., degree of carbon neutrality of bioenergy	Concerns related to the carbon neutrality of bioenergy include indirect land use change (iLUC), local barriers, impossibility of large scale biomass production without environmental impacts [90]. Scrapping IAMs would not in itself address these concerns.	Could include explicit land use and land use change emissions (e.g., [91]) and assess how these emissions could change with increased bioenergy production (i.e., endogenise them into the model), with deep sensitivity analysis to make explicit the sources of uncertainty.	Could undertake detailed additional modelling of non-energy related emissions, e.g., Land Use and Land Use Change (LULUC) emissions in different regions under different socio-economic and political conditions, to complement IAM calculations (e.g., [92]).
Lack of clarity on what constitute model outputs as opposed to modeller inputs	Alternative approaches would face the same challenge of making explicit the inputs and outputs.	Inter-model comparisons would have a greater focus on the implications of different BECCS cost and availability assumptions on its take-up in mitigation scenarios. Recent analysis [39] makes inroads into better documenting bioenergy cost assumptions, though still provides no systematic assessment of how these drive cross-model differences.	Would allow the parametrisation of IAMs with biomass resource models which clearly document assumptions, (e.g., Ruiz Costello et al. [92] determine the bioenergy use pathways based on the biomass availability estimated by the CAPRI model [93]).
Reliance of mitigation costs on economic baseline assumptions	Any approach that sets a hypothetical counterfactual against which to measure use of BECCS faces the issue of which baseline to choose.	Could incorporate externality of air pollution, land use, water use in IAMs to evaluate the economic damage costs of using BECCS and other low-carbon technologies in low-carbon pathways (e.g., [94]).	IAMS could be combined with models which explicitly take into account climate and air pollution damages in baselines to understand the marginal benefit or cost of BECCS (e.g., [95])

and

Table 3. Implications of suggested approaches on the model design and representation of BECCS in low-carbon pathways.

Limitation (as Applied to BECCS)	Implications of “Scrap”	Implications of “Transform”	Implications of “Supplement”
Inadequate representation of innovation processes	Any new approach would struggle to quantify the benefits of innovation in BECCS in a systematic way—instead it would most likely rely on intuitions and mental models of experts.	IAMs could incorporate not just learning by doing but also cross-sectoral innovation spillovers, IAMs could be developed to better represent socio-technical transitions (see [96], for a review of these hybrid approaches).	IAMs could be complemented with models of socio-technical and initiative-based learning, (e.g., [83]). However, innovation for BECCS is still to be investigated [97].
Lack of representation of reality in economic behaviour and systems	Any new approach would still need to compare empirical reality or experience with hypotheses about how BECCS could develop.	IAMs could incorporate behaviour changes and preferences (e.g., with links to food and land use systems) for various socio-demographic groups (e.g., [98]).	Real world complexities of transition to low carbon economies (e.g., economic rigidity, informal markets for biomass) could be studied outside of IAMs (e.g., [99]).
Challenge in assessing the real-world feasibility of modelled low-carbon pathways	Any new approach could be explicitly calibrated to match BECCS deployment rates to historical technology diffusions, but without IAMs there would be less ability to quantify trade-offs between BECCS and other low-carbon technologies.	IAM studies could vary key assumptions of BECCS, such as commercialisation year and maximum potentials (e.g., [39]), and evaluate the results with key metrics, such as carbon prices and technology deployment rate (e.g., [63]).	Could supplement IAMs with expert opinions on real-world barriers and opportunities to try to quantify a realistic level of BECCS (e.g., [100]).
Interactions of energy transition pathways with other factors and policy goals	Any new approach might be able to more explicitly bring into play a multitude of factors, but would most likely lack the ability to quantitatively explore the interactions.	Could develop IAMs to represent other outputs as well as GHG emissions, e.g., air pollutants associated with pathways relying heavily on bioenergy (e.g., [101]).	IAM scenario runs could be complemented with other modelling approaches to assess trade-offs between climate mitigation and other policy goals, such as reducing health impacts caused by air pollution (e.g., [102]).
Lack of technology and energy system detail at fine (geographic and temporal) scales	Any new approach would still face the challenge of specifying local and regional differences between bioenergy and BECCS potential.	Geographic and temporal variability of resource availability would be improved in IAMs, e.g., regional and gridded land use scenarios for the five SSPs [103]; the spatial variability of CO2 storage sites has also been considered [104].	IAMs could be “soft-linked” to other temporal and or/ spatially explicit approaches, e.g., for assessing water availability affecting biomass yields [105].
Reliance on negative emissions to reach stringent climate targets	A non-IAM approach could produce scenarios containing many NETs, but any quantitative trade-offs between NETs and other options would be harder to explore.	More widespread expansion of IAMs’ technology portfolio to include other NETs such as Direct Air Capture (e.g., [77]).	Supplementing IAMs with other analyses of NETs would allow detailed consideration of the implications of using such technologies.

set out the criticisms around IAMs as applied to BECCS in particular—focusing first on model transparency and second on model design and representation of BECCS—and then outline the implications of each of the three proposed approaches.

Table 2 and Table 3 show the multi-level complexity of improving the treatment of BECCS—as a key deep decarbonisation technology in IAM model runs. However, this example does give insights and a cogent argument in whether to scrap, transform or supplement IAMs as the future analytical direction.

Many of the implications related to the first approach (“scrap”) of IAMs are likely to be unsatisfactory in terms of allowing a quantification and sensitivity-testing of the implications of (for example) more expensive or more limited biomass resource, or a slower, more tightly constrained rate of BECCS deployment over time. In some cases alternative approaches like expert elicitations may give a sense of the likely availability of cost-effective BECCS that doesn’t interfere with agriculture, land, water, ecosystems or other resources. However, it is unrealistic to expect the trade-offs between BECCS and these resources to be explored in detail outside of the structured, systematic framework that IAMs provide. For example, Larkin et al.’s [106] assertion that without BECCS we need vastly upscaled rates of decarbonisation makes no attempt to explore the economic trade-offs between this strategy and that of exploiting BECCS at different costs and scales, a trade-off that can be explored in IAMs.

Turning to the next suggested approach (“transform”), to a large extent the transformation of IAMs is already underway, with a vast number of developments and improvements being introduced, in many cases with implications for better understanding the role of BECCS. For example, IAMs are improving in terms of temporal resolution of electricity systems (e.g., [75]) which allows a more detailed representation of load-following BECCS power plants and their costs and benefits compared to intermittent renewables such as solar and wind. In addition, recent exercises (e.g., [72]) have explored the lifecycle emissions of a range of energy technologies including BECCS, to better represent their mitigation potential in low-carbon scenarios. IAMs have also incorporated growth constraints on BECCS as well as other energy supply technologies (e.g., [64,65]), to more fully understand its potential role in energy system transformation pathways. Nevertheless, there remain inherent limitations in the degree to which the models can represent the detailed differences between the cultivation and transformation of biomass resources in different regions, using different crops and different forms of BECCS plants for power, heat and biofuels production. This follows from the huge computational demands that such regional, crop and plant specificity would require, as well as the large number of assumptions required to feed the models. It would also require an in-depth understanding—and representation, of the non-market dynamics governing the cultivation and use of bioenergy and how this might compete with, or co-exist with, other demands for land use. A high degree of abstraction is therefore likely to continue to be required, as well as further issues in the transparency of model inputs and outputs.

The final suggested approach (“supplement”) would see IAMs used as part of a broader suite of approaches to understand the role of BECCS in low-carbon pathways. Such approaches include highly detailed regional and national land use models (building on [103]), expert elicitations and workshops to survey a range of views on the real-world feasibility and challenges of achieving different levels of BECCS [100] and earth system modelling to explore the planetary limits to BECCS [105,107]. This approach seems inherently more tractable than attempting to incorporate all nuances around BECCS into existing IAMs, given the computational and input assumption limitations already outlined. Indeed a recent multi-IAM study on the role of bioenergy [39], which provides a significant advance in terms of making input assumptions more explicit, as well as exploring a range of sensitivities, itself acknowledges that “Increased collaboration with experts from environmental, social, and political disciplines has great potential to improve IAMs” in the area of assessing the role of bioenergy and BECCS in low-carbon pathways. Expecting these disciplines to feed into IAMs directly is ambitious, and at least in the near-term it seems more tractable to accept that these additional and alternative approaches should supplement the use of IAMs, to provide a more complete and nuanced picture of the role of BECCS in low-carbon pathways.

It should be noted that even a set of highly improved, potentially transformed IAMs, or existing IAMS supplemented with a range of other analytical tools, would not necessarily provide more accurate predictions of the future transition of the economy toward a low-carbon end point. This is to fundamentally misunderstand the role of IAMs in most low-carbon pathways analysis. These models are frameworks in which to explore the implications of various assumptions about how future energy-agriculture-land systems may develop, with or without emissions constraints. They are not prediction tools. Furthermore, nor would any of the above approaches to addressing current IAM criticisms allow us to address our essential lack of certainty around how key factors affecting climate change mitigation, such as energy, food and land use, or technology cost, performance and availability, could change in the future. As such, even with the broadest and most robust toolset, we must prepare for several unknowns.

6. Conclusions

In this paper we have reviewed the multiple stated criticisms of IAMs when used for the analysis of low-carbon pathways, both in general and specifically with regard to BECCS, arguably the most critical and controversial technology group incorporated into stringent mitigation scenarios. We reflect on the three primary suggested approaches for addressing the limitations of IAMs—to scrap them, to transform them or to supplement them with other models and analytical approaches—with regard to their implications for better understanding the potential role of BECCS in low-carbon pathways.

We conclude that scrapping IAMs is unrealistic, given their considerable utility as structured frameworks within which a number of assumptions around the costs, performance characteristics and availability of different fuels and technologies can be explored. IAMs have consistently proven to be useful—and highly demanded—tools for producing low-carbon scenarios, and it is not difficult to see why: if IAMs were scrapped and we started from a blank slate, it is inevitable that—to understand the potential pathways to a low-carbon economy—energy and climate change analysts would construct models relating the extraction, transformation, distribution and use of energy resources in the world economy. Such models would be specified to represent the technologies involved in these processes with their associated emissions, and account for potential levels of energy demand in the future. Following demands from policy-makers, businesses and others concerned with the costs of the transition, the models would also be developed to account for the costs of different technology and energy resource combinations, by specifying costs and performance characteristics of technologies and extraction costs of primary energy fuels, as well as the “rules” under which technologies and fuels are selected (i.e., least-cost, welfare-maximising). We would then have reinvented IAMs—and rightly so.

We then explore the merit of transforming IAMs, in order that they can better represent the nuances and complexities of BECCS in low-carbon pathways. We note that this transformation is already underway, with more detailed and explicit representations of aspects such as the lifecycle emissions of bioenergy, the operating characteristics of BECCS power plants, and the potential deployment constraints of BECCS plants being increasingly incorporated into IAMs. Nevertheless, we conclude that the transformation of IAMs is at best an incomplete and limited solution to addressing the limitations of IAMs, given the considerable demands that incorporating huge numbers of temporal, spatial, crop and energy technology-specific assumptions would place on both the models and the modellers themselves. It is also likely to place even more challenges on the finite resources that modellers have to make their tools transparent and open to other analysts and decision makers.

Finally, we consider the option of supplementing IAMs with other analytical approaches, and conclude that this has considerable merit. It obviates the need for adding huge additional complexity to the already-complex IAMs, whilst allowing a broader range of considerations and consequences surrounding BECCS usage to be incorporated into low-carbon pathways analysis. In this way, the full and complex spatial, temporal, sectoral and methodological scope of including BECCS can be addressed through a range of alternative and additional analytical approaches. These should include regional-specific modelling of the potential for, and implications (on critical factors such as land use and agriculture, water and ecosystems) of cultivating biomass and deploying BECCS at scale, as well as expert elicitations and workshops, scenario analyses and other foresight methods which allow the detailed consideration of BECCS.

A critical element—and indeed advantage—of combining alternative approaches with IAMs in this way would be the bringing together of potentially diverse analytical communities from across the physical, engineering and social sciences, so as to achieve a genuinely cross-disciplinary perspective on BECCS and other key low-carbon technology options.